2024-09-15T21:09:26.387 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-15T21:09:26.403 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:09:26.499 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905683 branch: main description: rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/none msgr/async objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-comp-zstd} rados supported-random-distro$/{centos_latest}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7905683' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: mon election default strategy: 1 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true bluestore write v2 random: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: main sha1: 7498493279982dfe87f00616198a5967475f6169 owner: scheduled_teuthology@teuthology priority: 101 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 seed: 2855 sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 sleep_before_teardown: 0 subset: 17936/100000 suite: rados suite_branch: main suite_path: /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 7498493279982dfe87f00616198a5967475f6169 targets: smithi073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGUx6DoFVzHI4AnNYD70kDMB3EGAXXleMSaF8bc70CaGQuT0cM614eo1bby/4JWSrxgMvf7d+1J3OsIEE7OPRE= tasks: - install: null - ceph: log-ignorelist: - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(SLOW_OPS\) - slow request - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - lost_unfound: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_20:00:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-15T21:09:26.499 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa; will attempt to use it 2024-09-15T21:09:26.500 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa/tasks 2024-09-15T21:09:26.500 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-15T21:09:26.501 INFO:teuthology.task.internal:Checking packages... 2024-09-15T21:09:26.523 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'fc3e0bff7468fd55478adbae7f75508b749124b6' 2024-09-15T21:09:26.523 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-15T21:09:26.523 INFO:teuthology.packaging:ref: None 2024-09-15T21:09:26.523 INFO:teuthology.packaging:tag: None 2024-09-15T21:09:26.523 INFO:teuthology.packaging:branch: main 2024-09-15T21:09:26.523 INFO:teuthology.packaging:sha1: fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:09:26.524 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-09-15T21:09:26.749 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4930.gfc3e0bff 2024-09-15T21:09:26.750 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-15T21:09:26.757 INFO:teuthology.task.internal:no buildpackages task found 2024-09-15T21:09:26.757 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-15T21:09:26.770 INFO:teuthology.task.internal:Saving configuration 2024-09-15T21:09:26.781 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-15T21:09:26.793 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-15T21:09:26.816 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905683', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-15 21:04:26.544365', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGUx6DoFVzHI4AnNYD70kDMB3EGAXXleMSaF8bc70CaGQuT0cM614eo1bby/4JWSrxgMvf7d+1J3OsIEE7OPRE='} 2024-09-15T21:09:26.816 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-15T21:09:26.827 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2024-09-15T21:09:26.827 INFO:teuthology.run_tasks:Running task console_log... 2024-09-15T21:09:26.868 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fed7993bac0>, signals=[15]) 2024-09-15T21:09:26.868 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-15T21:09:26.877 INFO:teuthology.task.internal:Opening connections... 2024-09-15T21:09:26.877 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi073.front.sepia.ceph.com 2024-09-15T21:09:26.878 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:09:26.957 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-15T21:09:26.964 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2024-09-15T21:09:27.043 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2024-09-15T21:09:27.043 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-09-15T21:09:27.101 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="CentOS Stream" 2024-09-15T21:09:27.101 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="9" 2024-09-15T21:09:27.101 INFO:teuthology.orchestra.run.smithi073.stdout:ID="centos" 2024-09-15T21:09:27.102 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="rhel fedora" 2024-09-15T21:09:27.102 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="9" 2024-09-15T21:09:27.102 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el9" 2024-09-15T21:09:27.102 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-15T21:09:27.102 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2024-09-15T21:09:27.102 INFO:teuthology.orchestra.run.smithi073.stdout:LOGO="fedora-logo-icon" 2024-09-15T21:09:27.102 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-15T21:09:27.103 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://centos.org/" 2024-09-15T21:09:27.103 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-15T21:09:27.103 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-15T21:09:27.103 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-15T21:09:27.104 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-09-15T21:09:27.127 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-15T21:09:27.139 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-15T21:09:27.148 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-15T21:09:27.149 DEBUG:teuthology.orchestra.run.smithi073:> test '!' -e /home/ubuntu/cephtest 2024-09-15T21:09:27.165 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-15T21:09:27.173 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-15T21:09:27.173 DEBUG:teuthology.orchestra.run.smithi073:> test -z $(ls -A /var/lib/ceph) 2024-09-15T21:09:27.250 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-15T21:09:27.287 INFO:teuthology.run_tasks:Running task kernel... 2024-09-15T21:09:27.302 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-15T21:09:27.302 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-15T21:09:27.303 DEBUG:teuthology.orchestra.run.smithi073:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-15T21:09:27.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:09:27.318 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-09-15T21:09:27.375 INFO:teuthology.orchestra.run.smithi073.stdout:5.14.0-503.el9.x86_64 2024-09-15T21:09:27.375 INFO:teuthology.task.kernel:Running kernel on smithi073: 5.14.0-503.el9.x86_64 2024-09-15T21:09:27.375 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum install -y kernel 2024-09-15T21:09:29.617 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-15T21:09:34.459 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 20 MB 00:03 2024-09-15T21:09:39.585 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.5 MB 00:01 2024-09-15T21:09:41.236 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-15T21:09:42.206 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-15T21:09:47.637 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-09-15T21:09:49.660 INFO:teuthology.orchestra.run.smithi073.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-15T21:09:49.660 INFO:teuthology.orchestra.run.smithi073.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-15T21:09:49.756 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:09:49.766 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:09:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: Package Architecture Version Repository Size 2024-09-15T21:09:49.766 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:09:49.766 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:09:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-15T21:09:49.766 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-15T21:09:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-15T21:09:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-15T21:09:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-15T21:09:49.766 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:09:49.766 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:09:49.766 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:09:49.767 INFO:teuthology.orchestra.run.smithi073.stdout:Install 4 Packages 2024-09-15T21:09:49.767 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:09:49.767 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 79 M 2024-09-15T21:09:49.767 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 126 M 2024-09-15T21:09:49.767 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:09:49.897 INFO:teuthology.orchestra.run.smithi073.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 273 kB/s | 23 kB 00:00 2024-09-15T21:09:52.047 INFO:teuthology.orchestra.run.smithi073.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 7.0 MB/s | 16 MB 00:02 2024-09-15T21:09:53.664 INFO:teuthology.orchestra.run.smithi073.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 7.6 MB/s | 29 MB 00:03 2024-09-15T21:09:55.015 INFO:teuthology.orchestra.run.smithi073.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 6.6 MB/s | 35 MB 00:05 2024-09-15T21:09:55.018 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:09:55.018 INFO:teuthology.orchestra.run.smithi073.stdout:Total 15 MB/s | 79 MB 00:05 2024-09-15T21:09:55.701 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:09:56.049 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:09:56.049 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:09:57.038 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:09:57.038 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:09:59.464 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:10:00.122 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:10:00.280 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:10:03.075 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:10:03.199 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:10:07.998 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:10:08.041 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:10:12.859 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:10:51.166 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:10:51.206 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:10:51.560 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:10:51.560 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-15T21:10:51.560 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-15T21:10:51.560 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-15T21:10:52.449 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-15T21:10:52.449 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:10:52.449 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:10:52.449 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-15T21:10:52.450 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-15T21:10:52.450 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-15T21:10:52.450 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-15T21:10:52.450 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:10:52.450 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:10:52.820 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:10:52.896 INFO:teuthology.orchestra.run.smithi073.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:10:52.896 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:10:52.897 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-15T21:10:52.897 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-15T21:10:52.897 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-15T21:10:52.897 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-15T21:10:52.897 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi073.front.sepia.ceph.com, path=None, version=distro) 2024-09-15T21:10:52.897 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q kernel | sort -rV | head -n 1 2024-09-15T21:10:52.956 INFO:teuthology.orchestra.run.smithi073.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-15T21:10:52.957 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-15T21:10:52.957 DEBUG:teuthology.orchestra.run.smithi073:> sudo rpm -qi grub2-tools 2024-09-15T21:10:52.996 INFO:teuthology.orchestra.run.smithi073.stdout:Name : grub2-tools 2024-09-15T21:10:52.996 INFO:teuthology.orchestra.run.smithi073.stdout:Epoch : 1 2024-09-15T21:10:52.996 INFO:teuthology.orchestra.run.smithi073.stdout:Version : 2.06 2024-09-15T21:10:52.996 INFO:teuthology.orchestra.run.smithi073.stdout:Release : 82.el9 2024-09-15T21:10:52.996 INFO:teuthology.orchestra.run.smithi073.stdout:Architecture: x86_64 2024-09-15T21:10:52.996 INFO:teuthology.orchestra.run.smithi073.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-15T21:10:52.996 INFO:teuthology.orchestra.run.smithi073.stdout:Group : Unspecified 2024-09-15T21:10:52.996 INFO:teuthology.orchestra.run.smithi073.stdout:Size : 8274670 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout:License : GPLv3+ 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout:Packager : builder@centos.org 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout:Vendor : CentOS 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout:Summary : Support tools for GRUB. 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout:Description : 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-15T21:10:52.997 INFO:teuthology.orchestra.run.smithi073.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-15T21:10:52.998 INFO:teuthology.orchestra.run.smithi073.stdout:hardware devices. 2024-09-15T21:10:52.998 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:10:52.998 INFO:teuthology.orchestra.run.smithi073.stdout:This subpackage provides tools for support of all platforms. 2024-09-15T21:10:52.998 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-15T21:10:52.999 INFO:teuthology.task.kernel:Updating grub on smithi073 to boot 5.14.0-505.el9.x86_64 2024-09-15T21:10:52.999 DEBUG:teuthology.orchestra.run.smithi073:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-15T21:10:53.741 INFO:teuthology.orchestra.run.smithi073.stderr:Generating grub configuration file ... 2024-09-15T21:10:55.058 INFO:teuthology.orchestra.run.smithi073.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-15T21:10:55.089 INFO:teuthology.orchestra.run.smithi073.stderr:done 2024-09-15T21:10:55.091 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-09-15T21:10:55.147 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.fVM5Qm1hjY 2024-09-15T21:10:55.147 DEBUG:teuthology.orchestra.run.smithi073:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.fVM5Qm1hjY 2024-09-15T21:10:55.214 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.fVM5Qm1hjY 2024-09-15T21:10:55.360 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.fVM5Qm1hjY is 6KB 2024-09-15T21:10:55.410 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.fVM5Qm1hjY 2024-09-15T21:10:55.425 DEBUG:teuthology.orchestra.run.smithi073:> sudo /bin/ls /boot/loader/entries || true 2024-09-15T21:10:55.488 INFO:teuthology.orchestra.run.smithi073.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-15T21:10:55.488 INFO:teuthology.orchestra.run.smithi073.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-15T21:10:55.488 INFO:teuthology.orchestra.run.smithi073.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-15T21:10:55.488 INFO:teuthology.orchestra.run.smithi073.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-15T21:10:55.490 DEBUG:teuthology.orchestra.run.smithi073:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-15T21:10:55.628 DEBUG:teuthology.orchestra.run.smithi073:> sudo shutdown -r now 2024-09-15T21:11:25.660 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-15T21:11:25.661 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-15T21:11:25.662 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:12:25.722 DEBUG:teuthology.orchestra.remote:timed out 2024-09-15T21:12:34.731 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-15T21:12:34.732 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:12:35.282 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-09-15T21:12:35.698 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-15T21:12:35.698 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-15T21:12:35.698 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-09-15T21:12:35.713 INFO:teuthology.orchestra.run.smithi073.stdout:5.14.0-505.el9.x86_64 2024-09-15T21:12:35.713 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-15T21:12:35.714 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-15T21:12:35.714 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-15T21:12:36.715 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-15T21:12:36.715 DEBUG:teuthology.orchestra.run.smithi073:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-15T21:12:36.857 INFO:teuthology.orchestra.run.smithi073.stdout:ttyS1 2024-09-15T21:12:36.877 DEBUG:teuthology.parallel:result is None 2024-09-15T21:12:36.878 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-15T21:12:36.885 INFO:teuthology.task.internal:Creating test directory... 2024-09-15T21:12:36.885 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-15T21:12:36.899 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-15T21:12:36.906 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-15T21:12:36.913 INFO:teuthology.task.internal:Creating archive directory... 2024-09-15T21:12:36.913 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-15T21:12:36.966 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-15T21:12:36.973 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-15T21:12:36.973 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-15T21:12:37.041 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:12:37.052 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-15T21:12:37.053 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-15T21:12:37.060 INFO:teuthology.task.internal:Configuring sudo... 2024-09-15T21:12:37.060 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-15T21:12:37.119 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-15T21:12:37.128 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-15T21:12:37.128 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-15T21:12:37.175 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:12:37.259 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-15T21:12:37.303 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:12:37.386 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-15T21:12:37.426 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:12:37.426 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-15T21:12:37.492 DEBUG:teuthology.orchestra.run.smithi073:> sudo service rsyslog restart 2024-09-15T21:12:37.608 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:12:37.862 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-15T21:12:37.869 INFO:teuthology.task.internal:Starting timer... 2024-09-15T21:12:37.869 INFO:teuthology.run_tasks:Running task pcp... 2024-09-15T21:12:37.880 INFO:teuthology.run_tasks:Running task selinux... 2024-09-15T21:12:37.915 DEBUG:teuthology.orchestra.run.smithi073:> sudo service auditd rotate 2024-09-15T21:12:37.947 INFO:teuthology.orchestra.run.smithi073.stdout:Rotating logs: 2024-09-15T21:12:37.949 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-15T21:12:37.949 DEBUG:teuthology.orchestra.run.smithi073:> /usr/sbin/getenforce 2024-09-15T21:12:38.012 INFO:teuthology.orchestra.run.smithi073.stdout:Permissive 2024-09-15T21:12:38.013 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi073.front.sepia.ceph.com': 'permissive'} 2024-09-15T21:12:38.013 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T21:12:38.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:12:38.083 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-15T21:12:38.083 DEBUG:teuthology.orchestra.run.smithi073:> sudo /usr/sbin/setenforce permissive 2024-09-15T21:12:38.163 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-15T21:12:38.172 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-15T21:12:38.177 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-15T21:12:38.290 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-15T21:12:38.301 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-15T21:12:38.302 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi073.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-15T21:16:21.365 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi073.front.sepia.ceph.com')] 2024-09-15T21:16:21.366 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-15T21:16:21.367 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-15T21:16:21.444 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-09-15T21:16:21.520 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-15T21:16:21.520 INFO:teuthology.run_tasks:Running task clock... 2024-09-15T21:16:21.535 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-15T21:16:21.535 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-15T21:16:21.536 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:16:21.589 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-15T21:16:21.604 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-15T21:16:21.627 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: ntpd: command not found 2024-09-15T21:16:21.638 INFO:teuthology.orchestra.run.smithi073.stdout:506 Cannot talk to daemon 2024-09-15T21:16:21.652 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-15T21:16:21.666 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-15T21:16:21.726 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:16:21.730 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:16:21.730 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-09-15T21:16:21.730 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:16:21.730 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:16:21.730 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:16:21.730 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:16:21.731 INFO:teuthology.run_tasks:Running task install... 2024-09-15T21:16:21.739 DEBUG:teuthology.task.install:project ceph 2024-09-15T21:16:21.739 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}} 2024-09-15T21:16:21.739 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'} 2024-09-15T21:16:21.739 INFO:teuthology.task.install:Using flavor: default 2024-09-15T21:16:21.744 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-15T21:16:21.744 INFO:teuthology.task.install:extra packages: [] 2024-09-15T21:16:21.745 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6', 'tag': None, 'wait_for_package': False} 2024-09-15T21:16:21.745 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:16:21.889 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/centos/9/flavors/default/ 2024-09-15T21:16:21.889 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4930.gfc3e0bff 2024-09-15T21:16:22.000 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/main/fc3e0bff7468fd55478adbae7f75508b749124b6/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-15T21:16:22.001 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:16:22.001 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-15T21:16:22.028 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-15T21:16:22.028 DEBUG:teuthology.orchestra.run.smithi073:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/fc3e0bff7468fd55478adbae7f75508b749124b6/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-15T21:16:22.094 DEBUG:teuthology.orchestra.run.smithi073:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-15T21:16:22.170 DEBUG:teuthology.orchestra.run.smithi073:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-15T21:16:22.235 INFO:teuthology.orchestra.run.smithi073.stdout:check_obsoletes = 1 2024-09-15T21:16:22.237 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum clean all 2024-09-15T21:16:22.481 INFO:teuthology.orchestra.run.smithi073.stdout:45 files removed 2024-09-15T21:16:22.507 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-radosgw 2024-09-15T21:16:23.066 INFO:teuthology.orchestra.run.smithi073.stdout:ceph packages for x86_64 305 kB/s | 84 kB 00:00 2024-09-15T21:16:23.358 INFO:teuthology.orchestra.run.smithi073.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-15T21:16:23.634 INFO:teuthology.orchestra.run.smithi073.stdout:ceph source packages 8.1 kB/s | 1.9 kB 00:00 2024-09-15T21:16:24.334 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-15T21:16:26.261 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-15T21:16:30.703 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-15T21:16:32.937 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - Extras packages 19 kB/s | 19 kB 00:00 2024-09-15T21:16:34.700 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 14 MB/s | 23 MB 00:01 2024-09-15T21:16:40.364 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-09-15T21:16:42.840 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:16:42.862 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:16:42.862 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-15T21:16:42.862 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:16:42.862 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:16:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 11 M 2024-09-15T21:16:42.863 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrading: 2024-09-15T21:16:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: librados2 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 3.5 M 2024-09-15T21:16:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 3.2 M 2024-09-15T21:16:42.863 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-15T21:16:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-15T21:16:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 5.5 M 2024-09-15T21:16:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 22 M 2024-09-15T21:16:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 25 k 2024-09-15T21:16:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-15T21:16:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-15T21:16:42.863 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-15T21:16:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 755 k 2024-09-15T21:16:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-15T21:16:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-15T21:16:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-15T21:16:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-15T21:16:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 508 k 2024-09-15T21:16:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-15T21:16:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2 x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 5.5 M 2024-09-15T21:16:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-15T21:16:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-15T21:16:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-15T21:16:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 45 k 2024-09-15T21:16:42.864 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 143 k 2024-09-15T21:16:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 161 k 2024-09-15T21:16:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-15T21:16:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 320 k 2024-09-15T21:16:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 305 k 2024-09-15T21:16:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 99 k 2024-09-15T21:16:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-15T21:16:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-15T21:16:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-15T21:16:42.865 INFO:teuthology.orchestra.run.smithi073.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-15T21:16:42.865 INFO:teuthology.orchestra.run.smithi073.stdout:Installing weak dependencies: 2024-09-15T21:16:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-15T21:16:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:16:42.866 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:16:42.866 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:16:42.866 INFO:teuthology.orchestra.run.smithi073.stdout:Install 31 Packages 2024-09-15T21:16:42.866 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrade 2 Packages 2024-09-15T21:16:42.866 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:16:42.866 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 62 M 2024-09-15T21:16:42.867 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:16:43.724 INFO:teuthology.orchestra.run.smithi073.stdout:(1/33): ceph-base-19.3.0-4930.gfc3e0bff.el9.x86 12 MB/s | 5.5 MB 00:00 2024-09-15T21:16:43.774 INFO:teuthology.orchestra.run.smithi073.stdout:(2/33): ceph-selinux-19.3.0-4930.gfc3e0bff.el9. 506 kB/s | 25 kB 00:00 2024-09-15T21:16:43.874 INFO:teuthology.orchestra.run.smithi073.stdout:(3/33): libcephfs2-19.3.0-4930.gfc3e0bff.el9.x8 7.4 MB/s | 755 kB 00:00 2024-09-15T21:16:43.966 INFO:teuthology.orchestra.run.smithi073.stdout:(4/33): libradosstriper1-19.3.0-4930.gfc3e0bff. 5.4 MB/s | 508 kB 00:00 2024-09-15T21:16:44.133 INFO:teuthology.orchestra.run.smithi073.stdout:(5/33): ceph-radosgw-19.3.0-4930.gfc3e0bff.el9. 13 MB/s | 11 MB 00:00 2024-09-15T21:16:44.175 INFO:teuthology.orchestra.run.smithi073.stdout:(6/33): python3-ceph-argparse-19.3.0-4930.gfc3e 1.1 MB/s | 45 kB 00:00 2024-09-15T21:16:44.292 INFO:teuthology.orchestra.run.smithi073.stdout:(7/33): librgw2-19.3.0-4930.gfc3e0bff.el9.x86_6 17 MB/s | 5.5 MB 00:00 2024-09-15T21:16:44.317 INFO:teuthology.orchestra.run.smithi073.stdout:(8/33): python3-ceph-common-19.3.0-4930.gfc3e0b 1.0 MB/s | 143 kB 00:00 2024-09-15T21:16:44.351 INFO:teuthology.orchestra.run.smithi073.stdout:(9/33): python3-cephfs-19.3.0-4930.gfc3e0bff.el 2.7 MB/s | 161 kB 00:00 2024-09-15T21:16:44.376 INFO:teuthology.orchestra.run.smithi073.stdout:(10/33): python3-rados-19.3.0-4930.gfc3e0bff.el 5.3 MB/s | 320 kB 00:00 2024-09-15T21:16:44.401 INFO:teuthology.orchestra.run.smithi073.stdout:(11/33): python3-rbd-19.3.0-4930.gfc3e0bff.el9. 5.9 MB/s | 305 kB 00:00 2024-09-15T21:16:44.427 INFO:teuthology.orchestra.run.smithi073.stdout:(12/33): python3-rgw-19.3.0-4930.gfc3e0bff.el9. 1.9 MB/s | 99 kB 00:00 2024-09-15T21:16:44.627 INFO:teuthology.orchestra.run.smithi073.stdout:(13/33): ceph-common-19.3.0-4930.gfc3e0bff.el9. 16 MB/s | 22 MB 00:01 2024-09-15T21:16:46.011 INFO:teuthology.orchestra.run.smithi073.stdout:(14/33): boost-program-options-1.75.0-8.el9.x86 66 kB/s | 107 kB 00:01 2024-09-15T21:16:46.036 INFO:teuthology.orchestra.run.smithi073.stdout:(15/33): libnbd-1.20.2-2.el9.x86_64.rpm 106 kB/s | 171 kB 00:01 2024-09-15T21:16:46.061 INFO:teuthology.orchestra.run.smithi073.stdout:(16/33): libpmemobj-1.12.1-1.el9.x86_64.rpm 112 kB/s | 160 kB 00:01 2024-09-15T21:16:46.578 INFO:teuthology.orchestra.run.smithi073.stdout:(17/33): librabbitmq-0.11.0-7.el9.x86_64.rpm 80 kB/s | 45 kB 00:00 2024-09-15T21:16:46.904 INFO:teuthology.orchestra.run.smithi073.stdout:(18/33): lttng-ust-2.12.0-6.el9.x86_64.rpm 347 kB/s | 292 kB 00:00 2024-09-15T21:16:47.087 INFO:teuthology.orchestra.run.smithi073.stdout:(19/33): librdkafka-1.6.1-102.el9.x86_64.rpm 630 kB/s | 662 kB 00:01 2024-09-15T21:16:47.146 INFO:teuthology.orchestra.run.smithi073.stdout:(20/33): python3-prettytable-0.7.2-27.el9.noarc 74 kB/s | 42 kB 00:00 2024-09-15T21:16:47.654 INFO:teuthology.orchestra.run.smithi073.stdout:(21/33): qatlib-service-24.02.0-1.el9.x86_64.rp 63 kB/s | 36 kB 00:00 2024-09-15T21:16:47.688 INFO:teuthology.orchestra.run.smithi073.stdout:(22/33): qatlib-24.02.0-1.el9.x86_64.rpm 282 kB/s | 221 kB 00:00 2024-09-15T21:16:47.721 INFO:teuthology.orchestra.run.smithi073.stdout:(23/33): qatzip-libs-1.2.0-1.el9.x86_64.rpm 82 kB/s | 47 kB 00:00 2024-09-15T21:16:47.763 INFO:teuthology.orchestra.run.smithi073.stdout:(24/33): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-15T21:16:47.797 INFO:teuthology.orchestra.run.smithi073.stdout:(25/33): liboath-2.6.7-2.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2024-09-15T21:16:47.822 INFO:teuthology.orchestra.run.smithi073.stdout:(26/33): libarrow-doc-9.0.0-11.el9.noarch.rpm 263 kB/s | 26 kB 00:00 2024-09-15T21:16:47.848 INFO:teuthology.orchestra.run.smithi073.stdout:(27/33): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-15T21:16:47.884 INFO:teuthology.orchestra.run.smithi073.stdout:(28/33): re2-20211101-20.el9.x86_64.rpm 5.3 MB/s | 191 kB 00:00 2024-09-15T21:16:47.926 INFO:teuthology.orchestra.run.smithi073.stdout:(29/33): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.0 MB/s | 839 kB 00:00 2024-09-15T21:16:47.984 INFO:teuthology.orchestra.run.smithi073.stdout:(30/33): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-15T21:16:48.059 INFO:teuthology.orchestra.run.smithi073.stdout:(31/33): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-15T21:16:48.243 INFO:teuthology.orchestra.run.smithi073.stdout:(32/33): librados2-19.3.0-4930.gfc3e0bff.el9.x8 11 MB/s | 3.5 MB 00:00 2024-09-15T21:16:48.302 INFO:teuthology.orchestra.run.smithi073.stdout:(33/33): librbd1-19.3.0-4930.gfc3e0bff.el9.x86_ 10 MB/s | 3.2 MB 00:00 2024-09-15T21:16:48.307 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:16:48.308 INFO:teuthology.orchestra.run.smithi073.stdout:Total 11 MB/s | 62 MB 00:05 2024-09-15T21:16:48.424 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:16:48.460 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:16:48.461 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:16:49.149 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:16:49.150 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:16:50.686 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:16:50.777 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/35 2024-09-15T21:16:50.807 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/35 2024-09-15T21:16:50.832 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/35 2024-09-15T21:16:50.855 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-15T21:16:50.881 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/35 2024-09-15T21:16:51.074 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-15T21:16:51.126 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/35 2024-09-15T21:16:51.488 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/35 2024-09-15T21:16:51.575 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 7/35 2024-09-15T21:16:51.677 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 7/35 2024-09-15T21:16:51.765 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rados-2:19.3.0-4930.gfc3e0bff.el9.x86_64 8/35 2024-09-15T21:16:51.802 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 9/35 2024-09-15T21:16:51.865 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 9/35 2024-09-15T21:16:51.903 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/35 2024-09-15T21:16:51.951 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-argparse-2:19.3.0-4930.gfc3e0bff.el9. 11/35 2024-09-15T21:16:52.024 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cephfs-2:19.3.0-4930.gfc3e0bff.el9.x86_64 12/35 2024-09-15T21:16:52.054 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libradosstriper1-2:19.3.0-4930.gfc3e0bff.el9.x86_6 13/35 2024-09-15T21:16:52.110 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4930.gfc3e0bff.el9.x86_6 13/35 2024-09-15T21:16:52.153 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/35 2024-09-15T21:16:52.263 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/35 2024-09-15T21:16:52.304 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/35 2024-09-15T21:16:52.356 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/35 2024-09-15T21:16:52.385 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/35 2024-09-15T21:16:52.419 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/35 2024-09-15T21:16:52.448 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-15T21:16:52.485 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/35 2024-09-15T21:16:52.805 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 22/35 2024-09-15T21:16:52.853 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 23/35 2024-09-15T21:16:52.945 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 23/35 2024-09-15T21:16:52.979 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 24/35 2024-09-15T21:16:53.027 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 25/35 2024-09-15T21:16:53.518 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 26/35 2024-09-15T21:16:53.547 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librgw2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 27/35 2024-09-15T21:16:53.618 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librgw2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 27/35 2024-09-15T21:16:53.696 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 28/35 2024-09-15T21:16:53.749 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x8 29/35 2024-09-15T21:16:56.060 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 30/35 2024-09-15T21:16:56.104 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 30/35 2024-09-15T21:16:56.669 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 30/35 2024-09-15T21:16:56.707 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 31/35 2024-09-15T21:16:56.781 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 31/35 2024-09-15T21:16:56.781 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-15T21:16:56.781 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-15T21:16:56.781 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:16:56.817 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-selinux-2:19.3.0-4930.gfc3e0bff.el9.x86_64 32/35 2024-09-15T21:17:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4930.gfc3e0bff.el9.x86_64 32/35 2024-09-15T21:17:06.827 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /sys 2024-09-15T21:17:06.827 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /proc 2024-09-15T21:17:06.828 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /mnt 2024-09-15T21:17:06.828 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /var/tmp 2024-09-15T21:17:06.828 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /home 2024-09-15T21:17:06.828 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /root 2024-09-15T21:17:06.828 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /tmp 2024-09-15T21:17:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-radosgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 33/35 2024-09-15T21:17:07.861 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 33/35 2024-09-15T21:17:07.862 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:17:07.862 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-15T21:17:07.862 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:17:07.862 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-15T21:17:07.862 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:07.864 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-15T21:17:07.947 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 34/35 2024-09-15T21:17:07.947 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T21:17:09.180 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T21:17:09.180 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/35 2024-09-15T21:17:09.180 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/35 2024-09-15T21:17:09.180 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-radosgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 3/35 2024-09-15T21:17:09.181 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-selinux-2:19.3.0-4930.gfc3e0bff.el9.x86_64 4/35 2024-09-15T21:17:09.181 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 5/35 2024-09-15T21:17:09.181 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libradosstriper1-2:19.3.0-4930.gfc3e0bff.el9.x86_6 6/35 2024-09-15T21:17:09.181 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librgw2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 7/35 2024-09-15T21:17:09.181 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4930.gfc3e0bff.el9. 8/35 2024-09-15T21:17:09.181 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x8 9/35 2024-09-15T21:17:09.181 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cephfs-2:19.3.0-4930.gfc3e0bff.el9.x86_64 10/35 2024-09-15T21:17:09.181 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rados-2:19.3.0-4930.gfc3e0bff.el9.x86_64 11/35 2024-09-15T21:17:09.181 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 12/35 2024-09-15T21:17:09.182 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 13/35 2024-09-15T21:17:09.183 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/35 2024-09-15T21:17:09.183 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 15/35 2024-09-15T21:17:09.183 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 16/35 2024-09-15T21:17:09.183 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 17/35 2024-09-15T21:17:09.183 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 18/35 2024-09-15T21:17:09.183 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 19/35 2024-09-15T21:17:09.183 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 20/35 2024-09-15T21:17:09.183 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 21/35 2024-09-15T21:17:09.183 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 22/35 2024-09-15T21:17:09.183 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 23/35 2024-09-15T21:17:09.183 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 24/35 2024-09-15T21:17:09.184 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 25/35 2024-09-15T21:17:09.184 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 26/35 2024-09-15T21:17:09.184 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 27/35 2024-09-15T21:17:09.184 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 28/35 2024-09-15T21:17:09.184 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 29/35 2024-09-15T21:17:09.184 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : re2-1:20211101-20.el9.x86_64 30/35 2024-09-15T21:17:09.184 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 31/35 2024-09-15T21:17:09.184 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 32/35 2024-09-15T21:17:09.184 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 33/35 2024-09-15T21:17:09.184 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 34/35 2024-09-15T21:17:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 35/35 2024-09-15T21:17:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:10.113 INFO:teuthology.orchestra.run.smithi073.stdout:Upgraded: 2024-09-15T21:17:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.113 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:17:10.113 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-15T21:17:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-15T21:17:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-15T21:17:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-15T21:17:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-15T21:17:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-15T21:17:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-15T21:17:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-15T21:17:10.114 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-15T21:17:10.115 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-15T21:17:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-15T21:17:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: re2-1:20211101-20.el9.x86_64 2024-09-15T21:17:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-15T21:17:10.116 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:10.116 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:17:10.483 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-test 2024-09-15T21:17:11.048 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:31 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:17:11.183 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:17:11.184 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:11.184 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-15T21:17:11.184 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:11.184 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:17:11.184 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 46 M 2024-09-15T21:17:11.184 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-15T21:17:11.184 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 161 k 2024-09-15T21:17:11.185 INFO:teuthology.orchestra.run.smithi073.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-15T21:17:11.185 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-15T21:17:11.185 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:11.185 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:17:11.185 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:11.185 INFO:teuthology.orchestra.run.smithi073.stdout:Install 4 Packages 2024-09-15T21:17:11.185 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:11.186 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 46 M 2024-09-15T21:17:11.186 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 201 M 2024-09-15T21:17:11.186 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:17:11.378 INFO:teuthology.orchestra.run.smithi073.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.1 MB/s | 304 kB 00:00 2024-09-15T21:17:11.420 INFO:teuthology.orchestra.run.smithi073.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-09-15T21:17:11.462 INFO:teuthology.orchestra.run.smithi073.stdout:(3/4): libcephsqlite-19.3.0-4930.gfc3e0bff.el9. 713 kB/s | 161 kB 00:00 2024-09-15T21:17:12.662 INFO:teuthology.orchestra.run.smithi073.stdout:(4/4): ceph-test-19.3.0-4930.gfc3e0bff.el9.x86_ 32 MB/s | 46 MB 00:01 2024-09-15T21:17:12.664 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:17:12.664 INFO:teuthology.orchestra.run.smithi073.stdout:Total 31 MB/s | 46 MB 00:01 2024-09-15T21:17:12.690 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:17:12.710 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:17:12.710 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:17:13.075 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:17:13.075 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:17:13.588 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:17:13.680 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-15T21:17:13.720 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-15T21:17:13.747 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephsqlite-2:19.3.0-4930.gfc3e0bff.el9.x86_64 3/4 2024-09-15T21:17:18.111 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4930.gfc3e0bff.el9.x86_64 3/4 2024-09-15T21:17:18.195 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-test-2:19.3.0-4930.gfc3e0bff.el9.x86_64 4/4 2024-09-15T21:17:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-test-2:19.3.0-4930.gfc3e0bff.el9.x86_64 4/4 2024-09-15T21:17:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-test-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/4 2024-09-15T21:17:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephsqlite-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/4 2024-09-15T21:17:20.516 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-15T21:17:20.864 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-15T21:17:20.864 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:20.864 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:17:20.864 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:20.864 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:20.864 INFO:teuthology.orchestra.run.smithi073.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-15T21:17:20.864 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-15T21:17:20.864 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:20.864 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:17:21.141 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph 2024-09-15T21:17:21.724 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:41 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:17:21.877 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:17:21.879 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:21.879 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-15T21:17:21.879 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: ceph x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 6.6 k 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 2.4 M 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 1.5 M 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 291 k 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 4.8 M 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 17 M 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-15T21:17:21.880 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-15T21:17:21.881 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-15T21:17:21.882 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-15T21:17:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-15T21:17:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:21.883 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:17:21.883 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:21.883 INFO:teuthology.orchestra.run.smithi073.stdout:Install 37 Packages 2024-09-15T21:17:21.883 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:21.883 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 31 M 2024-09-15T21:17:21.884 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 109 M 2024-09-15T21:17:21.884 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:17:22.792 INFO:teuthology.orchestra.run.smithi073.stdout:(1/37): ceph-19.3.0-4930.gfc3e0bff.el9.x86_64.r 46 kB/s | 6.6 kB 00:00 2024-09-15T21:17:22.968 INFO:teuthology.orchestra.run.smithi073.stdout:(2/37): ceph-mgr-19.3.0-4930.gfc3e0bff.el9.x86_ 4.8 MB/s | 1.5 MB 00:00 2024-09-15T21:17:23.018 INFO:teuthology.orchestra.run.smithi073.stdout:(3/37): ceph-mds-19.3.0-4930.gfc3e0bff.el9.x86_ 6.4 MB/s | 2.4 MB 00:00 2024-09-15T21:17:23.093 INFO:teuthology.orchestra.run.smithi073.stdout:(4/37): ceph-mgr-modules-core-19.3.0-4930.gfc3e 3.8 MB/s | 291 kB 00:00 2024-09-15T21:17:23.244 INFO:teuthology.orchestra.run.smithi073.stdout:(5/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.6 MB/s | 253 kB 00:00 2024-09-15T21:17:23.336 INFO:teuthology.orchestra.run.smithi073.stdout:(6/37): ceph-mon-19.3.0-4930.gfc3e0bff.el9.x86_ 8.8 MB/s | 4.8 MB 00:00 2024-09-15T21:17:23.386 INFO:teuthology.orchestra.run.smithi073.stdout:(7/37): python3-cryptography-36.0.1-4.el9.x86_6 8.7 MB/s | 1.2 MB 00:00 2024-09-15T21:17:23.553 INFO:teuthology.orchestra.run.smithi073.stdout:(8/37): ceph-osd-19.3.0-4930.gfc3e0bff.el9.x86_ 29 MB/s | 17 MB 00:00 2024-09-15T21:17:23.586 INFO:teuthology.orchestra.run.smithi073.stdout:(9/37): python3-pycparser-2.20-6.el9.noarch.rpm 675 kB/s | 135 kB 00:00 2024-09-15T21:17:23.612 INFO:teuthology.orchestra.run.smithi073.stdout:(10/37): python3-ply-3.11-14.el9.noarch.rpm 386 kB/s | 106 kB 00:00 2024-09-15T21:17:23.637 INFO:teuthology.orchestra.run.smithi073.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-15T21:17:23.696 INFO:teuthology.orchestra.run.smithi073.stdout:(12/37): python3-requests-2.25.1-8.el9.noarch.r 878 kB/s | 125 kB 00:00 2024-09-15T21:17:23.987 INFO:teuthology.orchestra.run.smithi073.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 119 kB/s | 35 kB 00:00 2024-09-15T21:17:24.196 INFO:teuthology.orchestra.run.smithi073.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 308 kB/s | 172 kB 00:00 2024-09-15T21:17:24.221 INFO:teuthology.orchestra.run.smithi073.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 309 kB/s | 188 kB 00:00 2024-09-15T21:17:24.272 INFO:teuthology.orchestra.run.smithi073.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 272 kB/s | 77 kB 00:00 2024-09-15T21:17:24.305 INFO:teuthology.orchestra.run.smithi073.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 468 kB/s | 51 kB 00:00 2024-09-15T21:17:24.339 INFO:teuthology.orchestra.run.smithi073.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 355 kB/s | 42 kB 00:00 2024-09-15T21:17:24.364 INFO:teuthology.orchestra.run.smithi073.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 241 kB/s | 22 kB 00:00 2024-09-15T21:17:24.406 INFO:teuthology.orchestra.run.smithi073.stdout:(20/37): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-15T21:17:24.431 INFO:teuthology.orchestra.run.smithi073.stdout:(21/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 481 kB/s | 43 kB 00:00 2024-09-15T21:17:24.457 INFO:teuthology.orchestra.run.smithi073.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 428 kB/s | 11 kB 00:00 2024-09-15T21:17:24.482 INFO:teuthology.orchestra.run.smithi073.stdout:(23/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.5 MB/s | 172 kB 00:00 2024-09-15T21:17:24.516 INFO:teuthology.orchestra.run.smithi073.stdout:(24/37): python3-jaraco-classes-3.2.1-5.el9.noa 301 kB/s | 18 kB 00:00 2024-09-15T21:17:24.541 INFO:teuthology.orchestra.run.smithi073.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 937 kB/s | 23 kB 00:00 2024-09-15T21:17:24.566 INFO:teuthology.orchestra.run.smithi073.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 389 kB/s | 19 kB 00:00 2024-09-15T21:17:24.600 INFO:teuthology.orchestra.run.smithi073.stdout:(27/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.8 MB/s | 358 kB 00:00 2024-09-15T21:17:24.625 INFO:teuthology.orchestra.run.smithi073.stdout:(28/37): python3-jaraco-text-3.2.0-6.el9.noarch 234 kB/s | 20 kB 00:00 2024-09-15T21:17:24.651 INFO:teuthology.orchestra.run.smithi073.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 551 kB/s | 46 kB 00:00 2024-09-15T21:17:24.676 INFO:teuthology.orchestra.run.smithi073.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-15T21:17:24.701 INFO:teuthology.orchestra.run.smithi073.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-15T21:17:24.729 INFO:teuthology.orchestra.run.smithi073.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-15T21:17:24.752 INFO:teuthology.orchestra.run.smithi073.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-15T21:17:24.777 INFO:teuthology.orchestra.run.smithi073.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-15T21:17:24.802 INFO:teuthology.orchestra.run.smithi073.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 805 kB/s | 20 kB 00:00 2024-09-15T21:17:24.853 INFO:teuthology.orchestra.run.smithi073.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-15T21:17:24.886 INFO:teuthology.orchestra.run.smithi073.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.1 MB/s | 427 kB 00:00 2024-09-15T21:17:24.890 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:17:24.890 INFO:teuthology.orchestra.run.smithi073.stdout:Total 10 MB/s | 31 MB 00:03 2024-09-15T21:17:25.043 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:17:25.083 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:17:25.083 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:17:25.579 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:17:25.580 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:17:26.955 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:17:27.022 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-15T21:17:27.059 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-15T21:17:27.100 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-15T21:17:27.179 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-15T21:17:27.249 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-15T21:17:27.282 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-15T21:17:27.373 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-15T21:17:27.452 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-15T21:17:27.495 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-15T21:17:27.529 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-15T21:17:27.568 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-15T21:17:27.610 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-15T21:17:27.652 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-15T21:17:27.709 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-15T21:17:27.744 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-15T21:17:27.787 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-15T21:17:27.844 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-15T21:17:27.876 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-15T21:17:27.949 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-15T21:17:28.089 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-15T21:17:28.167 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-15T21:17:28.220 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-15T21:17:28.267 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-15T21:17:28.333 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-15T21:17:28.413 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-15T21:17:28.600 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-15T21:17:28.655 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-15T21:17:28.741 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-15T21:17:28.838 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-15T21:17:28.887 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-15T21:17:29.016 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-15T21:17:29.189 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4930.gfc3e0bff.el9. 32/37 2024-09-15T21:17:29.215 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 33/37 2024-09-15T21:17:29.241 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 33/37 2024-09-15T21:17:29.241 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:17:29.241 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-15T21:17:29.242 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:17:29.242 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-15T21:17:29.242 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:30.716 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-osd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 34/37 2024-09-15T21:17:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-osd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 34/37 2024-09-15T21:17:30.743 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:17:30.743 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-15T21:17:30.743 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:17:30.743 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-15T21:17:30.743 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:31.216 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mon-2:19.3.0-4930.gfc3e0bff.el9.x86_64 35/37 2024-09-15T21:17:31.240 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mon-2:19.3.0-4930.gfc3e0bff.el9.x86_64 35/37 2024-09-15T21:17:31.240 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:17:31.241 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-15T21:17:31.241 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:17:31.241 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-15T21:17:31.241 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:31.467 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mds-2:19.3.0-4930.gfc3e0bff.el9.x86_64 36/37 2024-09-15T21:17:31.491 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mds-2:19.3.0-4930.gfc3e0bff.el9.x86_64 36/37 2024-09-15T21:17:31.491 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:17:31.491 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-15T21:17:31.491 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:17:31.492 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-15T21:17:31.492 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:31.556 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-2:19.3.0-4930.gfc3e0bff.el9.x86_64 37/37 2024-09-15T21:17:33.467 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-2:19.3.0-4930.gfc3e0bff.el9.x86_64 37/37 2024-09-15T21:17:33.467 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/37 2024-09-15T21:17:33.468 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mds-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/37 2024-09-15T21:17:33.468 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 3/37 2024-09-15T21:17:33.468 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mon-2:19.3.0-4930.gfc3e0bff.el9.x86_64 4/37 2024-09-15T21:17:33.468 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-osd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 5/37 2024-09-15T21:17:33.468 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4930.gfc3e0bff.el9. 6/37 2024-09-15T21:17:33.468 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-15T21:17:33.468 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-15T21:17:33.468 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-15T21:17:33.469 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-15T21:17:33.469 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-15T21:17:33.469 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-15T21:17:33.469 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-15T21:17:33.469 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-15T21:17:33.469 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-15T21:17:33.469 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-15T21:17:33.469 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-15T21:17:33.469 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-15T21:17:33.469 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-15T21:17:33.470 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-15T21:17:33.470 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-15T21:17:33.470 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-15T21:17:33.470 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-15T21:17:33.470 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-15T21:17:33.470 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-15T21:17:33.470 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-15T21:17:33.470 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-15T21:17:33.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-15T21:17:33.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-15T21:17:33.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-15T21:17:33.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-15T21:17:33.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-15T21:17:33.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-15T21:17:33.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-15T21:17:33.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-15T21:17:33.472 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-15T21:17:34.537 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-15T21:17:34.537 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:34.538 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:17:34.538 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:34.538 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:34.538 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:34.538 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:17:34.538 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:34.538 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:34.538 INFO:teuthology.orchestra.run.smithi073.stdout: lua-5.4.4-4.el9.x86_64 2024-09-15T21:17:34.538 INFO:teuthology.orchestra.run.smithi073.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-15T21:17:34.538 INFO:teuthology.orchestra.run.smithi073.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-15T21:17:34.538 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-15T21:17:34.538 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-15T21:17:34.538 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-15T21:17:34.539 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply-3.11-14.el9.noarch 2024-09-15T21:17:34.540 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-15T21:17:34.540 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-15T21:17:34.540 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-15T21:17:34.540 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-15T21:17:34.540 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-15T21:17:34.540 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-15T21:17:34.540 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-15T21:17:34.540 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-15T21:17:34.540 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-15T21:17:34.540 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-15T21:17:34.540 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-15T21:17:34.540 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:34.540 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:17:34.900 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-base 2024-09-15T21:17:35.471 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:55 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:17:35.560 INFO:teuthology.orchestra.run.smithi073.stdout:Package ceph-base-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:17:35.602 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:17:35.603 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-15T21:17:35.604 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:17:35.679 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install cephadm 2024-09-15T21:17:36.249 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:56 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:17:36.383 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:17:36.384 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:36.384 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-15T21:17:36.384 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:36.384 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:17:36.384 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 784 k 2024-09-15T21:17:36.384 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:36.384 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:17:36.384 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:36.384 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-15T21:17:36.384 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:36.384 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 784 k 2024-09-15T21:17:36.384 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 791 k 2024-09-15T21:17:36.385 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:17:36.652 INFO:teuthology.orchestra.run.smithi073.stdout:cephadm-19.3.0-4930.gfc3e0bff.el9.noarch.rpm 2.9 MB/s | 784 kB 00:00 2024-09-15T21:17:36.653 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:17:36.653 INFO:teuthology.orchestra.run.smithi073.stdout:Total 2.9 MB/s | 784 kB 00:00 2024-09-15T21:17:36.653 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:17:36.658 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:17:36.658 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:17:36.664 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:17:36.664 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:17:36.740 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:17:37.181 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:17:37.308 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:17:37.664 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:17:37.955 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:17:37.956 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:37.956 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:17:37.956 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:17:37.956 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:37.956 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:17:38.166 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-immutable-object-cache 2024-09-15T21:17:38.745 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:58 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:17:38.882 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:17:38.883 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:38.883 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repo Size 2024-09-15T21:17:38.883 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:38.883 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:17:38.883 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 145 k 2024-09-15T21:17:38.883 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:38.883 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:17:38.883 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:38.883 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-15T21:17:38.883 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:38.883 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 145 k 2024-09-15T21:17:38.884 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 432 k 2024-09-15T21:17:38.884 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:17:39.094 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-immutable-object-cache-19.3.0-4930.gfc3e0b 690 kB/s | 145 kB 00:00 2024-09-15T21:17:39.094 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:17:39.095 INFO:teuthology.orchestra.run.smithi073.stdout:Total 685 kB/s | 145 kB 00:00 2024-09-15T21:17:39.095 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:17:39.102 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:17:39.103 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:17:39.159 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:17:39.160 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:17:39.299 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:17:39.370 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4930.gfc3e0bff. 1/1 2024-09-15T21:17:39.395 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4930.gfc3e0bff. 1/1 2024-09-15T21:17:39.395 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:17:39.395 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-15T21:17:39.396 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:40.327 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4930.gfc3e0bff. 1/1 2024-09-15T21:17:40.327 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:40.327 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:17:40.327 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:17:40.327 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:40.327 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:17:40.542 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-mgr 2024-09-15T21:17:41.117 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:01 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:17:41.211 INFO:teuthology.orchestra.run.smithi073.stdout:Package ceph-mgr-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:17:41.255 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:17:41.256 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-15T21:17:41.256 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:17:41.317 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-mgr-dashboard 2024-09-15T21:17:41.874 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:01 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:17:42.006 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:17:42.007 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:42.007 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-15T21:17:42.007 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:42.008 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:17:42.008 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 14 M 2024-09-15T21:17:42.008 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-15T21:17:42.008 INFO:teuthology.orchestra.run.smithi073.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-15T21:17:42.008 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 34 k 2024-09-15T21:17:42.008 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 16 k 2024-09-15T21:17:42.008 INFO:teuthology.orchestra.run.smithi073.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-15T21:17:42.008 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-15T21:17:42.008 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-15T21:17:42.008 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-15T21:17:42.008 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-15T21:17:42.008 INFO:teuthology.orchestra.run.smithi073.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-15T21:17:42.008 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-15T21:17:42.009 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-15T21:17:42.009 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:42.009 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:17:42.009 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:17:42.009 INFO:teuthology.orchestra.run.smithi073.stdout:Install 12 Packages 2024-09-15T21:17:42.009 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:42.009 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 19 M 2024-09-15T21:17:42.010 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 159 M 2024-09-15T21:17:42.010 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:17:42.506 INFO:teuthology.orchestra.run.smithi073.stdout:(1/12): ceph-grafana-dashboards-19.3.0-4930.gfc 243 kB/s | 34 kB 00:00 2024-09-15T21:17:42.523 INFO:teuthology.orchestra.run.smithi073.stdout:(2/12): ceph-prometheus-alerts-19.3.0-4930.gfc3 104 kB/s | 16 kB 00:00 2024-09-15T21:17:42.999 INFO:teuthology.orchestra.run.smithi073.stdout:(3/12): ceph-mgr-dashboard-19.3.0-4930.gfc3e0bf 22 MB/s | 14 MB 00:00 2024-09-15T21:17:43.149 INFO:teuthology.orchestra.run.smithi073.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 5.6 MB/s | 863 kB 00:00 2024-09-15T21:17:43.233 INFO:teuthology.orchestra.run.smithi073.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 6.4 MB/s | 548 kB 00:00 2024-09-15T21:17:43.249 INFO:teuthology.orchestra.run.smithi073.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 1.1 MB/s | 19 kB 00:00 2024-09-15T21:17:43.416 INFO:teuthology.orchestra.run.smithi073.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 12 MB/s | 2.0 MB 00:00 2024-09-15T21:17:43.441 INFO:teuthology.orchestra.run.smithi073.stdout:(8/12): python3-protobuf-3.14.0-13.el9.noarch.r 293 kB/s | 269 kB 00:00 2024-09-15T21:17:43.475 INFO:teuthology.orchestra.run.smithi073.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.4 MB/s | 144 kB 00:00 2024-09-15T21:17:43.492 INFO:teuthology.orchestra.run.smithi073.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 611 kB/s | 31 kB 00:00 2024-09-15T21:17:43.517 INFO:teuthology.orchestra.run.smithi073.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 4.6 MB/s | 188 kB 00:00 2024-09-15T21:17:43.568 INFO:teuthology.orchestra.run.smithi073.stdout:(12/12): protobuf-3.14.0-13.el9.x86_64.rpm 974 kB/s | 1.0 MB 00:01 2024-09-15T21:17:43.573 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:17:43.573 INFO:teuthology.orchestra.run.smithi073.stdout:Total 12 MB/s | 19 MB 00:01 2024-09-15T21:17:43.625 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:17:43.645 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:17:43.645 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:17:43.909 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:17:43.909 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:17:44.512 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:17:44.585 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-15T21:17:44.629 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-15T21:17:44.675 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-15T21:17:44.714 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-15T21:17:44.800 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-15T21:17:44.924 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-15T21:17:45.017 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-15T21:17:45.069 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-15T21:17:45.097 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-15T21:17:45.130 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4930.gfc3e0bff.el9 10/12 2024-09-15T21:17:48.290 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4930.gfc3e0bff.el 11/12 2024-09-15T21:17:48.359 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4930.gfc3e0bff.el9.noa 12/12 2024-09-15T21:17:50.381 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4930.gfc3e0bff.el9.noa 12/12 2024-09-15T21:17:50.381 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4930.gfc3e0bff.el 1/12 2024-09-15T21:17:50.381 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4930.gfc3e0bff.el9.noa 2/12 2024-09-15T21:17:50.381 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4930.gfc3e0bff.el9 3/12 2024-09-15T21:17:50.381 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-15T21:17:50.381 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-15T21:17:50.382 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-15T21:17:50.382 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-15T21:17:50.382 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-15T21:17:50.382 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-15T21:17:50.382 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-15T21:17:50.382 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-15T21:17:50.726 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-15T21:17:50.727 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:50.727 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:17:50.727 INFO:teuthology.orchestra.run.smithi073.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-15T21:17:50.727 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:17:50.727 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:17:50.727 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:17:50.727 INFO:teuthology.orchestra.run.smithi073.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-15T21:17:50.727 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-15T21:17:50.727 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-15T21:17:50.727 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-15T21:17:50.727 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-15T21:17:50.727 INFO:teuthology.orchestra.run.smithi073.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-15T21:17:50.727 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-15T21:17:50.728 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-15T21:17:50.728 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:50.728 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:17:51.003 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-15T21:17:51.581 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:11 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:17:51.721 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:17:51.723 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-09-15T21:17:51.723 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-15T21:17:51.723 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-09-15T21:17:51.723 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:17:51.723 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 7.4 M 2024-09-15T21:17:51.723 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-15T21:17:51.723 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-15T21:17:51.723 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-15T21:17:51.723 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-15T21:17:51.723 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-15T21:17:51.723 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-15T21:17:51.723 INFO:teuthology.orchestra.run.smithi073.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-15T21:17:51.724 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-15T21:17:51.724 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-15T21:17:51.724 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-15T21:17:51.724 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-15T21:17:51.724 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-15T21:17:51.724 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:51.724 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:17:51.724 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-09-15T21:17:51.724 INFO:teuthology.orchestra.run.smithi073.stdout:Install 12 Packages 2024-09-15T21:17:51.724 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:17:51.725 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 38 M 2024-09-15T21:17:51.725 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 212 M 2024-09-15T21:17:51.725 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:17:52.025 INFO:teuthology.orchestra.run.smithi073.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-15T21:17:52.067 INFO:teuthology.orchestra.run.smithi073.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.5 MB/s | 801 kB 00:00 2024-09-15T21:17:52.292 INFO:teuthology.orchestra.run.smithi073.stdout:(3/12): ceph-mgr-diskprediction-local-19.3.0-49 17 MB/s | 7.4 MB 00:00 2024-09-15T21:17:52.667 INFO:teuthology.orchestra.run.smithi073.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 51 kB/s | 33 kB 00:00 2024-09-15T21:17:52.826 INFO:teuthology.orchestra.run.smithi073.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 35 kB/s | 18 kB 00:00 2024-09-15T21:17:53.218 INFO:teuthology.orchestra.run.smithi073.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2024-09-15T21:17:53.268 INFO:teuthology.orchestra.run.smithi073.stdout:(7/12): openblas-0.3.26-2.el9.x86_64.rpm 64 kB/s | 38 kB 00:00 2024-09-15T21:17:53.961 INFO:teuthology.orchestra.run.smithi073.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.4 MB/s | 4.9 MB 00:01 2024-09-15T21:17:53.986 INFO:teuthology.orchestra.run.smithi073.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 319 kB/s | 245 kB 00:00 2024-09-15T21:17:54.436 INFO:teuthology.orchestra.run.smithi073.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.4 MB/s | 5.1 MB 00:01 2024-09-15T21:17:54.744 INFO:teuthology.orchestra.run.smithi073.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 320 kB/s | 250 kB 00:00 2024-09-15T21:17:55.469 INFO:teuthology.orchestra.run.smithi073.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-15T21:17:55.471 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:17:55.471 INFO:teuthology.orchestra.run.smithi073.stdout:Total 10 MB/s | 38 MB 00:03 2024-09-15T21:17:55.828 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:17:55.853 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:17:55.853 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:17:56.287 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:17:56.287 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:17:57.047 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:17:57.125 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-15T21:17:57.159 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-15T21:17:57.256 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-15T21:17:57.302 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-15T21:17:57.486 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-15T21:17:57.518 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-15T21:17:57.596 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-15T21:17:58.243 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-15T21:17:58.360 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-15T21:17:59.580 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T21:18:00.613 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-15T21:18:00.642 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4930.gfc3e0 12/12 2024-09-15T21:18:03.204 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4930.gfc3e0 12/12 2024-09-15T21:18:03.204 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4930.gfc3e0 1/12 2024-09-15T21:18:03.204 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-15T21:18:03.204 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-15T21:18:03.205 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-15T21:18:03.205 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-15T21:18:03.205 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-15T21:18:03.205 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-15T21:18:03.205 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-15T21:18:03.205 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-15T21:18:03.205 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-15T21:18:03.205 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-15T21:18:03.636 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-15T21:18:03.636 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:03.636 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:18:03.636 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:18:03.636 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-15T21:18:03.636 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-15T21:18:03.636 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-15T21:18:03.636 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-15T21:18:03.637 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-15T21:18:03.637 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-15T21:18:03.637 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-15T21:18:03.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-15T21:18:03.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-15T21:18:03.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-15T21:18:03.637 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-15T21:18:03.637 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:03.637 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:04.065 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-mgr-rook 2024-09-15T21:18:04.642 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:24 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:04.777 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:04.778 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:04.778 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-15T21:18:04.779 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:04.779 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:18:04.779 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 49 k 2024-09-15T21:18:04.779 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-15T21:18:04.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-15T21:18:04.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-15T21:18:04.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-15T21:18:04.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-15T21:18:04.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-15T21:18:04.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-15T21:18:04.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-15T21:18:04.779 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-15T21:18:04.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-15T21:18:04.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-15T21:18:04.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-15T21:18:04.780 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-15T21:18:04.780 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:04.780 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:18:04.780 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:04.780 INFO:teuthology.orchestra.run.smithi073.stdout:Install 13 Packages 2024-09-15T21:18:04.780 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:04.781 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 2.2 M 2024-09-15T21:18:04.781 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 26 M 2024-09-15T21:18:04.781 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:18:05.284 INFO:teuthology.orchestra.run.smithi073.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 217 kB/s | 19 kB 00:00 2024-09-15T21:18:05.309 INFO:teuthology.orchestra.run.smithi073.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 226 kB/s | 26 kB 00:00 2024-09-15T21:18:05.476 INFO:teuthology.orchestra.run.smithi073.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 956 kB/s | 159 kB 00:00 2024-09-15T21:18:05.502 INFO:teuthology.orchestra.run.smithi073.stdout:(4/13): ceph-mgr-rook-19.3.0-4930.gfc3e0bff.el9 161 kB/s | 49 kB 00:00 2024-09-15T21:18:05.527 INFO:teuthology.orchestra.run.smithi073.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 916 kB/s | 222 kB 00:00 2024-09-15T21:18:05.560 INFO:teuthology.orchestra.run.smithi073.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 915 kB/s | 54 kB 00:00 2024-09-15T21:18:05.586 INFO:teuthology.orchestra.run.smithi073.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 279 kB 00:00 2024-09-15T21:18:05.620 INFO:teuthology.orchestra.run.smithi073.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 349 kB/s | 32 kB 00:00 2024-09-15T21:18:05.653 INFO:teuthology.orchestra.run.smithi073.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 156 kB/s | 14 kB 00:00 2024-09-15T21:18:05.687 INFO:teuthology.orchestra.run.smithi073.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-09-15T21:18:05.720 INFO:teuthology.orchestra.run.smithi073.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.6 MB/s | 90 kB 00:00 2024-09-15T21:18:05.745 INFO:teuthology.orchestra.run.smithi073.stdout:(12/13): python3-google-auth-2.34.0-1.el9.noarc 1.4 MB/s | 220 kB 00:00 2024-09-15T21:18:05.787 INFO:teuthology.orchestra.run.smithi073.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.4 MB/s | 1.0 MB 00:00 2024-09-15T21:18:05.790 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:18:05.790 INFO:teuthology.orchestra.run.smithi073.stdout:Total 2.2 MB/s | 2.2 MB 00:01 2024-09-15T21:18:05.857 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:18:05.872 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:18:05.873 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:18:05.963 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:18:05.965 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:18:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:18:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-15T21:18:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-15T21:18:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-15T21:18:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-15T21:18:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-15T21:18:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-15T21:18:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-15T21:18:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-15T21:18:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-15T21:18:07.863 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-15T21:18:07.897 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-15T21:18:07.944 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-15T21:18:07.967 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-rook-2:19.3.0-4930.gfc3e0bff.el9.noarch 13/13 2024-09-15T21:18:08.761 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4930.gfc3e0bff.el9.noarch 13/13 2024-09-15T21:18:08.762 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/13 2024-09-15T21:18:08.762 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-15T21:18:08.762 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-15T21:18:08.762 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-15T21:18:08.762 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-15T21:18:08.762 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-15T21:18:08.762 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-15T21:18:08.762 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-15T21:18:08.762 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-15T21:18:08.762 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-15T21:18:08.762 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-15T21:18:08.763 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-15T21:18:09.107 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-15T21:18:09.107 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:09.107 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:18:09.107 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:18:09.107 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-15T21:18:09.107 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-15T21:18:09.108 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-15T21:18:09.108 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-15T21:18:09.108 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-15T21:18:09.108 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-15T21:18:09.108 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-15T21:18:09.108 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-15T21:18:09.108 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-15T21:18:09.108 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-15T21:18:09.108 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-15T21:18:09.108 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-15T21:18:09.108 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:09.108 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:09.378 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-mgr-cephadm 2024-09-15T21:18:09.945 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:29 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:10.079 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:10.079 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-15T21:18:10.080 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:10.080 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:18:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 154 k 2024-09-15T21:18:10.080 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-15T21:18:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-15T21:18:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-15T21:18:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-15T21:18:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-15T21:18:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-15T21:18:10.080 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:10.081 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:18:10.081 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:10.081 INFO:teuthology.orchestra.run.smithi073.stdout:Install 6 Packages 2024-09-15T21:18:10.081 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:10.081 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 7.0 M 2024-09-15T21:18:10.081 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 33 M 2024-09-15T21:18:10.081 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:18:10.406 INFO:teuthology.orchestra.run.smithi073.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.7 MB/s | 248 kB 00:00 2024-09-15T21:18:10.489 INFO:teuthology.orchestra.run.smithi073.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 6.5 MB/s | 548 kB 00:00 2024-09-15T21:18:10.506 INFO:teuthology.orchestra.run.smithi073.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 3.4 MB/s | 58 kB 00:00 2024-09-15T21:18:10.532 INFO:teuthology.orchestra.run.smithi073.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-15T21:18:10.632 INFO:teuthology.orchestra.run.smithi073.stdout:(5/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-09-15T21:18:10.765 INFO:teuthology.orchestra.run.smithi073.stdout:(6/6): ceph-mgr-cephadm-19.3.0-4930.gfc3e0bff.e 308 kB/s | 154 kB 00:00 2024-09-15T21:18:10.767 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:18:10.767 INFO:teuthology.orchestra.run.smithi073.stdout:Total 10 MB/s | 7.0 MB 00:00 2024-09-15T21:18:10.837 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:18:10.848 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:18:10.848 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:18:10.946 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:18:10.951 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:18:11.295 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:18:11.465 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-15T21:18:11.510 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-15T21:18:12.030 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-15T21:18:12.121 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-15T21:18:12.197 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-15T21:18:12.221 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 6/6 2024-09-15T21:18:12.954 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 6/6 2024-09-15T21:18:12.955 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/6 2024-09-15T21:18:12.955 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-15T21:18:12.955 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-15T21:18:12.955 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-15T21:18:12.955 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-15T21:18:13.246 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-15T21:18:13.246 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:13.246 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:18:13.246 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:18:13.246 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-15T21:18:13.246 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-15T21:18:13.246 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-15T21:18:13.246 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-15T21:18:13.246 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-15T21:18:13.246 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:13.246 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:13.514 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-fuse 2024-09-15T21:18:14.081 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:34 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:14.216 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:14.217 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-15T21:18:14.217 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:14.217 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:18:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 900 k 2024-09-15T21:18:14.217 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-15T21:18:14.217 INFO:teuthology.orchestra.run.smithi073.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-15T21:18:14.218 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:14.218 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:18:14.218 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:14.218 INFO:teuthology.orchestra.run.smithi073.stdout:Install 2 Packages 2024-09-15T21:18:14.218 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:14.218 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 980 k 2024-09-15T21:18:14.218 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 2.9 M 2024-09-15T21:18:14.218 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:18:14.676 INFO:teuthology.orchestra.run.smithi073.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 390 kB/s | 80 kB 00:00 2024-09-15T21:18:15.010 INFO:teuthology.orchestra.run.smithi073.stdout:(2/2): ceph-fuse-19.3.0-4930.gfc3e0bff.el9.x86_ 1.6 MB/s | 900 kB 00:00 2024-09-15T21:18:15.010 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:18:15.010 INFO:teuthology.orchestra.run.smithi073.stdout:Total 1.2 MB/s | 980 kB 00:00 2024-09-15T21:18:15.023 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:18:15.045 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:18:15.045 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:18:15.126 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:18:15.126 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:18:15.313 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:18:15.454 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-15T21:18:15.535 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/2 2024-09-15T21:18:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2/2 2024-09-15T21:18:16.231 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/2 2024-09-15T21:18:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-15T21:18:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:16.532 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:18:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-15T21:18:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:16.533 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:16.814 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-volume 2024-09-15T21:18:17.400 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:37 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:17.538 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:17.539 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:17.539 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-15T21:18:17.539 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:17.539 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:18:17.539 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume noarch 2:19.3.0-4930.gfc3e0bff.el9 ceph-noarch 285 k 2024-09-15T21:18:17.539 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:17.540 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:18:17.540 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:17.540 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-15T21:18:17.540 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:17.540 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 285 k 2024-09-15T21:18:17.540 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 1.3 M 2024-09-15T21:18:17.540 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:18:17.798 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-volume-19.3.0-4930.gfc3e0bff.el9.noarch.rp 1.1 MB/s | 285 kB 00:00 2024-09-15T21:18:17.798 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:18:17.799 INFO:teuthology.orchestra.run.smithi073.stdout:Total 1.1 MB/s | 285 kB 00:00 2024-09-15T21:18:17.799 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:18:17.807 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:18:17.807 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:18:17.836 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:18:17.837 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:18:18.056 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:18:18.150 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-volume-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:18:18.167 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-volume-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:18:18.167 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:18:18.167 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-15T21:18:18.167 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:19.214 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-volume-2:19.3.0-4930.gfc3e0bff.el9.noarch 1/1 2024-09-15T21:18:19.214 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:19.214 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:18:19.215 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume-2:19.3.0-4930.gfc3e0bff.el9.noarch 2024-09-15T21:18:19.215 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:19.215 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:19.547 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install librados-devel 2024-09-15T21:18:20.133 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:40 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:20.265 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:20.266 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:20.266 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repo Size 2024-09-15T21:18:20.266 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:20.266 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:18:20.266 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 127 k 2024-09-15T21:18:20.266 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:20.266 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:18:20.266 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:20.266 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-15T21:18:20.266 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:20.267 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 127 k 2024-09-15T21:18:20.267 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 456 k 2024-09-15T21:18:20.267 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:18:20.487 INFO:teuthology.orchestra.run.smithi073.stdout:librados-devel-19.3.0-4930.gfc3e0bff.el9.x86_64 575 kB/s | 127 kB 00:00 2024-09-15T21:18:20.487 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:18:20.487 INFO:teuthology.orchestra.run.smithi073.stdout:Total 571 kB/s | 127 kB 00:00 2024-09-15T21:18:20.488 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:18:20.493 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:18:20.493 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:18:20.557 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:18:20.557 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:18:20.713 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:18:20.826 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librados-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:21.261 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:21.539 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:21.539 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:21.539 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:18:21.539 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:18:21.539 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:21.540 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:21.813 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install libcephfs2 2024-09-15T21:18:22.415 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:42 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:22.510 INFO:teuthology.orchestra.run.smithi073.stdout:Package libcephfs2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:18:22.557 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:22.558 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-15T21:18:22.558 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:22.620 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install libcephfs-devel 2024-09-15T21:18:23.212 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:43 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:23.346 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:23.347 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:23.347 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repo Size 2024-09-15T21:18:23.347 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:23.347 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:18:23.347 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 32 k 2024-09-15T21:18:23.347 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:23.347 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:18:23.347 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:23.348 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-15T21:18:23.348 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:23.348 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 32 k 2024-09-15T21:18:23.348 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 142 k 2024-09-15T21:18:23.348 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:18:23.491 INFO:teuthology.orchestra.run.smithi073.stdout:libcephfs-devel-19.3.0-4930.gfc3e0bff.el9.x86_6 222 kB/s | 32 kB 00:00 2024-09-15T21:18:23.492 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:18:23.492 INFO:teuthology.orchestra.run.smithi073.stdout:Total 219 kB/s | 32 kB 00:00 2024-09-15T21:18:23.492 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:18:23.496 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:18:23.496 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:18:23.508 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:18:23.508 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:18:23.602 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:18:23.714 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:24.115 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephfs-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:24.362 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:18:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:18:24.362 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:24.362 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:24.604 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install librados2 2024-09-15T21:18:25.164 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:45 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:25.253 INFO:teuthology.orchestra.run.smithi073.stdout:Package librados2-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:18:25.296 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:25.297 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-15T21:18:25.297 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:25.360 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install librbd1 2024-09-15T21:18:25.923 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:45 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:26.013 INFO:teuthology.orchestra.run.smithi073.stdout:Package librbd1-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:18:26.056 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:26.057 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-15T21:18:26.057 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:26.117 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install python3-rados 2024-09-15T21:18:26.679 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:46 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:26.768 INFO:teuthology.orchestra.run.smithi073.stdout:Package python3-rados-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:18:26.812 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:26.813 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-15T21:18:26.813 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:26.884 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install python3-rgw 2024-09-15T21:18:27.448 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:47 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:27.536 INFO:teuthology.orchestra.run.smithi073.stdout:Package python3-rgw-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:18:27.580 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:27.581 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-15T21:18:27.581 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:27.645 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install python3-cephfs 2024-09-15T21:18:28.214 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:48 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:28.305 INFO:teuthology.orchestra.run.smithi073.stdout:Package python3-cephfs-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:18:28.350 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:28.351 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-15T21:18:28.351 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:28.418 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install python3-rbd 2024-09-15T21:18:28.983 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:48 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:29.073 INFO:teuthology.orchestra.run.smithi073.stdout:Package python3-rbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 is already installed. 2024-09-15T21:18:29.116 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:29.117 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2024-09-15T21:18:29.117 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:29.177 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install rbd-fuse 2024-09-15T21:18:29.745 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:49 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:29.879 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:29.880 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:29.880 INFO:teuthology.orchestra.run.smithi073.stdout: Package Architecture Version Repository Size 2024-09-15T21:18:29.881 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:29.881 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:18:29.881 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 85 k 2024-09-15T21:18:29.881 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:29.881 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:18:29.881 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:29.881 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-15T21:18:29.881 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:29.881 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 85 k 2024-09-15T21:18:29.882 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 231 k 2024-09-15T21:18:29.882 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:18:30.078 INFO:teuthology.orchestra.run.smithi073.stdout:rbd-fuse-19.3.0-4930.gfc3e0bff.el9.x86_64.rpm 433 kB/s | 85 kB 00:00 2024-09-15T21:18:30.078 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:18:30.078 INFO:teuthology.orchestra.run.smithi073.stdout:Total 430 kB/s | 85 kB 00:00 2024-09-15T21:18:30.079 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:18:30.086 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:18:30.086 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:18:30.150 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:18:30.150 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:18:30.313 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:18:30.400 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:30.776 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:31.053 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:31.053 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:31.053 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:18:31.053 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:18:31.053 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:31.053 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:31.302 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install rbd-mirror 2024-09-15T21:18:31.885 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:51 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:32.022 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:32.023 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:32.023 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repo Size 2024-09-15T21:18:32.023 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:32.024 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:18:32.024 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 3.2 M 2024-09-15T21:18:32.024 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:32.024 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:18:32.024 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:32.024 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-15T21:18:32.025 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:32.025 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 3.2 M 2024-09-15T21:18:32.025 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 14 M 2024-09-15T21:18:32.025 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:18:32.365 INFO:teuthology.orchestra.run.smithi073.stdout:rbd-mirror-19.3.0-4930.gfc3e0bff.el9.x86_64.rpm 9.3 MB/s | 3.2 MB 00:00 2024-09-15T21:18:32.365 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:18:32.366 INFO:teuthology.orchestra.run.smithi073.stdout:Total 9.3 MB/s | 3.2 MB 00:00 2024-09-15T21:18:32.366 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:18:32.375 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:18:32.375 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:18:32.458 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:18:32.458 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:18:32.905 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:18:32.974 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-mirror-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:32.999 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:32.999 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-15T21:18:32.999 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-15T21:18:32.999 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:18:32.999 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-15T21:18:32.999 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:34.188 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-mirror-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:34.188 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:34.188 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:18:34.189 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:18:34.189 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:34.189 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:34.440 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install rbd-nbd 2024-09-15T21:18:35.022 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:55 ago on Sun 15 Sep 2024 09:16:40 PM UTC. 2024-09-15T21:18:35.155 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-15T21:18:35.156 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:35.156 INFO:teuthology.orchestra.run.smithi073.stdout: Package Architecture Version Repository Size 2024-09-15T21:18:35.156 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:35.156 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-15T21:18:35.156 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd x86_64 2:19.3.0-4930.gfc3e0bff.el9 ceph 170 k 2024-09-15T21:18:35.156 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:35.156 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-15T21:18:35.157 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-09-15T21:18:35.157 INFO:teuthology.orchestra.run.smithi073.stdout:Install 1 Package 2024-09-15T21:18:35.157 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:35.157 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 170 k 2024-09-15T21:18:35.157 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 490 k 2024-09-15T21:18:35.157 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-15T21:18:35.369 INFO:teuthology.orchestra.run.smithi073.stdout:rbd-nbd-19.3.0-4930.gfc3e0bff.el9.x86_64.rpm 804 kB/s | 170 kB 00:00 2024-09-15T21:18:35.370 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-15T21:18:35.370 INFO:teuthology.orchestra.run.smithi073.stdout:Total 798 kB/s | 170 kB 00:00 2024-09-15T21:18:35.370 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-15T21:18:35.378 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-15T21:18:35.378 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-15T21:18:35.443 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-15T21:18:35.443 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-15T21:18:35.597 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-15T21:18:35.718 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-nbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:36.133 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-nbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:36.428 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-nbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 1/1 2024-09-15T21:18:36.429 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:36.429 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-15T21:18:36.429 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd-2:19.3.0-4930.gfc3e0bff.el9.x86_64 2024-09-15T21:18:36.429 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:36.429 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-15T21:18:36.690 DEBUG:teuthology.parallel:result is None 2024-09-15T21:18:36.691 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=fc3e0bff7468fd55478adbae7f75508b749124b6 2024-09-15T21:18:36.834 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-15T21:18:36.857 INFO:teuthology.orchestra.run.smithi073.stdout:19.3.0-4930.gfc3e0bff.el9 2024-09-15T21:18:36.858 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4930.gfc3e0bff.el9 2024-09-15T21:18:36.858 INFO:teuthology.task.install:The correct ceph version 19.3.0-4930.gfc3e0bff is installed. 2024-09-15T21:18:36.860 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-15T21:18:36.860 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:36.860 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-15T21:18:36.924 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-15T21:18:36.924 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:36.924 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/daemon-helper 2024-09-15T21:18:36.990 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-15T21:18:37.055 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-15T21:18:37.055 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:37.055 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-15T21:18:37.120 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-15T21:18:37.184 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-15T21:18:37.185 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:37.185 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/stdin-killer 2024-09-15T21:18:37.249 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-15T21:18:37.314 INFO:teuthology.run_tasks:Running task ceph... 2024-09-15T21:18:37.405 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-15T21:18:37.406 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 777 /var/log/ceph 2024-09-15T21:18:37.430 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-15T21:18:37.430 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-15T21:18:37.511 INFO:tasks.ceph:Creating extra log directories... 2024-09-15T21:18:37.512 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-15T21:18:37.583 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-15T21:18:37.583 INFO:tasks.ceph:config {'conf': {'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-15T21:18:37.583 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905683', 'branch': 'main', 'description': 'rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/none msgr/async objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-comp-zstd} rados supported-random-distro$/{centos_latest}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7905683', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'main', 'sha1': '7498493279982dfe87f00616198a5967475f6169'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 101, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2']], 'seed': 2855, 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6', 'sleep_before_teardown': 0, 'subset': '17936/100000', 'suite': 'rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_7498493279982dfe87f00616198a5967475f6169/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '7498493279982dfe87f00616198a5967475f6169', 'targets': {'smithi073.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIGUx6DoFVzHI4AnNYD70kDMB3EGAXXleMSaF8bc70CaGQuT0cM614eo1bby/4JWSrxgMvf7d+1J3OsIEE7OPRE='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'conf': {'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'sha1': 'fc3e0bff7468fd55478adbae7f75508b749124b6', 'cluster': 'ceph'}}, {'lost_unfound': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-15_20:00:16', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-15T21:18:37.583 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-15T21:18:37.631 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/run/ceph 2024-09-15T21:18:37.697 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:37.697 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/scratch_devs of=/dev/stdout 2024-09-15T21:18:37.754 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-15T21:18:37.754 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_1 2024-09-15T21:18:37.809 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-15T21:18:37.809 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:18:37.809 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-15T21:18:37.810 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:18:37.810 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:18:37.810 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-15 21:18:33.262479881 +0000 2024-09-15T21:18:37.810 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-15 21:16:02.330521615 +0000 2024-09-15T21:18:37.810 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-15 21:16:02.330521615 +0000 2024-09-15T21:18:37.810 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-09-15 21:16:02.330521615 +0000 2024-09-15T21:18:37.810 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-15T21:18:37.875 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-15T21:18:37.875 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-15T21:18:37.875 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000203037 s, 2.5 MB/s 2024-09-15T21:18:37.877 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-15T21:18:37.933 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_2 2024-09-15T21:18:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-15T21:18:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:18:37.990 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-15T21:18:37.990 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:18:37.990 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:18:37.990 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-15 21:18:33.262479881 +0000 2024-09-15T21:18:37.990 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-15 21:16:02.670513607 +0000 2024-09-15T21:18:37.990 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-15 21:16:02.670513607 +0000 2024-09-15T21:18:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-09-15 21:16:02.670513607 +0000 2024-09-15T21:18:37.991 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-15T21:18:38.054 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-15T21:18:38.054 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-15T21:18:38.054 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000208555 s, 2.5 MB/s 2024-09-15T21:18:38.056 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-15T21:18:38.112 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_3 2024-09-15T21:18:38.168 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-15T21:18:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:18:38.169 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-15T21:18:38.169 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:18:38.169 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:18:38.169 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-15 21:18:33.263479857 +0000 2024-09-15T21:18:38.169 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-15 21:16:03.004505740 +0000 2024-09-15T21:18:38.169 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-15 21:16:03.004505740 +0000 2024-09-15T21:18:38.169 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-09-15 21:16:03.004505740 +0000 2024-09-15T21:18:38.169 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-15T21:18:38.232 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-15T21:18:38.232 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-15T21:18:38.232 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.00019753 s, 2.6 MB/s 2024-09-15T21:18:38.233 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-15T21:18:38.289 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_4 2024-09-15T21:18:38.344 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-15T21:18:38.344 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-15T21:18:38.344 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-15T21:18:38.344 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-15T21:18:38.344 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-09-15T21:18:38.344 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-15 21:18:33.263479857 +0000 2024-09-15T21:18:38.344 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-15 21:16:03.344497732 +0000 2024-09-15T21:18:38.345 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-15 21:16:03.344497732 +0000 2024-09-15T21:18:38.345 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-09-15 21:16:03.344497732 +0000 2024-09-15T21:18:38.345 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-15T21:18:38.407 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-15T21:18:38.407 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-15T21:18:38.407 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.00020759 s, 2.5 MB/s 2024-09-15T21:18:38.408 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-15T21:18:38.465 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T21:18:38.465 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-15T21:18:38.465 INFO:tasks.ceph:Generating config... 2024-09-15T21:18:38.466 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-15T21:18:38.466 INFO:tasks.ceph:[global] ms type = async 2024-09-15T21:18:38.466 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-15T21:18:38.467 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-15T21:18:38.467 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-15T21:18:38.467 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-15T21:18:38.467 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-15T21:18:38.467 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-15T21:18:38.467 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-15T21:18:38.467 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-09-15T21:18:38.467 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-15T21:18:38.467 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-15T21:18:38.467 INFO:tasks.ceph:[osd] bluestore write v2 random = True 2024-09-15T21:18:38.467 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-15T21:18:38.467 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-15T21:18:38.468 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-15T21:18:38.469 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-15T21:18:38.469 INFO:tasks.ceph:Setting up mon.a... 2024-09-15T21:18:38.469 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-15T21:18:38.557 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.keyring 2024-09-15T21:18:38.560 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-15T21:18:38.602 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:18:38.667 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.73'), ('mon.b', '[v2:172.21.15.73:3301,v1:172.21.15.73:6790]'), ('mon.c', '[v2:172.21.15.73:3302,v1:172.21.15.73:6791]')] 2024-09-15T21:18:38.667 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790],[v2:172.21.15.73:3302,v1:172.21.15.73:6791]', 'mon election default strategy': 1, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore write v2 random': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-15T21:18:38.668 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:38.668 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T21:18:38.722 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.73 --addv b '[v2:172.21.15.73:3301,v1:172.21.15.73:6790]' --addv c '[v2:172.21.15.73:3302,v1:172.21.15.73:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:18:38.809 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:18:38.809 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: generated fsid 1d60fae4-b8c3-4364-9331-b0445493a043 2024-09-15T21:18:38.809 INFO:teuthology.orchestra.run.smithi073.stdout:setting min_mon_release = quincy 2024-09-15T21:18:38.809 INFO:teuthology.orchestra.run.smithi073.stdout:epoch 0 2024-09-15T21:18:38.809 INFO:teuthology.orchestra.run.smithi073.stdout:fsid 1d60fae4-b8c3-4364-9331-b0445493a043 2024-09-15T21:18:38.809 INFO:teuthology.orchestra.run.smithi073.stdout:last_changed 2024-09-15T21:18:38.806621+0000 2024-09-15T21:18:38.809 INFO:teuthology.orchestra.run.smithi073.stdout:created 2024-09-15T21:18:38.806621+0000 2024-09-15T21:18:38.809 INFO:teuthology.orchestra.run.smithi073.stdout:min_mon_release 17 (quincy) 2024-09-15T21:18:38.809 INFO:teuthology.orchestra.run.smithi073.stdout:election_strategy: 1 2024-09-15T21:18:38.809 INFO:teuthology.orchestra.run.smithi073.stdout:0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-09-15T21:18:38.809 INFO:teuthology.orchestra.run.smithi073.stdout:1: [v2:172.21.15.73:3301/0,v1:172.21.15.73:6790/0] mon.b 2024-09-15T21:18:38.810 INFO:teuthology.orchestra.run.smithi073.stdout:2: [v2:172.21.15.73:3302/0,v1:172.21.15.73:6791/0] mon.c 2024-09-15T21:18:38.810 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-15T21:18:38.811 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-15T21:18:38.826 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1d60fae4-b8c3-4364-9331-b0445493a043... 2024-09-15T21:18:38.827 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-15T21:18:38.909 INFO:teuthology.orchestra.run.smithi073.stdout:[global] 2024-09-15T21:18:38.909 INFO:teuthology.orchestra.run.smithi073.stdout: chdir = "" 2024-09-15T21:18:38.909 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-15T21:18:38.909 INFO:teuthology.orchestra.run.smithi073.stdout: auth supported = cephx 2024-09-15T21:18:38.909 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.909 INFO:teuthology.orchestra.run.smithi073.stdout: filestore xattr use omap = true 2024-09-15T21:18:38.909 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.909 INFO:teuthology.orchestra.run.smithi073.stdout: mon clock drift allowed = 1.000 2024-09-15T21:18:38.909 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.909 INFO:teuthology.orchestra.run.smithi073.stdout: osd crush chooseleaf type = 0 2024-09-15T21:18:38.909 INFO:teuthology.orchestra.run.smithi073.stdout: auth debug = true 2024-09-15T21:18:38.910 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.910 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on old message = true 2024-09-15T21:18:38.910 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on bug = true 2024-09-15T21:18:38.910 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.910 INFO:teuthology.orchestra.run.smithi073.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-15T21:18:38.910 INFO:teuthology.orchestra.run.smithi073.stdout: mon pg warn max object skew = 0 2024-09-15T21:18:38.910 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.911 INFO:teuthology.orchestra.run.smithi073.stdout: # disable pg_autoscaler by default for new pools 2024-09-15T21:18:38.911 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default size = 2 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow primary affinity = true 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow pg remap = true 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on legacy crush tunables = false 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on crush straw calc version zero = false 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on no sortbitwise = false 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on osd down out interval zero = false 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on too few osds = false 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: mon_allow_pool_size_one = true 2024-09-15T21:18:38.912 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: osd default data pool replay window = 5 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: mon allow pool delete = true 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: mon cluster log file level = debug 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: debug asserts on shutdown = true 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: mon health detail to clog = false 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: mon host = "172.21.15.73,[v2:172.21.15.73:3301,v1:172.21.15.73:6790],[v2:172.21.15.73:3302,v1:172.21.15.73:6791]" 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: mon election default strategy = 1 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: ms type = async 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: fsid = 1d60fae4-b8c3-4364-9331-b0445493a043 2024-09-15T21:18:38.913 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout:[osd] 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: osd journal size = 100 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub load threshold = 5.0 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub max interval = 600 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock profile = high_recovery_ops 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: osd recover clone overlap = true 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: osd recovery max chunk = 1048576 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug shutdown = true 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug op order = true 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify stray on activate = true 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug trim objects = true 2024-09-15T21:18:38.914 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: osd open classes on start = true 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug pg log writeout = true 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: osd deep scrub update digest min age = 30 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: osd map max advance = 10 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: journal zero on create = true 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: filestore ondisk finisher threads = 3 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: filestore apply finisher threads = 3 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: bdev debug aio = true 2024-09-15T21:18:38.915 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug misdirected ops = true 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore block size = 96636764160 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore compression algorithm = zstd 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore compression mode = aggressive 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore fsck on mount = True 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore write v2 random = True 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore zero block detection = True 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluefs = 20 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluestore = 20 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: debug osd = 20 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: debug rocksdb = 10 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd backfillfull_ratio = 0.85 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd full ratio = 0.9 2024-09-15T21:18:38.916 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd nearfull ratio = 0.8 2024-09-15T21:18:38.917 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify cached snaps = True 2024-09-15T21:18:38.917 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify missing on start = True 2024-09-15T21:18:38.917 INFO:teuthology.orchestra.run.smithi073.stdout: osd failsafe full ratio = 0.95 2024-09-15T21:18:38.917 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock override recovery settings = True 2024-09-15T21:18:38.917 INFO:teuthology.orchestra.run.smithi073.stdout: osd objectstore = bluestore 2024-09-15T21:18:38.917 INFO:teuthology.orchestra.run.smithi073.stdout: osd op queue = debug_random 2024-09-15T21:18:38.917 INFO:teuthology.orchestra.run.smithi073.stdout: osd op queue cut off = debug_random 2024-09-15T21:18:38.917 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.917 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr] 2024-09-15T21:18:38.917 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-09-15T21:18:38.917 INFO:teuthology.orchestra.run.smithi073.stdout: debug mgr = 20 2024-09-15T21:18:38.917 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-09-15T21:18:38.917 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: mgr/telemetry/nag = false 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout:[mon] 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: debug paxos = 20 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: mon data avail warn = 5 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: mon mgr mkfs grace = 240 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd reporter subtree level = osd 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd prime pg temp = true 2024-09-15T21:18:38.918 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: auth mon ticket ttl = 660 # 11m 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: auth service ticket ttl = 240 # 4m 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: # don't complain about insecure global_id in the test suite 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: # 1m isn't quite enough 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: mon_down_mkfs_grace = 2m 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_filestore_osds = false 2024-09-15T21:18:38.919 INFO:teuthology.orchestra.run.smithi073.stdout: mon scrub interval = 300 2024-09-15T21:18:38.920 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:18:38.920 INFO:teuthology.orchestra.run.smithi073.stdout:[client] 2024-09-15T21:18:38.920 INFO:teuthology.orchestra.run.smithi073.stdout: rgw cache enabled = true 2024-09-15T21:18:38.920 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable ops log = true 2024-09-15T21:18:38.920 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable usage log = true 2024-09-15T21:18:38.920 INFO:teuthology.orchestra.run.smithi073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-15T21:18:38.920 INFO:teuthology.orchestra.run.smithi073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-15T21:18:38.920 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.a] 2024-09-15T21:18:38.920 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.b] 2024-09-15T21:18:38.920 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.c] 2024-09-15T21:18:38.921 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-15T21:18:38.921 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-15T21:18:39.016 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-15T21:18:39.017 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:39.017 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-15T21:18:39.032 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:39.032 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-15T21:18:39.087 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi073.front.sepia.ceph.com 2024-09-15T21:18:39.087 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:39.088 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-15T21:18:39.088 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-15T21:18:39.162 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:39.162 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:18:39.217 INFO:tasks.ceph:Setting up mon nodes... 2024-09-15T21:18:39.217 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-15T21:18:39.218 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T21:18:39.309 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-15T21:18:39.311 INFO:tasks.ceph:Setting up mds nodes... 2024-09-15T21:18:39.311 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-15T21:18:39.311 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-15T21:18:39.312 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-15T21:18:39.312 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-15T21:18:39.337 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T21:18:39.337 INFO:tasks.ceph:role: osd.0 2024-09-15T21:18:39.337 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-15T21:18:39.337 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-15T21:18:39.409 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:18:39.409 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:18:39.409 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:18:39.409 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:18:39.410 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:18:39.410 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:18:39.410 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:18:39.410 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:18:39.410 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:18:39.410 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:18:39.912 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-09-15T21:18:39.914 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-09-15T21:18:39.914 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-15T21:18:40.060 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-15T21:18:40.091 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-15T21:18:40.156 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T21:18:40.156 INFO:tasks.ceph:role: osd.1 2024-09-15T21:18:40.156 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-15T21:18:40.157 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-15T21:18:40.229 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:18:40.229 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:18:40.229 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:18:40.229 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:18:40.229 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:18:40.230 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:18:40.230 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:18:40.230 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:18:40.230 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:18:40.230 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:18:40.682 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-09-15T21:18:40.685 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-09-15T21:18:40.685 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-15T21:18:40.775 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-15T21:18:40.804 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-15T21:18:40.868 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-15T21:18:40.868 INFO:tasks.ceph:role: osd.2 2024-09-15T21:18:40.868 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-15T21:18:40.868 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-15T21:18:40.940 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-15T21:18:40.941 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-15T21:18:40.941 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-15T21:18:40.941 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-15T21:18:40.941 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-15T21:18:40.941 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-09-15T21:18:40.941 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-15T21:18:40.941 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-15T21:18:40.941 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-15T21:18:40.941 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-15T21:18:41.429 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-09-15T21:18:41.431 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-09-15T21:18:41.431 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-15T21:18:41.577 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-15T21:18:41.607 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:18:41.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-15T21:18:41.700+0000 7ff065a12640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-15T21:18:41.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-15T21:18:41.704+0000 7ff065a12640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-15T21:18:41.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-15T21:18:41.705+0000 7ff065a12640 -1 bdev(0x557c57cbb800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-15T21:18:41.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-15T21:18:41.705+0000 7ff065a12640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-15T21:18:44.610 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-15T21:18:44.640 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:18:44.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-15T21:18:44.732+0000 7f86c060d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-15T21:18:44.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-15T21:18:44.732+0000 7f86c060d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-15T21:18:44.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-15T21:18:44.732+0000 7f86c060d640 -1 bdev(0x55b1f79c7800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-15T21:18:44.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-15T21:18:44.732+0000 7f86c060d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-15T21:18:47.894 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-15T21:18:47.922 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:18:48.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-15T21:18:48.007+0000 7f4d78015640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-15T21:18:48.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-15T21:18:48.008+0000 7f4d78015640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-15T21:18:48.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-15T21:18:48.008+0000 7f4d78015640 -1 bdev(0x558644aa7800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-15T21:18:48.011 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-15T21:18:48.008+0000 7f4d78015640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-15T21:18:50.931 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-15T21:18:50.959 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-15T21:18:50.959 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:50.960 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-15T21:18:51.025 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:51.026 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-15T21:18:51.092 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:51.093 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-15T21:18:51.157 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:18:51.157 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-15T21:18:51.224 INFO:tasks.ceph:Adding keys to all mons... 2024-09-15T21:18:51.225 DEBUG:teuthology.orchestra.run.smithi073:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-15T21:18:51.288 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.x] 2024-09-15T21:18:51.288 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAvT+dmt05KEhAAo4xxdaSvZaIFnELck+OmmQ== 2024-09-15T21:18:51.288 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.0] 2024-09-15T21:18:51.288 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAxT+dmH3LNKRAAepGtTZK7YaqUngXvQ1mCkg== 2024-09-15T21:18:51.288 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.1] 2024-09-15T21:18:51.288 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA0T+dmure1KxAAn+DU+yK7MEbTKTCc3zbJww== 2024-09-15T21:18:51.288 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.2] 2024-09-15T21:18:51.288 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA4T+dmAs+HABAADEnQjRTDtRX+4dfbdZ0wDQ== 2024-09-15T21:18:51.289 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-15T21:18:51.380 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:18:51.432 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:18:51.484 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-15T21:18:51.536 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-15T21:18:51.536 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-15T21:18:51.561 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:18:51.657 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-15T21:18:51.681 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-15T21:18:51.747 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:18:51.844 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-15T21:18:51.868 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-15T21:18:51.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-15T21:18:52.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-15T21:18:52.055 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-15T21:18:52.110 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-15T21:18:52.110 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-15T21:18:52.110 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-15T21:18:52.152 INFO:tasks.ceph.mon.a:Started 2024-09-15T21:18:52.153 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-15T21:18:52.153 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-15T21:18:52.155 INFO:tasks.ceph.mon.b:Started 2024-09-15T21:18:52.155 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-15T21:18:52.155 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-15T21:18:52.157 INFO:tasks.ceph.mon.c:Started 2024-09-15T21:18:52.158 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph config set mgr mgr_pool false --force 2024-09-15T21:19:10.348 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-15T21:19:10.349 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-15T21:19:10.349 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-15T21:19:10.351 INFO:tasks.ceph.mgr.x:Started 2024-09-15T21:19:10.351 DEBUG:tasks.ceph:set 0 configs 2024-09-15T21:19:10.351 INFO:tasks.ceph:Setting crush tunables to default 2024-09-15T21:19:10.352 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd crush tunables default 2024-09-15T21:19:10.534 INFO:teuthology.orchestra.run.smithi073.stderr:adjusted tunables profile to default 2024-09-15T21:19:10.545 INFO:tasks.ceph:check_enable_crimson: False 2024-09-15T21:19:10.545 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-15T21:19:10.545 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:19:10.545 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-15T21:19:10.570 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:19:10.570 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-15T21:19:10.638 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-15T21:19:10.638 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-15T21:19:10.703 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd new c55f097a-59ed-4ce1-9803-4c6019429ea8 0 2024-09-15T21:19:10.921 INFO:teuthology.orchestra.run.smithi073.stdout:0 2024-09-15T21:19:10.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd new 7daff6cd-a09c-4c15-a282-0859e8372dd3 1 2024-09-15T21:19:11.112 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-09-15T21:19:11.124 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd new 9abad4dd-4a88-4080-9ab4-5c4203d7fd51 2 2024-09-15T21:19:11.301 INFO:teuthology.orchestra.run.smithi073.stdout:2 2024-09-15T21:19:11.312 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T21:19:11.313 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-15T21:19:11.314 INFO:tasks.ceph.osd.0:Started 2024-09-15T21:19:11.315 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T21:19:11.315 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T21:19:11.317 INFO:tasks.ceph.osd.1:Started 2024-09-15T21:19:11.317 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T21:19:11.317 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T21:19:11.319 INFO:tasks.ceph.osd.2:Started 2024-09-15T21:19:11.320 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:19:11.495 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:11.496 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":5,"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","created":"2024-09-15T21:19:07.254022+0000","modified":"2024-09-15T21:19:11.297722+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:19:11.297722+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c55f097a-59ed-4ce1-9803-4c6019429ea8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7daff6cd-a09c-4c15-a282-0859e8372dd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9abad4dd-4a88-4080-9ab4-5c4203d7fd51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:19:11.507 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-15T21:19:11.507 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-15T21:19:11.808 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:19:11.967 INFO:teuthology.misc.health.smithi073.stdout: 2024-09-15T21:19:11.968 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":5,"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","created":"2024-09-15T21:19:07.254022+0000","modified":"2024-09-15T21:19:11.297722+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:19:11.297722+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c55f097a-59ed-4ce1-9803-4c6019429ea8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7daff6cd-a09c-4c15-a282-0859e8372dd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9abad4dd-4a88-4080-9ab4-5c4203d7fd51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:19:11.978 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-15T21:19:12.788 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:19:12.785+0000 7fee5580f640 -1 Falling back to public interface 2024-09-15T21:19:12.799 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-15T21:19:12.796+0000 7f1569c26640 -1 Falling back to public interface 2024-09-15T21:19:13.058 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-15T21:19:13.055+0000 7ff8cd224640 -1 Falling back to public interface 2024-09-15T21:19:15.673 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-09-15T21:19:15.669+0000 7f7b13f7d640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-15T21:19:15.673 INFO:tasks.ceph.mgr.x.smithi073.stderr:2024-09-15T21:19:15.669+0000 7f7b13f7d640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-15T21:19:16.614 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-15T21:19:16.611+0000 7f1569c26640 -1 osd.1 0 log_to_monitors true 2024-09-15T21:19:16.845 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-15T21:19:16.842+0000 7ff8cd224640 -1 osd.2 0 log_to_monitors true 2024-09-15T21:19:16.880 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:19:16.877+0000 7fee5580f640 -1 osd.0 0 log_to_monitors true 2024-09-15T21:19:17.689 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-15T21:19:17.686+0000 7f1565bb5640 -1 osd.1 0 waiting for initial osdmap 2024-09-15T21:19:17.692 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-15T21:19:17.689+0000 7f15619dc640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:19:18.280 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:19:18.568 INFO:teuthology.misc.health.smithi073.stdout: 2024-09-15T21:19:18.568 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":7,"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","created":"2024-09-15T21:19:07.254022+0000","modified":"2024-09-15T21:19:17.665248+0000","last_up_change":"0.000000","last_in_change":"2024-09-15T21:19:11.297722+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c55f097a-59ed-4ce1-9803-4c6019429ea8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7daff6cd-a09c-4c15-a282-0859e8372dd3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"9abad4dd-4a88-4080-9ab4-5c4203d7fd51","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:19:18.579 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-15T21:19:18.687 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:19:18.684+0000 7fee511f9640 -1 osd.0 0 waiting for initial osdmap 2024-09-15T21:19:18.691 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:19:18.687+0000 7fee4d020640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:19:19.252 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-15T21:19:19.249+0000 7ff8c8b14640 -1 osd.2 0 waiting for initial osdmap 2024-09-15T21:19:19.261 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-15T21:19:19.258+0000 7ff8c0933640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:19:24.882 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-15T21:19:25.169 INFO:teuthology.misc.health.smithi073.stdout: 2024-09-15T21:19:25.169 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":10,"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","created":"2024-09-15T21:19:07.254022+0000","modified":"2024-09-15T21:19:20.680981+0000","last_up_change":"2024-09-15T21:19:19.676559+0000","last_in_change":"2024-09-15T21:19:11.297722+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c55f097a-59ed-4ce1-9803-4c6019429ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6801","nonce":286810014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6803","nonce":286810014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6807","nonce":286810014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6805","nonce":286810014}]},"public_addr":"172.21.15.73:6801/286810014","cluster_addr":"172.21.15.73:6803/286810014","heartbeat_back_addr":"172.21.15.73:6807/286810014","heartbeat_front_addr":"172.21.15.73:6805/286810014","state":["exists","up"]},{"osd":1,"uuid":"7daff6cd-a09c-4c15-a282-0859e8372dd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6809","nonce":1755138815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6811","nonce":1755138815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6815","nonce":1755138815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6813","nonce":1755138815}]},"public_addr":"172.21.15.73:6809/1755138815","cluster_addr":"172.21.15.73:6811/1755138815","heartbeat_back_addr":"172.21.15.73:6815/1755138815","heartbeat_front_addr":"172.21.15.73:6813/1755138815","state":["exists","up"]},{"osd":2,"uuid":"9abad4dd-4a88-4080-9ab4-5c4203d7fd51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6817","nonce":1392278833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6819","nonce":1392278833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6823","nonce":1392278833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6821","nonce":1392278833}]},"public_addr":"172.21.15.73:6817/1392278833","cluster_addr":"172.21.15.73:6819/1392278833","heartbeat_back_addr":"172.21.15.73:6823/1392278833","heartbeat_front_addr":"172.21.15.73:6821/1392278833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.867529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.608114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.891920+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:19:25.180 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-15T21:19:25.180 INFO:tasks.ceph:Creating RBD pool 2024-09-15T21:19:25.180 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-15T21:19:25.700 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'rbd' created 2024-09-15T21:19:25.715 DEBUG:teuthology.orchestra.run.smithi073:> rbd --cluster ceph pool init rbd 2024-09-15T21:19:28.721 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-15T21:19:28.722 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-15T21:19:28.722 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-15T21:19:29.013 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:29.025 INFO:teuthology.orchestra.run.smithi073.stdout:[{"version":3,"timestamp":"2024-09-15T21:19:19.251173+0000","name":"","changes":[{"name":"osd.2/osd_mclock_max_capacity_iops_ssd","new_value":"9437.743297"}]}] 2024-09-15T21:19:29.025 INFO:tasks.ceph_manager:config epoch is 3 2024-09-15T21:19:29.026 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-15T21:19:29.026 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-15T21:19:29.026 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-15T21:19:29.358 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:29.369 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":4,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":1070491478},{"type":"v1","addr":"172.21.15.73:6825","nonce":1070491478}]},"active_addr":"172.21.15.73:6825/1070491478","active_change":"2024-09-15T21:19:14.656608+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":654698623}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":1413277412}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":1942050275}]}]} 2024-09-15T21:19:29.372 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-15T21:19:29.372 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-15T21:19:29.372 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:19:29.654 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:29.654 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":14,"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","created":"2024-09-15T21:19:07.254022+0000","modified":"2024-09-15T21:19:28.704508+0000","last_up_change":"2024-09-15T21:19:19.676559+0000","last_in_change":"2024-09-15T21:19:11.297722+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:19:25.465189+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c55f097a-59ed-4ce1-9803-4c6019429ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6801","nonce":286810014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6803","nonce":286810014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6807","nonce":286810014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6805","nonce":286810014}]},"public_addr":"172.21.15.73:6801/286810014","cluster_addr":"172.21.15.73:6803/286810014","heartbeat_back_addr":"172.21.15.73:6807/286810014","heartbeat_front_addr":"172.21.15.73:6805/286810014","state":["exists","up"]},{"osd":1,"uuid":"7daff6cd-a09c-4c15-a282-0859e8372dd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6809","nonce":1755138815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6811","nonce":1755138815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6815","nonce":1755138815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6813","nonce":1755138815}]},"public_addr":"172.21.15.73:6809/1755138815","cluster_addr":"172.21.15.73:6811/1755138815","heartbeat_back_addr":"172.21.15.73:6815/1755138815","heartbeat_front_addr":"172.21.15.73:6813/1755138815","state":["exists","up"]},{"osd":2,"uuid":"9abad4dd-4a88-4080-9ab4-5c4203d7fd51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6817","nonce":1392278833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6819","nonce":1392278833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6823","nonce":1392278833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6821","nonce":1392278833}]},"public_addr":"172.21.15.73:6817/1392278833","cluster_addr":"172.21.15.73:6819/1392278833","heartbeat_back_addr":"172.21.15.73:6823/1392278833","heartbeat_front_addr":"172.21.15.73:6821/1392278833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.867529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.608114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.891920+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:19:29.667 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-15T21:19:29.667 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:19:29.949 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:29.949 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":14,"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","created":"2024-09-15T21:19:07.254022+0000","modified":"2024-09-15T21:19:28.704508+0000","last_up_change":"2024-09-15T21:19:19.676559+0000","last_in_change":"2024-09-15T21:19:11.297722+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:19:25.465189+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c55f097a-59ed-4ce1-9803-4c6019429ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6801","nonce":286810014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6803","nonce":286810014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6807","nonce":286810014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6805","nonce":286810014}]},"public_addr":"172.21.15.73:6801/286810014","cluster_addr":"172.21.15.73:6803/286810014","heartbeat_back_addr":"172.21.15.73:6807/286810014","heartbeat_front_addr":"172.21.15.73:6805/286810014","state":["exists","up"]},{"osd":1,"uuid":"7daff6cd-a09c-4c15-a282-0859e8372dd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6809","nonce":1755138815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6811","nonce":1755138815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6815","nonce":1755138815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6813","nonce":1755138815}]},"public_addr":"172.21.15.73:6809/1755138815","cluster_addr":"172.21.15.73:6811/1755138815","heartbeat_back_addr":"172.21.15.73:6815/1755138815","heartbeat_front_addr":"172.21.15.73:6813/1755138815","state":["exists","up"]},{"osd":2,"uuid":"9abad4dd-4a88-4080-9ab4-5c4203d7fd51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6817","nonce":1392278833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6819","nonce":1392278833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6823","nonce":1392278833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6821","nonce":1392278833}]},"public_addr":"172.21.15.73:6817/1392278833","cluster_addr":"172.21.15.73:6819/1392278833","heartbeat_back_addr":"172.21.15.73:6823/1392278833","heartbeat_front_addr":"172.21.15.73:6821/1392278833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.867529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.608114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.891920+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:19:29.962 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:19:29.962 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:19:29.963 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:19:30.098 INFO:teuthology.orchestra.run.smithi073.stdout:38654705669 2024-09-15T21:19:30.098 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:19:30.117 INFO:teuthology.orchestra.run.smithi073.stdout:34359738374 2024-09-15T21:19:30.117 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:19:30.118 INFO:teuthology.orchestra.run.smithi073.stdout:38654705669 2024-09-15T21:19:30.118 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:19:30.397 INFO:teuthology.orchestra.run.smithi073.stdout:38654705668 2024-09-15T21:19:30.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.0 2024-09-15T21:19:30.454 INFO:teuthology.orchestra.run.smithi073.stdout:34359738372 2024-09-15T21:19:30.455 INFO:teuthology.orchestra.run.smithi073.stdout:38654705668 2024-09-15T21:19:30.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738372 for osd.1 2024-09-15T21:19:30.469 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705668 for osd.2 2024-09-15T21:19:31.410 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:19:31.469 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:19:31.470 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:19:31.715 INFO:teuthology.orchestra.run.smithi073.stdout:38654705669 2024-09-15T21:19:31.729 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.0 2024-09-15T21:19:31.729 DEBUG:teuthology.parallel:result is None 2024-09-15T21:19:31.771 INFO:teuthology.orchestra.run.smithi073.stdout:38654705669 2024-09-15T21:19:31.777 INFO:teuthology.orchestra.run.smithi073.stdout:34359738374 2024-09-15T21:19:31.784 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705669 got 38654705669 for osd.2 2024-09-15T21:19:31.784 DEBUG:teuthology.parallel:result is None 2024-09-15T21:19:31.790 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738374 for osd.1 2024-09-15T21:19:31.790 DEBUG:teuthology.parallel:result is None 2024-09-15T21:19:31.791 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:19:31.791 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:32.113 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:32.113 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:19:32.125 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-15T21:19:30.667971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81452,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034068,"statfs":{"total":289910292480,"available":289826885632,"internally_reserved":0,"allocated":376832,"data_stored":96347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712487+0000","last_change":"2024-09-15T21:19:28.712487+0000","last_active":"2024-09-15T21:19:28.712487+0000","last_peered":"2024-09-15T21:19:28.712487+0000","last_clean":"2024-09-15T21:19:28.712487+0000","last_became_active":"2024-09-15T21:19:26.707803+0000","last_became_peered":"2024-09-15T21:19:26.707803+0000","last_unstale":"2024-09-15T21:19:28.712487+0000","last_undegraded":"2024-09-15T21:19:28.712487+0000","last_fullsized":"2024-09-15T21:19:28.712487+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:20.265021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712635+0000","last_change":"2024-09-15T21:19:28.712635+0000","last_active":"2024-09-15T21:19:28.712635+0000","last_peered":"2024-09-15T21:19:28.712635+0000","last_clean":"2024-09-15T21:19:28.712635+0000","last_became_active":"2024-09-15T21:19:26.705751+0000","last_became_peered":"2024-09-15T21:19:26.705751+0000","last_unstale":"2024-09-15T21:19:28.712635+0000","last_undegraded":"2024-09-15T21:19:28.712635+0000","last_fullsized":"2024-09-15T21:19:28.712635+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:12.537829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.957418+0000","last_change":"2024-09-15T21:19:28.957418+0000","last_active":"2024-09-15T21:19:28.957418+0000","last_peered":"2024-09-15T21:19:28.957418+0000","last_clean":"2024-09-15T21:19:28.957418+0000","last_became_active":"2024-09-15T21:19:26.710933+0000","last_became_peered":"2024-09-15T21:19:26.710933+0000","last_unstale":"2024-09-15T21:19:28.957418+0000","last_undegraded":"2024-09-15T21:19:28.957418+0000","last_fullsized":"2024-09-15T21:19:28.957418+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:15:42.313854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036527100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712419+0000","last_change":"2024-09-15T21:19:28.712419+0000","last_active":"2024-09-15T21:19:28.712419+0000","last_peered":"2024-09-15T21:19:28.712419+0000","last_clean":"2024-09-15T21:19:28.712419+0000","last_became_active":"2024-09-15T21:19:26.705010+0000","last_became_peered":"2024-09-15T21:19:26.705010+0000","last_unstale":"2024-09-15T21:19:28.712419+0000","last_undegraded":"2024-09-15T21:19:28.712419+0000","last_fullsized":"2024-09-15T21:19:28.712419+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:58.295608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712799+0000","last_change":"2024-09-15T21:19:28.712799+0000","last_active":"2024-09-15T21:19:28.712799+0000","last_peered":"2024-09-15T21:19:28.712799+0000","last_clean":"2024-09-15T21:19:28.712799+0000","last_became_active":"2024-09-15T21:19:26.707957+0000","last_became_peered":"2024-09-15T21:19:26.707957+0000","last_unstale":"2024-09-15T21:19:28.712799+0000","last_undegraded":"2024-09-15T21:19:28.712799+0000","last_fullsized":"2024-09-15T21:19:28.712799+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:12.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.714126+0000","last_change":"2024-09-15T21:19:28.711657+0000","last_active":"2024-09-15T21:19:28.714126+0000","last_peered":"2024-09-15T21:19:28.714126+0000","last_clean":"2024-09-15T21:19:28.714126+0000","last_became_active":"2024-09-15T21:19:26.706209+0000","last_became_peered":"2024-09-15T21:19:26.706209+0000","last_unstale":"2024-09-15T21:19:28.714126+0000","last_undegraded":"2024-09-15T21:19:28.714126+0000","last_fullsized":"2024-09-15T21:19:28.714126+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:45.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.957434+0000","last_change":"2024-09-15T21:19:28.957434+0000","last_active":"2024-09-15T21:19:28.957434+0000","last_peered":"2024-09-15T21:19:28.957434+0000","last_clean":"2024-09-15T21:19:28.957434+0000","last_became_active":"2024-09-15T21:19:26.710918+0000","last_became_peered":"2024-09-15T21:19:26.710918+0000","last_unstale":"2024-09-15T21:19:28.957434+0000","last_undegraded":"2024-09-15T21:19:28.957434+0000","last_fullsized":"2024-09-15T21:19:28.957434+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:30.090386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041061400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.711561+0000","last_change":"2024-09-15T21:19:28.711561+0000","last_active":"2024-09-15T21:19:28.711561+0000","last_peered":"2024-09-15T21:19:28.711561+0000","last_clean":"2024-09-15T21:19:28.711561+0000","last_became_active":"2024-09-15T21:19:26.704834+0000","last_became_peered":"2024-09-15T21:19:26.704834+0000","last_unstale":"2024-09-15T21:19:28.711561+0000","last_undegraded":"2024-09-15T21:19:28.711561+0000","last_fullsized":"2024-09-15T21:19:28.711561+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:09.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705669,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":122880,"data_stored":32103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":126976,"data_stored":32122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":32122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:32.126 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:32.408 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:32.408 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:19:32.420 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-15T21:19:32.238120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81452,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034068,"statfs":{"total":289910292480,"available":289826885632,"internally_reserved":0,"allocated":376832,"data_stored":96347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.570148"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712487+0000","last_change":"2024-09-15T21:19:28.712487+0000","last_active":"2024-09-15T21:19:28.712487+0000","last_peered":"2024-09-15T21:19:28.712487+0000","last_clean":"2024-09-15T21:19:28.712487+0000","last_became_active":"2024-09-15T21:19:26.707803+0000","last_became_peered":"2024-09-15T21:19:26.707803+0000","last_unstale":"2024-09-15T21:19:28.712487+0000","last_undegraded":"2024-09-15T21:19:28.712487+0000","last_fullsized":"2024-09-15T21:19:28.712487+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:20.265021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712635+0000","last_change":"2024-09-15T21:19:28.712635+0000","last_active":"2024-09-15T21:19:28.712635+0000","last_peered":"2024-09-15T21:19:28.712635+0000","last_clean":"2024-09-15T21:19:28.712635+0000","last_became_active":"2024-09-15T21:19:26.705751+0000","last_became_peered":"2024-09-15T21:19:26.705751+0000","last_unstale":"2024-09-15T21:19:28.712635+0000","last_undegraded":"2024-09-15T21:19:28.712635+0000","last_fullsized":"2024-09-15T21:19:28.712635+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:12.537829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.957418+0000","last_change":"2024-09-15T21:19:28.957418+0000","last_active":"2024-09-15T21:19:28.957418+0000","last_peered":"2024-09-15T21:19:28.957418+0000","last_clean":"2024-09-15T21:19:28.957418+0000","last_became_active":"2024-09-15T21:19:26.710933+0000","last_became_peered":"2024-09-15T21:19:26.710933+0000","last_unstale":"2024-09-15T21:19:28.957418+0000","last_undegraded":"2024-09-15T21:19:28.957418+0000","last_fullsized":"2024-09-15T21:19:28.957418+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:15:42.313854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036527100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712419+0000","last_change":"2024-09-15T21:19:28.712419+0000","last_active":"2024-09-15T21:19:28.712419+0000","last_peered":"2024-09-15T21:19:28.712419+0000","last_clean":"2024-09-15T21:19:28.712419+0000","last_became_active":"2024-09-15T21:19:26.705010+0000","last_became_peered":"2024-09-15T21:19:26.705010+0000","last_unstale":"2024-09-15T21:19:28.712419+0000","last_undegraded":"2024-09-15T21:19:28.712419+0000","last_fullsized":"2024-09-15T21:19:28.712419+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:58.295608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712799+0000","last_change":"2024-09-15T21:19:28.712799+0000","last_active":"2024-09-15T21:19:28.712799+0000","last_peered":"2024-09-15T21:19:28.712799+0000","last_clean":"2024-09-15T21:19:28.712799+0000","last_became_active":"2024-09-15T21:19:26.707957+0000","last_became_peered":"2024-09-15T21:19:26.707957+0000","last_unstale":"2024-09-15T21:19:28.712799+0000","last_undegraded":"2024-09-15T21:19:28.712799+0000","last_fullsized":"2024-09-15T21:19:28.712799+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:12.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.714126+0000","last_change":"2024-09-15T21:19:28.711657+0000","last_active":"2024-09-15T21:19:28.714126+0000","last_peered":"2024-09-15T21:19:28.714126+0000","last_clean":"2024-09-15T21:19:28.714126+0000","last_became_active":"2024-09-15T21:19:26.706209+0000","last_became_peered":"2024-09-15T21:19:26.706209+0000","last_unstale":"2024-09-15T21:19:28.714126+0000","last_undegraded":"2024-09-15T21:19:28.714126+0000","last_fullsized":"2024-09-15T21:19:28.714126+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:45.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.957434+0000","last_change":"2024-09-15T21:19:28.957434+0000","last_active":"2024-09-15T21:19:28.957434+0000","last_peered":"2024-09-15T21:19:28.957434+0000","last_clean":"2024-09-15T21:19:28.957434+0000","last_became_active":"2024-09-15T21:19:26.710918+0000","last_became_peered":"2024-09-15T21:19:26.710918+0000","last_unstale":"2024-09-15T21:19:28.957434+0000","last_undegraded":"2024-09-15T21:19:28.957434+0000","last_fullsized":"2024-09-15T21:19:28.957434+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:30.090386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041061400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.711561+0000","last_change":"2024-09-15T21:19:28.711561+0000","last_active":"2024-09-15T21:19:28.711561+0000","last_peered":"2024-09-15T21:19:28.711561+0000","last_clean":"2024-09-15T21:19:28.711561+0000","last_became_active":"2024-09-15T21:19:26.704834+0000","last_became_peered":"2024-09-15T21:19:26.704834+0000","last_unstale":"2024-09-15T21:19:28.711561+0000","last_undegraded":"2024-09-15T21:19:28.711561+0000","last_fullsized":"2024-09-15T21:19:28.711561+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:09.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":122880,"data_stored":32103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":126976,"data_stored":32122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":32122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:32.420 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:19:32.421 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-15T21:19:32.421 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-15T21:19:32.421 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-15T21:19:32.744 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:32.744 INFO:teuthology.orchestra.run.smithi073.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-15T21:19:32.758 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-15T21:19:32.758 INFO:teuthology.run_tasks:Running task lost_unfound... 2024-09-15T21:19:32.783 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:19:33.069 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:33.069 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":15,"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","created":"2024-09-15T21:19:07.254022+0000","modified":"2024-09-15T21:19:32.235840+0000","last_up_change":"2024-09-15T21:19:19.676559+0000","last_in_change":"2024-09-15T21:19:11.297722+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:19:25.465189+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c55f097a-59ed-4ce1-9803-4c6019429ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6801","nonce":286810014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6803","nonce":286810014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6807","nonce":286810014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":286810014},{"type":"v1","addr":"172.21.15.73:6805","nonce":286810014}]},"public_addr":"172.21.15.73:6801/286810014","cluster_addr":"172.21.15.73:6803/286810014","heartbeat_back_addr":"172.21.15.73:6807/286810014","heartbeat_front_addr":"172.21.15.73:6805/286810014","state":["exists","up"]},{"osd":1,"uuid":"7daff6cd-a09c-4c15-a282-0859e8372dd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6809","nonce":1755138815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6811","nonce":1755138815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6815","nonce":1755138815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6813","nonce":1755138815}]},"public_addr":"172.21.15.73:6809/1755138815","cluster_addr":"172.21.15.73:6811/1755138815","heartbeat_back_addr":"172.21.15.73:6815/1755138815","heartbeat_front_addr":"172.21.15.73:6813/1755138815","state":["exists","up"]},{"osd":2,"uuid":"9abad4dd-4a88-4080-9ab4-5c4203d7fd51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6817","nonce":1392278833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6819","nonce":1392278833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6823","nonce":1392278833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6821","nonce":1392278833}]},"public_addr":"172.21.15.73:6817/1392278833","cluster_addr":"172.21.15.73:6819/1392278833","heartbeat_back_addr":"172.21.15.73:6823/1392278833","heartbeat_front_addr":"172.21.15.73:6821/1392278833","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.867529+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.608114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.891920+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:19:33.081 INFO:tasks.lost_unfound.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-15T21:19:25.465189+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 14, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-15T21:19:33.082 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-15T21:19:33.367 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2024-09-15T21:19:33.380 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-15T21:19:33.661 INFO:teuthology.orchestra.run.smithi073.stdout:epoch 15 2024-09-15T21:19:33.661 INFO:teuthology.orchestra.run.smithi073.stdout:fsid 1d60fae4-b8c3-4364-9331-b0445493a043 2024-09-15T21:19:33.661 INFO:teuthology.orchestra.run.smithi073.stdout:created 2024-09-15T21:19:07.254022+0000 2024-09-15T21:19:33.661 INFO:teuthology.orchestra.run.smithi073.stdout:modified 2024-09-15T21:19:32.235840+0000 2024-09-15T21:19:33.661 INFO:teuthology.orchestra.run.smithi073.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-15T21:19:33.662 INFO:teuthology.orchestra.run.smithi073.stdout:crush_version 5 2024-09-15T21:19:33.662 INFO:teuthology.orchestra.run.smithi073.stdout:full_ratio 0.95 2024-09-15T21:19:33.662 INFO:teuthology.orchestra.run.smithi073.stdout:backfillfull_ratio 0.9 2024-09-15T21:19:33.662 INFO:teuthology.orchestra.run.smithi073.stdout:nearfull_ratio 0.85 2024-09-15T21:19:33.662 INFO:teuthology.orchestra.run.smithi073.stdout:require_min_compat_client luminous 2024-09-15T21:19:33.662 INFO:teuthology.orchestra.run.smithi073.stdout:min_compat_client jewel 2024-09-15T21:19:33.662 INFO:teuthology.orchestra.run.smithi073.stdout:require_osd_release squid 2024-09-15T21:19:33.662 INFO:teuthology.orchestra.run.smithi073.stdout:stretch_mode_enabled false 2024-09-15T21:19:33.662 INFO:teuthology.orchestra.run.smithi073.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 14 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.88 2024-09-15T21:19:33.662 INFO:teuthology.orchestra.run.smithi073.stdout:max_osd 3 2024-09-15T21:19:33.662 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6800/286810014,v1:172.21.15.73:6801/286810014] [v2:172.21.15.73:6802/286810014,v1:172.21.15.73:6803/286810014] exists,up c55f097a-59ed-4ce1-9803-4c6019429ea8 2024-09-15T21:19:33.662 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6808/1755138815,v1:172.21.15.73:6809/1755138815] [v2:172.21.15.73:6810/1755138815,v1:172.21.15.73:6811/1755138815] exists,up 7daff6cd-a09c-4c15-a282-0859e8372dd3 2024-09-15T21:19:33.662 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6816/1392278833,v1:172.21.15.73:6817/1392278833] [v2:172.21.15.73:6818/1392278833,v1:172.21.15.73:6819/1392278833] exists,up 9abad4dd-4a88-4080-9ab4-5c4203d7fd51 2024-09-15T21:19:33.674 INFO:tasks.lost_unfound.ceph_manager:['osd.0 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6800/286810014,v1:172.21.15.73:6801/286810014] [v2:172.21.15.73:6802/286810014,v1:172.21.15.73:6803/286810014] exists,up c55f097a-59ed-4ce1-9803-4c6019429ea8', 'osd.1 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6808/1755138815,v1:172.21.15.73:6809/1755138815] [v2:172.21.15.73:6810/1755138815,v1:172.21.15.73:6811/1755138815] exists,up 7daff6cd-a09c-4c15-a282-0859e8372dd3', 'osd.2 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6816/1392278833,v1:172.21.15.73:6817/1392278833] [v2:172.21.15.73:6818/1392278833,v1:172.21.15.73:6819/1392278833] exists,up 9abad4dd-4a88-4080-9ab4-5c4203d7fd51'] 2024-09-15T21:19:33.674 INFO:tasks.lost_unfound.ceph_manager:waiting for clean 2024-09-15T21:19:33.674 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:33.956 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:33.957 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:19:33.966 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-15T21:19:32.668224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81452,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034068,"statfs":{"total":289910292480,"available":289826885632,"internally_reserved":0,"allocated":376832,"data_stored":96347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000252"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712487+0000","last_change":"2024-09-15T21:19:28.712487+0000","last_active":"2024-09-15T21:19:28.712487+0000","last_peered":"2024-09-15T21:19:28.712487+0000","last_clean":"2024-09-15T21:19:28.712487+0000","last_became_active":"2024-09-15T21:19:26.707803+0000","last_became_peered":"2024-09-15T21:19:26.707803+0000","last_unstale":"2024-09-15T21:19:28.712487+0000","last_undegraded":"2024-09-15T21:19:28.712487+0000","last_fullsized":"2024-09-15T21:19:28.712487+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:20.265021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712635+0000","last_change":"2024-09-15T21:19:28.712635+0000","last_active":"2024-09-15T21:19:28.712635+0000","last_peered":"2024-09-15T21:19:28.712635+0000","last_clean":"2024-09-15T21:19:28.712635+0000","last_became_active":"2024-09-15T21:19:26.705751+0000","last_became_peered":"2024-09-15T21:19:26.705751+0000","last_unstale":"2024-09-15T21:19:28.712635+0000","last_undegraded":"2024-09-15T21:19:28.712635+0000","last_fullsized":"2024-09-15T21:19:28.712635+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:12.537829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.957418+0000","last_change":"2024-09-15T21:19:28.957418+0000","last_active":"2024-09-15T21:19:28.957418+0000","last_peered":"2024-09-15T21:19:28.957418+0000","last_clean":"2024-09-15T21:19:28.957418+0000","last_became_active":"2024-09-15T21:19:26.710933+0000","last_became_peered":"2024-09-15T21:19:26.710933+0000","last_unstale":"2024-09-15T21:19:28.957418+0000","last_undegraded":"2024-09-15T21:19:28.957418+0000","last_fullsized":"2024-09-15T21:19:28.957418+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:15:42.313854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036527100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712419+0000","last_change":"2024-09-15T21:19:28.712419+0000","last_active":"2024-09-15T21:19:28.712419+0000","last_peered":"2024-09-15T21:19:28.712419+0000","last_clean":"2024-09-15T21:19:28.712419+0000","last_became_active":"2024-09-15T21:19:26.705010+0000","last_became_peered":"2024-09-15T21:19:26.705010+0000","last_unstale":"2024-09-15T21:19:28.712419+0000","last_undegraded":"2024-09-15T21:19:28.712419+0000","last_fullsized":"2024-09-15T21:19:28.712419+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:58.295608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712799+0000","last_change":"2024-09-15T21:19:28.712799+0000","last_active":"2024-09-15T21:19:28.712799+0000","last_peered":"2024-09-15T21:19:28.712799+0000","last_clean":"2024-09-15T21:19:28.712799+0000","last_became_active":"2024-09-15T21:19:26.707957+0000","last_became_peered":"2024-09-15T21:19:26.707957+0000","last_unstale":"2024-09-15T21:19:28.712799+0000","last_undegraded":"2024-09-15T21:19:28.712799+0000","last_fullsized":"2024-09-15T21:19:28.712799+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:12.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.714126+0000","last_change":"2024-09-15T21:19:28.711657+0000","last_active":"2024-09-15T21:19:28.714126+0000","last_peered":"2024-09-15T21:19:28.714126+0000","last_clean":"2024-09-15T21:19:28.714126+0000","last_became_active":"2024-09-15T21:19:26.706209+0000","last_became_peered":"2024-09-15T21:19:26.706209+0000","last_unstale":"2024-09-15T21:19:28.714126+0000","last_undegraded":"2024-09-15T21:19:28.714126+0000","last_fullsized":"2024-09-15T21:19:28.714126+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:45.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.957434+0000","last_change":"2024-09-15T21:19:28.957434+0000","last_active":"2024-09-15T21:19:28.957434+0000","last_peered":"2024-09-15T21:19:28.957434+0000","last_clean":"2024-09-15T21:19:28.957434+0000","last_became_active":"2024-09-15T21:19:26.710918+0000","last_became_peered":"2024-09-15T21:19:26.710918+0000","last_unstale":"2024-09-15T21:19:28.957434+0000","last_undegraded":"2024-09-15T21:19:28.957434+0000","last_fullsized":"2024-09-15T21:19:28.957434+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:30.090386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041061400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.711561+0000","last_change":"2024-09-15T21:19:28.711561+0000","last_active":"2024-09-15T21:19:28.711561+0000","last_peered":"2024-09-15T21:19:28.711561+0000","last_clean":"2024-09-15T21:19:28.711561+0000","last_became_active":"2024-09-15T21:19:26.704834+0000","last_became_peered":"2024-09-15T21:19:26.704834+0000","last_unstale":"2024-09-15T21:19:28.711561+0000","last_undegraded":"2024-09-15T21:19:28.711561+0000","last_fullsized":"2024-09-15T21:19:28.711561+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:09.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":122880,"data_stored":32103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":126976,"data_stored":32122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":32122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:33.967 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:34.247 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:34.247 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:19:34.259 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2024-09-15T21:19:32.668224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81452,"kb_used_data":368,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034068,"statfs":{"total":289910292480,"available":289826885632,"internally_reserved":0,"allocated":376832,"data_stored":96347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000252"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712487+0000","last_change":"2024-09-15T21:19:28.712487+0000","last_active":"2024-09-15T21:19:28.712487+0000","last_peered":"2024-09-15T21:19:28.712487+0000","last_clean":"2024-09-15T21:19:28.712487+0000","last_became_active":"2024-09-15T21:19:26.707803+0000","last_became_peered":"2024-09-15T21:19:26.707803+0000","last_unstale":"2024-09-15T21:19:28.712487+0000","last_undegraded":"2024-09-15T21:19:28.712487+0000","last_fullsized":"2024-09-15T21:19:28.712487+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:20.265021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712635+0000","last_change":"2024-09-15T21:19:28.712635+0000","last_active":"2024-09-15T21:19:28.712635+0000","last_peered":"2024-09-15T21:19:28.712635+0000","last_clean":"2024-09-15T21:19:28.712635+0000","last_became_active":"2024-09-15T21:19:26.705751+0000","last_became_peered":"2024-09-15T21:19:26.705751+0000","last_unstale":"2024-09-15T21:19:28.712635+0000","last_undegraded":"2024-09-15T21:19:28.712635+0000","last_fullsized":"2024-09-15T21:19:28.712635+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:12.537829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.957418+0000","last_change":"2024-09-15T21:19:28.957418+0000","last_active":"2024-09-15T21:19:28.957418+0000","last_peered":"2024-09-15T21:19:28.957418+0000","last_clean":"2024-09-15T21:19:28.957418+0000","last_became_active":"2024-09-15T21:19:26.710933+0000","last_became_peered":"2024-09-15T21:19:26.710933+0000","last_unstale":"2024-09-15T21:19:28.957418+0000","last_undegraded":"2024-09-15T21:19:28.957418+0000","last_fullsized":"2024-09-15T21:19:28.957418+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:15:42.313854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00036527100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712419+0000","last_change":"2024-09-15T21:19:28.712419+0000","last_active":"2024-09-15T21:19:28.712419+0000","last_peered":"2024-09-15T21:19:28.712419+0000","last_clean":"2024-09-15T21:19:28.712419+0000","last_became_active":"2024-09-15T21:19:26.705010+0000","last_became_peered":"2024-09-15T21:19:26.705010+0000","last_unstale":"2024-09-15T21:19:28.712419+0000","last_undegraded":"2024-09-15T21:19:28.712419+0000","last_fullsized":"2024-09-15T21:19:28.712419+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:58.295608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.712799+0000","last_change":"2024-09-15T21:19:28.712799+0000","last_active":"2024-09-15T21:19:28.712799+0000","last_peered":"2024-09-15T21:19:28.712799+0000","last_clean":"2024-09-15T21:19:28.712799+0000","last_became_active":"2024-09-15T21:19:26.707957+0000","last_became_peered":"2024-09-15T21:19:26.707957+0000","last_unstale":"2024-09-15T21:19:28.712799+0000","last_undegraded":"2024-09-15T21:19:28.712799+0000","last_fullsized":"2024-09-15T21:19:28.712799+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:35:12.518025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.714126+0000","last_change":"2024-09-15T21:19:28.711657+0000","last_active":"2024-09-15T21:19:28.714126+0000","last_peered":"2024-09-15T21:19:28.714126+0000","last_clean":"2024-09-15T21:19:28.714126+0000","last_became_active":"2024-09-15T21:19:26.706209+0000","last_became_peered":"2024-09-15T21:19:26.706209+0000","last_unstale":"2024-09-15T21:19:28.714126+0000","last_undegraded":"2024-09-15T21:19:28.714126+0000","last_fullsized":"2024-09-15T21:19:28.714126+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:45.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.957434+0000","last_change":"2024-09-15T21:19:28.957434+0000","last_active":"2024-09-15T21:19:28.957434+0000","last_peered":"2024-09-15T21:19:28.957434+0000","last_clean":"2024-09-15T21:19:28.957434+0000","last_became_active":"2024-09-15T21:19:26.710918+0000","last_became_peered":"2024-09-15T21:19:26.710918+0000","last_unstale":"2024-09-15T21:19:28.957434+0000","last_undegraded":"2024-09-15T21:19:28.957434+0000","last_fullsized":"2024-09-15T21:19:28.957434+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:58:30.090386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041061400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-15T21:19:28.711561+0000","last_change":"2024-09-15T21:19:28.711561+0000","last_active":"2024-09-15T21:19:28.711561+0000","last_peered":"2024-09-15T21:19:28.711561+0000","last_clean":"2024-09-15T21:19:28.711561+0000","last_became_active":"2024-09-15T21:19:26.704834+0000","last_became_peered":"2024-09-15T21:19:26.704834+0000","last_unstale":"2024-09-15T21:19:28.711561+0000","last_undegraded":"2024-09-15T21:19:28.711561+0000","last_fullsized":"2024-09-15T21:19:28.711561+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:09.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705670,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":122880,"data_stored":32103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705670,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":126976,"data_stored":32122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":126976,"data_stored":32122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:34.260 INFO:tasks.lost_unfound.ceph_manager:clean! 2024-09-15T21:19:34.261 INFO:tasks.lost_unfound.ceph_manager:creating pool_name unfound_pool 2024-09-15T21:19:34.261 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unfound_pool 16 2024-09-15T21:19:34.725 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'unfound_pool' created 2024-09-15T21:19:34.743 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unfound_pool rados --yes-i-really-mean-it || true 2024-09-15T21:19:35.731 INFO:teuthology.orchestra.run.smithi073.stderr:enabled application 'rados' on pool 'unfound_pool' 2024-09-15T21:19:36.757 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T21:19:36.757 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:19:36.782 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T21:19:36.782 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-15T21:19:37.127 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-15T21:19:37.123+0000 7f8e1ee55640 -1 mon.a@0(leader).osd e18 definitely_dead 0 2024-09-15T21:19:37.744 INFO:teuthology.orchestra.run.smithi073.stderr:marked down osd.2. 2024-09-15T21:19:37.759 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-15T21:19:38.054 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-15T21:19:38.050+0000 7f8e1ee55640 -1 mon.a@0(leader).osd e19 definitely_dead 0 2024-09-15T21:19:38.742 INFO:teuthology.orchestra.run.smithi073.stderr:marked out osd.2. 2024-09-15T21:19:38.761 INFO:tasks.util.rados:rados -p unfound_pool put dummy /etc/fstab 2024-09-15T21:19:38.762 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put dummy /etc/fstab 2024-09-15T21:19:38.814 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:19:38.815 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:19:38.948 INFO:teuthology.orchestra.run.smithi073.stdout:38654705672 2024-09-15T21:19:38.949 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:19:38.949 INFO:teuthology.orchestra.run.smithi073.stdout:34359738376 2024-09-15T21:19:38.950 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:19:39.275 INFO:teuthology.orchestra.run.smithi073.stdout:38654705671 2024-09-15T21:19:39.280 INFO:teuthology.orchestra.run.smithi073.stdout:34359738375 2024-09-15T21:19:39.289 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705672 got 38654705671 for osd.0 2024-09-15T21:19:39.291 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738376 got 34359738375 for osd.1 2024-09-15T21:19:40.290 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:19:40.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:19:40.592 INFO:teuthology.orchestra.run.smithi073.stdout:34359738375 2024-09-15T21:19:40.594 INFO:teuthology.orchestra.run.smithi073.stdout:38654705671 2024-09-15T21:19:40.609 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738376 got 34359738375 for osd.1 2024-09-15T21:19:40.611 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705672 got 38654705671 for osd.0 2024-09-15T21:19:41.610 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:19:41.611 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:19:41.912 INFO:teuthology.orchestra.run.smithi073.stdout:38654705672 2024-09-15T21:19:41.919 INFO:teuthology.orchestra.run.smithi073.stdout:34359738377 2024-09-15T21:19:41.925 INFO:tasks.lost_unfound.ceph_manager:need seq 38654705672 got 38654705672 for osd.0 2024-09-15T21:19:41.926 DEBUG:teuthology.parallel:result is None 2024-09-15T21:19:41.932 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738376 got 34359738377 for osd.1 2024-09-15T21:19:41.932 DEBUG:teuthology.parallel:result is None 2024-09-15T21:19:41.932 INFO:tasks.lost_unfound.ceph_manager:waiting for recovery to complete 2024-09-15T21:19:41.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:42.251 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:42.251 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:19:42.263 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-15T21:19:40.669717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54408,"kb_used_data":340,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689272,"statfs":{"total":193273528320,"available":193217814528,"internally_reserved":0,"allocated":348160,"data_stored":101227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000810"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.742617+0000","last_change":"2024-09-15T21:19:35.736448+0000","last_active":"2024-09-15T21:19:38.742617+0000","last_peered":"2024-09-15T21:19:38.742617+0000","last_clean":"2024-09-15T21:19:38.742617+0000","last_became_active":"2024-09-15T21:19:35.736099+0000","last_became_peered":"2024-09-15T21:19:35.736099+0000","last_unstale":"2024-09-15T21:19:38.742617+0000","last_undegraded":"2024-09-15T21:19:38.742617+0000","last_fullsized":"2024-09-15T21:19:38.742617+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:18.340741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.743370+0000","last_change":"2024-09-15T21:19:37.742118+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.743370+0000","last_undegraded":"2024-09-15T21:19:38.743370+0000","last_fullsized":"2024-09-15T21:19:38.743370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.745563+0000","last_change":"2024-09-15T21:19:37.743880+0000","last_active":"2024-09-15T21:19:37.743880+0000","last_peered":"2024-09-15T21:19:36.740990+0000","last_clean":"2024-09-15T21:19:36.740990+0000","last_became_active":"2024-09-15T21:19:35.742040+0000","last_became_peered":"2024-09-15T21:19:35.742040+0000","last_unstale":"2024-09-15T21:19:38.745563+0000","last_undegraded":"2024-09-15T21:19:38.745563+0000","last_fullsized":"2024-09-15T21:19:38.745563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.743250+0000","last_change":"2024-09-15T21:19:37.742063+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.743250+0000","last_undegraded":"2024-09-15T21:19:38.743250+0000","last_fullsized":"2024-09-15T21:19:38.743250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.742226+0000","last_change":"2024-09-15T21:19:37.742226+0000","last_active":"2024-09-15T21:19:37.742226+0000","last_peered":"2024-09-15T21:19:36.741108+0000","last_clean":"2024-09-15T21:19:36.741108+0000","last_became_active":"2024-09-15T21:19:35.730478+0000","last_became_peered":"2024-09-15T21:19:35.730478+0000","last_unstale":"2024-09-15T21:19:37.742226+0000","last_undegraded":"2024-09-15T21:19:37.742226+0000","last_fullsized":"2024-09-15T21:19:37.742226+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744403+0000","last_change":"2024-09-15T21:19:35.740110+0000","last_active":"2024-09-15T21:19:38.744403+0000","last_peered":"2024-09-15T21:19:38.744403+0000","last_clean":"2024-09-15T21:19:38.744403+0000","last_became_active":"2024-09-15T21:19:35.739773+0000","last_became_peered":"2024-09-15T21:19:35.739773+0000","last_unstale":"2024-09-15T21:19:38.744403+0000","last_undegraded":"2024-09-15T21:19:38.744403+0000","last_fullsized":"2024-09-15T21:19:38.744403+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:49:46.277839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744485+0000","last_change":"2024-09-15T21:19:35.740711+0000","last_active":"2024-09-15T21:19:38.744485+0000","last_peered":"2024-09-15T21:19:38.744485+0000","last_clean":"2024-09-15T21:19:38.744485+0000","last_became_active":"2024-09-15T21:19:35.740436+0000","last_became_peered":"2024-09-15T21:19:35.740436+0000","last_unstale":"2024-09-15T21:19:38.744485+0000","last_undegraded":"2024-09-15T21:19:38.744485+0000","last_fullsized":"2024-09-15T21:19:38.744485+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:56.312225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.741258+0000","last_change":"2024-09-15T21:19:37.741258+0000","last_active":"2024-09-15T21:19:37.741258+0000","last_peered":"2024-09-15T21:19:36.740313+0000","last_clean":"2024-09-15T21:19:36.740313+0000","last_became_active":"2024-09-15T21:19:35.730414+0000","last_became_peered":"2024-09-15T21:19:35.730414+0000","last_unstale":"2024-09-15T21:19:37.741258+0000","last_undegraded":"2024-09-15T21:19:37.741258+0000","last_fullsized":"2024-09-15T21:19:37.741258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.743874+0000","last_change":"2024-09-15T21:19:28.711561+0000","last_active":"2024-09-15T21:19:38.743874+0000","last_peered":"2024-09-15T21:19:38.743874+0000","last_clean":"2024-09-15T21:19:38.743874+0000","last_became_active":"2024-09-15T21:19:26.704834+0000","last_became_peered":"2024-09-15T21:19:26.704834+0000","last_unstale":"2024-09-15T21:19:38.743874+0000","last_undegraded":"2024-09-15T21:19:38.743874+0000","last_fullsized":"2024-09-15T21:19:38.743874+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:09.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744905+0000","last_change":"2024-09-15T21:19:37.743194+0000","last_active":"2024-09-15T21:19:37.743194+0000","last_peered":"2024-09-15T21:19:36.740191+0000","last_clean":"2024-09-15T21:19:36.740191+0000","last_became_active":"2024-09-15T21:19:35.741860+0000","last_became_peered":"2024-09-15T21:19:35.741860+0000","last_unstale":"2024-09-15T21:19:38.744905+0000","last_undegraded":"2024-09-15T21:19:38.744905+0000","last_fullsized":"2024-09-15T21:19:38.744905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.742560+0000","last_change":"2024-09-15T21:19:37.741558+0000","last_active":"2024-09-15T21:19:25.689420+0000","last_peered":"2024-09-15T21:19:25.689420+0000","last_clean":"2024-09-15T21:19:25.689420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.742560+0000","last_undegraded":"2024-09-15T21:19:38.742560+0000","last_fullsized":"2024-09-15T21:19:38.742560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.742505+0000","last_change":"2024-09-15T21:19:35.736396+0000","last_active":"2024-09-15T21:19:38.742505+0000","last_peered":"2024-09-15T21:19:38.742505+0000","last_clean":"2024-09-15T21:19:38.742505+0000","last_became_active":"2024-09-15T21:19:35.736102+0000","last_became_peered":"2024-09-15T21:19:35.736102+0000","last_unstale":"2024-09-15T21:19:38.742505+0000","last_undegraded":"2024-09-15T21:19:38.742505+0000","last_fullsized":"2024-09-15T21:19:38.742505+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:44.186116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.742607+0000","last_change":"2024-09-15T21:19:28.711657+0000","last_active":"2024-09-15T21:19:38.742607+0000","last_peered":"2024-09-15T21:19:38.742607+0000","last_clean":"2024-09-15T21:19:38.742607+0000","last_became_active":"2024-09-15T21:19:26.706209+0000","last_became_peered":"2024-09-15T21:19:26.706209+0000","last_unstale":"2024-09-15T21:19:38.742607+0000","last_undegraded":"2024-09-15T21:19:38.742607+0000","last_fullsized":"2024-09-15T21:19:38.742607+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:45.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744258+0000","last_change":"2024-09-15T21:19:37.741255+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.744258+0000","last_undegraded":"2024-09-15T21:19:38.744258+0000","last_fullsized":"2024-09-15T21:19:38.744258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.743917+0000","last_change":"2024-09-15T21:19:37.743917+0000","last_active":"2024-09-15T21:19:37.743917+0000","last_peered":"2024-09-15T21:19:36.740385+0000","last_clean":"2024-09-15T21:19:36.740385+0000","last_became_active":"2024-09-15T21:19:26.707957+0000","last_became_peered":"2024-09-15T21:19:26.707957+0000","last_unstale":"2024-09-15T21:19:37.743917+0000","last_undegraded":"2024-09-15T21:19:37.743917+0000","last_fullsized":"2024-09-15T21:19:37.743917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744204+0000","last_change":"2024-09-15T21:19:37.741218+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.744204+0000","last_undegraded":"2024-09-15T21:19:38.744204+0000","last_fullsized":"2024-09-15T21:19:38.744204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745204+0000","last_change":"2024-09-15T21:19:35.740749+0000","last_active":"2024-09-15T21:19:38.745204+0000","last_peered":"2024-09-15T21:19:38.745204+0000","last_clean":"2024-09-15T21:19:38.745204+0000","last_became_active":"2024-09-15T21:19:35.740462+0000","last_became_peered":"2024-09-15T21:19:35.740462+0000","last_unstale":"2024-09-15T21:19:38.745204+0000","last_undegraded":"2024-09-15T21:19:38.745204+0000","last_fullsized":"2024-09-15T21:19:38.745204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:49.440175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744969+0000","last_change":"2024-09-15T21:19:28.712419+0000","last_active":"2024-09-15T21:19:38.744969+0000","last_peered":"2024-09-15T21:19:38.744969+0000","last_clean":"2024-09-15T21:19:38.744969+0000","last_became_active":"2024-09-15T21:19:26.705010+0000","last_became_peered":"2024-09-15T21:19:26.705010+0000","last_unstale":"2024-09-15T21:19:38.744969+0000","last_undegraded":"2024-09-15T21:19:38.744969+0000","last_fullsized":"2024-09-15T21:19:38.744969+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:58.295608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745013+0000","last_change":"2024-09-15T21:19:35.741310+0000","last_active":"2024-09-15T21:19:38.745013+0000","last_peered":"2024-09-15T21:19:38.745013+0000","last_clean":"2024-09-15T21:19:38.745013+0000","last_became_active":"2024-09-15T21:19:35.741015+0000","last_became_peered":"2024-09-15T21:19:35.741015+0000","last_unstale":"2024-09-15T21:19:38.745013+0000","last_undegraded":"2024-09-15T21:19:38.745013+0000","last_fullsized":"2024-09-15T21:19:38.745013+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:53:24.766090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.743517+0000","last_change":"2024-09-15T21:19:37.742310+0000","last_active":"2024-09-15T21:19:25.689420+0000","last_peered":"2024-09-15T21:19:25.689420+0000","last_clean":"2024-09-15T21:19:25.689420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.743517+0000","last_undegraded":"2024-09-15T21:19:38.743517+0000","last_fullsized":"2024-09-15T21:19:38.743517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745447+0000","last_change":"2024-09-15T21:19:35.741013+0000","last_active":"2024-09-15T21:19:38.745447+0000","last_peered":"2024-09-15T21:19:38.745447+0000","last_clean":"2024-09-15T21:19:38.745447+0000","last_became_active":"2024-09-15T21:19:35.740206+0000","last_became_peered":"2024-09-15T21:19:35.740206+0000","last_unstale":"2024-09-15T21:19:38.745447+0000","last_undegraded":"2024-09-15T21:19:38.745447+0000","last_fullsized":"2024-09-15T21:19:38.745447+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:37.768461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745102+0000","last_change":"2024-09-15T21:19:28.712635+0000","last_active":"2024-09-15T21:19:38.745102+0000","last_peered":"2024-09-15T21:19:38.745102+0000","last_clean":"2024-09-15T21:19:38.745102+0000","last_became_active":"2024-09-15T21:19:26.705751+0000","last_became_peered":"2024-09-15T21:19:26.705751+0000","last_unstale":"2024-09-15T21:19:38.745102+0000","last_undegraded":"2024-09-15T21:19:38.745102+0000","last_fullsized":"2024-09-15T21:19:38.745102+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:12.537829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.744483+0000","last_change":"2024-09-15T21:19:37.744483+0000","last_active":"2024-09-15T21:19:37.744483+0000","last_peered":"2024-09-15T21:19:36.741546+0000","last_clean":"2024-09-15T21:19:36.741546+0000","last_became_active":"2024-09-15T21:19:26.707803+0000","last_became_peered":"2024-09-15T21:19:26.707803+0000","last_unstale":"2024-09-15T21:19:37.744483+0000","last_undegraded":"2024-09-15T21:19:37.744483+0000","last_fullsized":"2024-09-15T21:19:37.744483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.808745+0000","last_change":"2024-09-15T21:19:35.740618+0000","last_active":"2024-09-15T21:19:38.808745+0000","last_peered":"2024-09-15T21:19:38.808745+0000","last_clean":"2024-09-15T21:19:38.808745+0000","last_became_active":"2024-09-15T21:19:35.740282+0000","last_became_peered":"2024-09-15T21:19:35.740282+0000","last_unstale":"2024-09-15T21:19:38.808745+0000","last_undegraded":"2024-09-15T21:19:38.808745+0000","last_fullsized":"2024-09-15T21:19:38.808745+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:01.218654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":167936,"data_stored":48776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":180224,"data_stored":52451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:42.264 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:42.542 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:42.543 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:19:42.555 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-15T21:19:40.669717+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54408,"kb_used_data":340,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689272,"statfs":{"total":193273528320,"available":193217814528,"internally_reserved":0,"allocated":348160,"data_stored":101227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000810"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.742617+0000","last_change":"2024-09-15T21:19:35.736448+0000","last_active":"2024-09-15T21:19:38.742617+0000","last_peered":"2024-09-15T21:19:38.742617+0000","last_clean":"2024-09-15T21:19:38.742617+0000","last_became_active":"2024-09-15T21:19:35.736099+0000","last_became_peered":"2024-09-15T21:19:35.736099+0000","last_unstale":"2024-09-15T21:19:38.742617+0000","last_undegraded":"2024-09-15T21:19:38.742617+0000","last_fullsized":"2024-09-15T21:19:38.742617+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:18.340741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.743370+0000","last_change":"2024-09-15T21:19:37.742118+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.743370+0000","last_undegraded":"2024-09-15T21:19:38.743370+0000","last_fullsized":"2024-09-15T21:19:38.743370+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.745563+0000","last_change":"2024-09-15T21:19:37.743880+0000","last_active":"2024-09-15T21:19:37.743880+0000","last_peered":"2024-09-15T21:19:36.740990+0000","last_clean":"2024-09-15T21:19:36.740990+0000","last_became_active":"2024-09-15T21:19:35.742040+0000","last_became_peered":"2024-09-15T21:19:35.742040+0000","last_unstale":"2024-09-15T21:19:38.745563+0000","last_undegraded":"2024-09-15T21:19:38.745563+0000","last_fullsized":"2024-09-15T21:19:38.745563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.743250+0000","last_change":"2024-09-15T21:19:37.742063+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.743250+0000","last_undegraded":"2024-09-15T21:19:38.743250+0000","last_fullsized":"2024-09-15T21:19:38.743250+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.742226+0000","last_change":"2024-09-15T21:19:37.742226+0000","last_active":"2024-09-15T21:19:37.742226+0000","last_peered":"2024-09-15T21:19:36.741108+0000","last_clean":"2024-09-15T21:19:36.741108+0000","last_became_active":"2024-09-15T21:19:35.730478+0000","last_became_peered":"2024-09-15T21:19:35.730478+0000","last_unstale":"2024-09-15T21:19:37.742226+0000","last_undegraded":"2024-09-15T21:19:37.742226+0000","last_fullsized":"2024-09-15T21:19:37.742226+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744403+0000","last_change":"2024-09-15T21:19:35.740110+0000","last_active":"2024-09-15T21:19:38.744403+0000","last_peered":"2024-09-15T21:19:38.744403+0000","last_clean":"2024-09-15T21:19:38.744403+0000","last_became_active":"2024-09-15T21:19:35.739773+0000","last_became_peered":"2024-09-15T21:19:35.739773+0000","last_unstale":"2024-09-15T21:19:38.744403+0000","last_undegraded":"2024-09-15T21:19:38.744403+0000","last_fullsized":"2024-09-15T21:19:38.744403+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:49:46.277839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744485+0000","last_change":"2024-09-15T21:19:35.740711+0000","last_active":"2024-09-15T21:19:38.744485+0000","last_peered":"2024-09-15T21:19:38.744485+0000","last_clean":"2024-09-15T21:19:38.744485+0000","last_became_active":"2024-09-15T21:19:35.740436+0000","last_became_peered":"2024-09-15T21:19:35.740436+0000","last_unstale":"2024-09-15T21:19:38.744485+0000","last_undegraded":"2024-09-15T21:19:38.744485+0000","last_fullsized":"2024-09-15T21:19:38.744485+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:56.312225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.741258+0000","last_change":"2024-09-15T21:19:37.741258+0000","last_active":"2024-09-15T21:19:37.741258+0000","last_peered":"2024-09-15T21:19:36.740313+0000","last_clean":"2024-09-15T21:19:36.740313+0000","last_became_active":"2024-09-15T21:19:35.730414+0000","last_became_peered":"2024-09-15T21:19:35.730414+0000","last_unstale":"2024-09-15T21:19:37.741258+0000","last_undegraded":"2024-09-15T21:19:37.741258+0000","last_fullsized":"2024-09-15T21:19:37.741258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.743874+0000","last_change":"2024-09-15T21:19:28.711561+0000","last_active":"2024-09-15T21:19:38.743874+0000","last_peered":"2024-09-15T21:19:38.743874+0000","last_clean":"2024-09-15T21:19:38.743874+0000","last_became_active":"2024-09-15T21:19:26.704834+0000","last_became_peered":"2024-09-15T21:19:26.704834+0000","last_unstale":"2024-09-15T21:19:38.743874+0000","last_undegraded":"2024-09-15T21:19:38.743874+0000","last_fullsized":"2024-09-15T21:19:38.743874+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:09.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744905+0000","last_change":"2024-09-15T21:19:37.743194+0000","last_active":"2024-09-15T21:19:37.743194+0000","last_peered":"2024-09-15T21:19:36.740191+0000","last_clean":"2024-09-15T21:19:36.740191+0000","last_became_active":"2024-09-15T21:19:35.741860+0000","last_became_peered":"2024-09-15T21:19:35.741860+0000","last_unstale":"2024-09-15T21:19:38.744905+0000","last_undegraded":"2024-09-15T21:19:38.744905+0000","last_fullsized":"2024-09-15T21:19:38.744905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.742560+0000","last_change":"2024-09-15T21:19:37.741558+0000","last_active":"2024-09-15T21:19:25.689420+0000","last_peered":"2024-09-15T21:19:25.689420+0000","last_clean":"2024-09-15T21:19:25.689420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.742560+0000","last_undegraded":"2024-09-15T21:19:38.742560+0000","last_fullsized":"2024-09-15T21:19:38.742560+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.742505+0000","last_change":"2024-09-15T21:19:35.736396+0000","last_active":"2024-09-15T21:19:38.742505+0000","last_peered":"2024-09-15T21:19:38.742505+0000","last_clean":"2024-09-15T21:19:38.742505+0000","last_became_active":"2024-09-15T21:19:35.736102+0000","last_became_peered":"2024-09-15T21:19:35.736102+0000","last_unstale":"2024-09-15T21:19:38.742505+0000","last_undegraded":"2024-09-15T21:19:38.742505+0000","last_fullsized":"2024-09-15T21:19:38.742505+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:44.186116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.742607+0000","last_change":"2024-09-15T21:19:28.711657+0000","last_active":"2024-09-15T21:19:38.742607+0000","last_peered":"2024-09-15T21:19:38.742607+0000","last_clean":"2024-09-15T21:19:38.742607+0000","last_became_active":"2024-09-15T21:19:26.706209+0000","last_became_peered":"2024-09-15T21:19:26.706209+0000","last_unstale":"2024-09-15T21:19:38.742607+0000","last_undegraded":"2024-09-15T21:19:38.742607+0000","last_fullsized":"2024-09-15T21:19:38.742607+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:45.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744258+0000","last_change":"2024-09-15T21:19:37.741255+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.744258+0000","last_undegraded":"2024-09-15T21:19:38.744258+0000","last_fullsized":"2024-09-15T21:19:38.744258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.743917+0000","last_change":"2024-09-15T21:19:37.743917+0000","last_active":"2024-09-15T21:19:37.743917+0000","last_peered":"2024-09-15T21:19:36.740385+0000","last_clean":"2024-09-15T21:19:36.740385+0000","last_became_active":"2024-09-15T21:19:26.707957+0000","last_became_peered":"2024-09-15T21:19:26.707957+0000","last_unstale":"2024-09-15T21:19:37.743917+0000","last_undegraded":"2024-09-15T21:19:37.743917+0000","last_fullsized":"2024-09-15T21:19:37.743917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744204+0000","last_change":"2024-09-15T21:19:37.741218+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.744204+0000","last_undegraded":"2024-09-15T21:19:38.744204+0000","last_fullsized":"2024-09-15T21:19:38.744204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745204+0000","last_change":"2024-09-15T21:19:35.740749+0000","last_active":"2024-09-15T21:19:38.745204+0000","last_peered":"2024-09-15T21:19:38.745204+0000","last_clean":"2024-09-15T21:19:38.745204+0000","last_became_active":"2024-09-15T21:19:35.740462+0000","last_became_peered":"2024-09-15T21:19:35.740462+0000","last_unstale":"2024-09-15T21:19:38.745204+0000","last_undegraded":"2024-09-15T21:19:38.745204+0000","last_fullsized":"2024-09-15T21:19:38.745204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:49.440175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744969+0000","last_change":"2024-09-15T21:19:28.712419+0000","last_active":"2024-09-15T21:19:38.744969+0000","last_peered":"2024-09-15T21:19:38.744969+0000","last_clean":"2024-09-15T21:19:38.744969+0000","last_became_active":"2024-09-15T21:19:26.705010+0000","last_became_peered":"2024-09-15T21:19:26.705010+0000","last_unstale":"2024-09-15T21:19:38.744969+0000","last_undegraded":"2024-09-15T21:19:38.744969+0000","last_fullsized":"2024-09-15T21:19:38.744969+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:58.295608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745013+0000","last_change":"2024-09-15T21:19:35.741310+0000","last_active":"2024-09-15T21:19:38.745013+0000","last_peered":"2024-09-15T21:19:38.745013+0000","last_clean":"2024-09-15T21:19:38.745013+0000","last_became_active":"2024-09-15T21:19:35.741015+0000","last_became_peered":"2024-09-15T21:19:35.741015+0000","last_unstale":"2024-09-15T21:19:38.745013+0000","last_undegraded":"2024-09-15T21:19:38.745013+0000","last_fullsized":"2024-09-15T21:19:38.745013+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:53:24.766090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.743517+0000","last_change":"2024-09-15T21:19:37.742310+0000","last_active":"2024-09-15T21:19:25.689420+0000","last_peered":"2024-09-15T21:19:25.689420+0000","last_clean":"2024-09-15T21:19:25.689420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.743517+0000","last_undegraded":"2024-09-15T21:19:38.743517+0000","last_fullsized":"2024-09-15T21:19:38.743517+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745447+0000","last_change":"2024-09-15T21:19:35.741013+0000","last_active":"2024-09-15T21:19:38.745447+0000","last_peered":"2024-09-15T21:19:38.745447+0000","last_clean":"2024-09-15T21:19:38.745447+0000","last_became_active":"2024-09-15T21:19:35.740206+0000","last_became_peered":"2024-09-15T21:19:35.740206+0000","last_unstale":"2024-09-15T21:19:38.745447+0000","last_undegraded":"2024-09-15T21:19:38.745447+0000","last_fullsized":"2024-09-15T21:19:38.745447+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:37.768461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745102+0000","last_change":"2024-09-15T21:19:28.712635+0000","last_active":"2024-09-15T21:19:38.745102+0000","last_peered":"2024-09-15T21:19:38.745102+0000","last_clean":"2024-09-15T21:19:38.745102+0000","last_became_active":"2024-09-15T21:19:26.705751+0000","last_became_peered":"2024-09-15T21:19:26.705751+0000","last_unstale":"2024-09-15T21:19:38.745102+0000","last_undegraded":"2024-09-15T21:19:38.745102+0000","last_fullsized":"2024-09-15T21:19:38.745102+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:12.537829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.744483+0000","last_change":"2024-09-15T21:19:37.744483+0000","last_active":"2024-09-15T21:19:37.744483+0000","last_peered":"2024-09-15T21:19:36.741546+0000","last_clean":"2024-09-15T21:19:36.741546+0000","last_became_active":"2024-09-15T21:19:26.707803+0000","last_became_peered":"2024-09-15T21:19:26.707803+0000","last_unstale":"2024-09-15T21:19:37.744483+0000","last_undegraded":"2024-09-15T21:19:37.744483+0000","last_fullsized":"2024-09-15T21:19:37.744483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.808745+0000","last_change":"2024-09-15T21:19:35.740618+0000","last_active":"2024-09-15T21:19:38.808745+0000","last_peered":"2024-09-15T21:19:38.808745+0000","last_clean":"2024-09-15T21:19:38.808745+0000","last_became_active":"2024-09-15T21:19:35.740282+0000","last_became_peered":"2024-09-15T21:19:35.740282+0000","last_unstale":"2024-09-15T21:19:38.808745+0000","last_undegraded":"2024-09-15T21:19:38.808745+0000","last_fullsized":"2024-09-15T21:19:38.808745+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:01.218654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27200,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344640,"statfs":{"total":96636764160,"available":96608911360,"internally_reserved":0,"allocated":167936,"data_stored":48776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":180224,"data_stored":52451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:42.556 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:42.837 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:42.837 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:19:42.849 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-15T21:19:42.669936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54424,"kb_used_data":360,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689256,"statfs":{"total":193273528320,"available":193217798144,"internally_reserved":0,"allocated":368640,"data_stored":108036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.931339"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.743101+0000","last_change":"2024-09-15T21:19:35.736448+0000","last_active":"2024-09-15T21:19:39.743101+0000","last_peered":"2024-09-15T21:19:39.743101+0000","last_clean":"2024-09-15T21:19:39.743101+0000","last_became_active":"2024-09-15T21:19:35.736099+0000","last_became_peered":"2024-09-15T21:19:35.736099+0000","last_unstale":"2024-09-15T21:19:39.743101+0000","last_undegraded":"2024-09-15T21:19:39.743101+0000","last_fullsized":"2024-09-15T21:19:39.743101+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:18.340741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.747208+0000","last_change":"2024-09-15T21:19:39.747139+0000","last_active":"2024-09-15T21:19:39.747208+0000","last_peered":"2024-09-15T21:19:39.747208+0000","last_clean":"2024-09-15T21:19:39.747208+0000","last_became_active":"2024-09-15T21:19:39.746905+0000","last_became_peered":"2024-09-15T21:19:39.746905+0000","last_unstale":"2024-09-15T21:19:39.747208+0000","last_undegraded":"2024-09-15T21:19:39.747208+0000","last_fullsized":"2024-09-15T21:19:39.747208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:46.017048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.745563+0000","last_change":"2024-09-15T21:19:37.743880+0000","last_active":"2024-09-15T21:19:37.743880+0000","last_peered":"2024-09-15T21:19:36.740990+0000","last_clean":"2024-09-15T21:19:36.740990+0000","last_became_active":"2024-09-15T21:19:35.742040+0000","last_became_peered":"2024-09-15T21:19:35.742040+0000","last_unstale":"2024-09-15T21:19:38.745563+0000","last_undegraded":"2024-09-15T21:19:38.745563+0000","last_fullsized":"2024-09-15T21:19:38.745563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.748081+0000","last_change":"2024-09-15T21:19:39.747971+0000","last_active":"2024-09-15T21:19:39.748081+0000","last_peered":"2024-09-15T21:19:39.748081+0000","last_clean":"2024-09-15T21:19:39.748081+0000","last_became_active":"2024-09-15T21:19:39.747670+0000","last_became_peered":"2024-09-15T21:19:39.747670+0000","last_unstale":"2024-09-15T21:19:39.748081+0000","last_undegraded":"2024-09-15T21:19:39.748081+0000","last_fullsized":"2024-09-15T21:19:39.748081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:00.689805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.748025+0000","last_change":"2024-09-15T21:19:39.747922+0000","last_active":"2024-09-15T21:19:39.748025+0000","last_peered":"2024-09-15T21:19:39.748025+0000","last_clean":"2024-09-15T21:19:39.748025+0000","last_became_active":"2024-09-15T21:19:39.747606+0000","last_became_peered":"2024-09-15T21:19:39.747606+0000","last_unstale":"2024-09-15T21:19:39.748025+0000","last_undegraded":"2024-09-15T21:19:39.748025+0000","last_fullsized":"2024-09-15T21:19:39.748025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:28.550257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744403+0000","last_change":"2024-09-15T21:19:35.740110+0000","last_active":"2024-09-15T21:19:38.744403+0000","last_peered":"2024-09-15T21:19:38.744403+0000","last_clean":"2024-09-15T21:19:38.744403+0000","last_became_active":"2024-09-15T21:19:35.739773+0000","last_became_peered":"2024-09-15T21:19:35.739773+0000","last_unstale":"2024-09-15T21:19:38.744403+0000","last_undegraded":"2024-09-15T21:19:38.744403+0000","last_fullsized":"2024-09-15T21:19:38.744403+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:49:46.277839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744485+0000","last_change":"2024-09-15T21:19:35.740711+0000","last_active":"2024-09-15T21:19:38.744485+0000","last_peered":"2024-09-15T21:19:38.744485+0000","last_clean":"2024-09-15T21:19:38.744485+0000","last_became_active":"2024-09-15T21:19:35.740436+0000","last_became_peered":"2024-09-15T21:19:35.740436+0000","last_unstale":"2024-09-15T21:19:38.744485+0000","last_undegraded":"2024-09-15T21:19:38.744485+0000","last_fullsized":"2024-09-15T21:19:38.744485+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:56.312225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.747984+0000","last_change":"2024-09-15T21:19:39.747882+0000","last_active":"2024-09-15T21:19:39.747984+0000","last_peered":"2024-09-15T21:19:39.747984+0000","last_clean":"2024-09-15T21:19:39.747984+0000","last_became_active":"2024-09-15T21:19:39.747601+0000","last_became_peered":"2024-09-15T21:19:39.747601+0000","last_unstale":"2024-09-15T21:19:39.747984+0000","last_undegraded":"2024-09-15T21:19:39.747984+0000","last_fullsized":"2024-09-15T21:19:39.747984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:03.580034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.743874+0000","last_change":"2024-09-15T21:19:28.711561+0000","last_active":"2024-09-15T21:19:38.743874+0000","last_peered":"2024-09-15T21:19:38.743874+0000","last_clean":"2024-09-15T21:19:38.743874+0000","last_became_active":"2024-09-15T21:19:26.704834+0000","last_became_peered":"2024-09-15T21:19:26.704834+0000","last_unstale":"2024-09-15T21:19:38.743874+0000","last_undegraded":"2024-09-15T21:19:38.743874+0000","last_fullsized":"2024-09-15T21:19:38.743874+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:09.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744905+0000","last_change":"2024-09-15T21:19:37.743194+0000","last_active":"2024-09-15T21:19:37.743194+0000","last_peered":"2024-09-15T21:19:36.740191+0000","last_clean":"2024-09-15T21:19:36.740191+0000","last_became_active":"2024-09-15T21:19:35.741860+0000","last_became_peered":"2024-09-15T21:19:35.741860+0000","last_unstale":"2024-09-15T21:19:38.744905+0000","last_undegraded":"2024-09-15T21:19:38.744905+0000","last_fullsized":"2024-09-15T21:19:38.744905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.748007+0000","last_change":"2024-09-15T21:19:39.747909+0000","last_active":"2024-09-15T21:19:39.748007+0000","last_peered":"2024-09-15T21:19:39.748007+0000","last_clean":"2024-09-15T21:19:39.748007+0000","last_became_active":"2024-09-15T21:19:39.747613+0000","last_became_peered":"2024-09-15T21:19:39.747613+0000","last_unstale":"2024-09-15T21:19:39.748007+0000","last_undegraded":"2024-09-15T21:19:39.748007+0000","last_fullsized":"2024-09-15T21:19:39.748007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:50.363505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.743142+0000","last_change":"2024-09-15T21:19:35.736396+0000","last_active":"2024-09-15T21:19:39.743142+0000","last_peered":"2024-09-15T21:19:39.743142+0000","last_clean":"2024-09-15T21:19:39.743142+0000","last_became_active":"2024-09-15T21:19:35.736102+0000","last_became_peered":"2024-09-15T21:19:35.736102+0000","last_unstale":"2024-09-15T21:19:39.743142+0000","last_undegraded":"2024-09-15T21:19:39.743142+0000","last_fullsized":"2024-09-15T21:19:39.743142+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:44.186116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":36,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.743238+0000","last_change":"2024-09-15T21:19:28.711657+0000","last_active":"2024-09-15T21:19:39.743238+0000","last_peered":"2024-09-15T21:19:39.743238+0000","last_clean":"2024-09-15T21:19:39.743238+0000","last_became_active":"2024-09-15T21:19:26.706209+0000","last_became_peered":"2024-09-15T21:19:26.706209+0000","last_unstale":"2024-09-15T21:19:39.743238+0000","last_undegraded":"2024-09-15T21:19:39.743238+0000","last_fullsized":"2024-09-15T21:19:39.743238+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:45.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744258+0000","last_change":"2024-09-15T21:19:37.741255+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.744258+0000","last_undegraded":"2024-09-15T21:19:38.744258+0000","last_fullsized":"2024-09-15T21:19:38.744258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.743917+0000","last_change":"2024-09-15T21:19:37.743917+0000","last_active":"2024-09-15T21:19:37.743917+0000","last_peered":"2024-09-15T21:19:36.740385+0000","last_clean":"2024-09-15T21:19:36.740385+0000","last_became_active":"2024-09-15T21:19:26.707957+0000","last_became_peered":"2024-09-15T21:19:26.707957+0000","last_unstale":"2024-09-15T21:19:37.743917+0000","last_undegraded":"2024-09-15T21:19:37.743917+0000","last_fullsized":"2024-09-15T21:19:37.743917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744204+0000","last_change":"2024-09-15T21:19:37.741218+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.744204+0000","last_undegraded":"2024-09-15T21:19:38.744204+0000","last_fullsized":"2024-09-15T21:19:38.744204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745204+0000","last_change":"2024-09-15T21:19:35.740749+0000","last_active":"2024-09-15T21:19:38.745204+0000","last_peered":"2024-09-15T21:19:38.745204+0000","last_clean":"2024-09-15T21:19:38.745204+0000","last_became_active":"2024-09-15T21:19:35.740462+0000","last_became_peered":"2024-09-15T21:19:35.740462+0000","last_unstale":"2024-09-15T21:19:38.745204+0000","last_undegraded":"2024-09-15T21:19:38.745204+0000","last_fullsized":"2024-09-15T21:19:38.745204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:49.440175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744969+0000","last_change":"2024-09-15T21:19:28.712419+0000","last_active":"2024-09-15T21:19:38.744969+0000","last_peered":"2024-09-15T21:19:38.744969+0000","last_clean":"2024-09-15T21:19:38.744969+0000","last_became_active":"2024-09-15T21:19:26.705010+0000","last_became_peered":"2024-09-15T21:19:26.705010+0000","last_unstale":"2024-09-15T21:19:38.744969+0000","last_undegraded":"2024-09-15T21:19:38.744969+0000","last_fullsized":"2024-09-15T21:19:38.744969+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:58.295608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745013+0000","last_change":"2024-09-15T21:19:35.741310+0000","last_active":"2024-09-15T21:19:38.745013+0000","last_peered":"2024-09-15T21:19:38.745013+0000","last_clean":"2024-09-15T21:19:38.745013+0000","last_became_active":"2024-09-15T21:19:35.741015+0000","last_became_peered":"2024-09-15T21:19:35.741015+0000","last_unstale":"2024-09-15T21:19:38.745013+0000","last_undegraded":"2024-09-15T21:19:38.745013+0000","last_fullsized":"2024-09-15T21:19:38.745013+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:53:24.766090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.746494+0000","last_change":"2024-09-15T21:19:39.746403+0000","last_active":"2024-09-15T21:19:39.746494+0000","last_peered":"2024-09-15T21:19:39.746494+0000","last_clean":"2024-09-15T21:19:39.746494+0000","last_became_active":"2024-09-15T21:19:39.746038+0000","last_became_peered":"2024-09-15T21:19:39.746038+0000","last_unstale":"2024-09-15T21:19:39.746494+0000","last_undegraded":"2024-09-15T21:19:39.746494+0000","last_fullsized":"2024-09-15T21:19:39.746494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:44.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745447+0000","last_change":"2024-09-15T21:19:35.741013+0000","last_active":"2024-09-15T21:19:38.745447+0000","last_peered":"2024-09-15T21:19:38.745447+0000","last_clean":"2024-09-15T21:19:38.745447+0000","last_became_active":"2024-09-15T21:19:35.740206+0000","last_became_peered":"2024-09-15T21:19:35.740206+0000","last_unstale":"2024-09-15T21:19:38.745447+0000","last_undegraded":"2024-09-15T21:19:38.745447+0000","last_fullsized":"2024-09-15T21:19:38.745447+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:37.768461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745102+0000","last_change":"2024-09-15T21:19:28.712635+0000","last_active":"2024-09-15T21:19:38.745102+0000","last_peered":"2024-09-15T21:19:38.745102+0000","last_clean":"2024-09-15T21:19:38.745102+0000","last_became_active":"2024-09-15T21:19:26.705751+0000","last_became_peered":"2024-09-15T21:19:26.705751+0000","last_unstale":"2024-09-15T21:19:38.745102+0000","last_undegraded":"2024-09-15T21:19:38.745102+0000","last_fullsized":"2024-09-15T21:19:38.745102+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:12.537829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.744483+0000","last_change":"2024-09-15T21:19:37.744483+0000","last_active":"2024-09-15T21:19:37.744483+0000","last_peered":"2024-09-15T21:19:36.741546+0000","last_clean":"2024-09-15T21:19:36.741546+0000","last_became_active":"2024-09-15T21:19:26.707803+0000","last_became_peered":"2024-09-15T21:19:26.707803+0000","last_unstale":"2024-09-15T21:19:37.744483+0000","last_undegraded":"2024-09-15T21:19:37.744483+0000","last_fullsized":"2024-09-15T21:19:37.744483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.808745+0000","last_change":"2024-09-15T21:19:35.740618+0000","last_active":"2024-09-15T21:19:38.808745+0000","last_peered":"2024-09-15T21:19:38.808745+0000","last_clean":"2024-09-15T21:19:38.808745+0000","last_became_active":"2024-09-15T21:19:35.740282+0000","last_became_peered":"2024-09-15T21:19:35.740282+0000","last_unstale":"2024-09-15T21:19:38.808745+0000","last_undegraded":"2024-09-15T21:19:38.808745+0000","last_fullsized":"2024-09-15T21:19:38.808745+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:01.218654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705673,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":55585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":180224,"data_stored":52451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:45.852 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:46.143 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:46.144 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:19:46.156 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-15T21:19:44.670252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54424,"kb_used_data":360,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689256,"statfs":{"total":193273528320,"available":193217798144,"internally_reserved":0,"allocated":368640,"data_stored":108036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.930562"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.743101+0000","last_change":"2024-09-15T21:19:35.736448+0000","last_active":"2024-09-15T21:19:39.743101+0000","last_peered":"2024-09-15T21:19:39.743101+0000","last_clean":"2024-09-15T21:19:39.743101+0000","last_became_active":"2024-09-15T21:19:35.736099+0000","last_became_peered":"2024-09-15T21:19:35.736099+0000","last_unstale":"2024-09-15T21:19:39.743101+0000","last_undegraded":"2024-09-15T21:19:39.743101+0000","last_fullsized":"2024-09-15T21:19:39.743101+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:18.340741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.747208+0000","last_change":"2024-09-15T21:19:39.747139+0000","last_active":"2024-09-15T21:19:39.747208+0000","last_peered":"2024-09-15T21:19:39.747208+0000","last_clean":"2024-09-15T21:19:39.747208+0000","last_became_active":"2024-09-15T21:19:39.746905+0000","last_became_peered":"2024-09-15T21:19:39.746905+0000","last_unstale":"2024-09-15T21:19:39.747208+0000","last_undegraded":"2024-09-15T21:19:39.747208+0000","last_fullsized":"2024-09-15T21:19:39.747208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:46.017048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.745563+0000","last_change":"2024-09-15T21:19:37.743880+0000","last_active":"2024-09-15T21:19:37.743880+0000","last_peered":"2024-09-15T21:19:36.740990+0000","last_clean":"2024-09-15T21:19:36.740990+0000","last_became_active":"2024-09-15T21:19:35.742040+0000","last_became_peered":"2024-09-15T21:19:35.742040+0000","last_unstale":"2024-09-15T21:19:38.745563+0000","last_undegraded":"2024-09-15T21:19:38.745563+0000","last_fullsized":"2024-09-15T21:19:38.745563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.748081+0000","last_change":"2024-09-15T21:19:39.747971+0000","last_active":"2024-09-15T21:19:39.748081+0000","last_peered":"2024-09-15T21:19:39.748081+0000","last_clean":"2024-09-15T21:19:39.748081+0000","last_became_active":"2024-09-15T21:19:39.747670+0000","last_became_peered":"2024-09-15T21:19:39.747670+0000","last_unstale":"2024-09-15T21:19:39.748081+0000","last_undegraded":"2024-09-15T21:19:39.748081+0000","last_fullsized":"2024-09-15T21:19:39.748081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:00.689805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.748025+0000","last_change":"2024-09-15T21:19:39.747922+0000","last_active":"2024-09-15T21:19:39.748025+0000","last_peered":"2024-09-15T21:19:39.748025+0000","last_clean":"2024-09-15T21:19:39.748025+0000","last_became_active":"2024-09-15T21:19:39.747606+0000","last_became_peered":"2024-09-15T21:19:39.747606+0000","last_unstale":"2024-09-15T21:19:39.748025+0000","last_undegraded":"2024-09-15T21:19:39.748025+0000","last_fullsized":"2024-09-15T21:19:39.748025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:28.550257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744403+0000","last_change":"2024-09-15T21:19:35.740110+0000","last_active":"2024-09-15T21:19:38.744403+0000","last_peered":"2024-09-15T21:19:38.744403+0000","last_clean":"2024-09-15T21:19:38.744403+0000","last_became_active":"2024-09-15T21:19:35.739773+0000","last_became_peered":"2024-09-15T21:19:35.739773+0000","last_unstale":"2024-09-15T21:19:38.744403+0000","last_undegraded":"2024-09-15T21:19:38.744403+0000","last_fullsized":"2024-09-15T21:19:38.744403+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:49:46.277839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744485+0000","last_change":"2024-09-15T21:19:35.740711+0000","last_active":"2024-09-15T21:19:38.744485+0000","last_peered":"2024-09-15T21:19:38.744485+0000","last_clean":"2024-09-15T21:19:38.744485+0000","last_became_active":"2024-09-15T21:19:35.740436+0000","last_became_peered":"2024-09-15T21:19:35.740436+0000","last_unstale":"2024-09-15T21:19:38.744485+0000","last_undegraded":"2024-09-15T21:19:38.744485+0000","last_fullsized":"2024-09-15T21:19:38.744485+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:56.312225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.747984+0000","last_change":"2024-09-15T21:19:39.747882+0000","last_active":"2024-09-15T21:19:39.747984+0000","last_peered":"2024-09-15T21:19:39.747984+0000","last_clean":"2024-09-15T21:19:39.747984+0000","last_became_active":"2024-09-15T21:19:39.747601+0000","last_became_peered":"2024-09-15T21:19:39.747601+0000","last_unstale":"2024-09-15T21:19:39.747984+0000","last_undegraded":"2024-09-15T21:19:39.747984+0000","last_fullsized":"2024-09-15T21:19:39.747984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:03.580034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.743874+0000","last_change":"2024-09-15T21:19:28.711561+0000","last_active":"2024-09-15T21:19:38.743874+0000","last_peered":"2024-09-15T21:19:38.743874+0000","last_clean":"2024-09-15T21:19:38.743874+0000","last_became_active":"2024-09-15T21:19:26.704834+0000","last_became_peered":"2024-09-15T21:19:26.704834+0000","last_unstale":"2024-09-15T21:19:38.743874+0000","last_undegraded":"2024-09-15T21:19:38.743874+0000","last_fullsized":"2024-09-15T21:19:38.743874+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:09.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744905+0000","last_change":"2024-09-15T21:19:37.743194+0000","last_active":"2024-09-15T21:19:37.743194+0000","last_peered":"2024-09-15T21:19:36.740191+0000","last_clean":"2024-09-15T21:19:36.740191+0000","last_became_active":"2024-09-15T21:19:35.741860+0000","last_became_peered":"2024-09-15T21:19:35.741860+0000","last_unstale":"2024-09-15T21:19:38.744905+0000","last_undegraded":"2024-09-15T21:19:38.744905+0000","last_fullsized":"2024-09-15T21:19:38.744905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.748007+0000","last_change":"2024-09-15T21:19:39.747909+0000","last_active":"2024-09-15T21:19:39.748007+0000","last_peered":"2024-09-15T21:19:39.748007+0000","last_clean":"2024-09-15T21:19:39.748007+0000","last_became_active":"2024-09-15T21:19:39.747613+0000","last_became_peered":"2024-09-15T21:19:39.747613+0000","last_unstale":"2024-09-15T21:19:39.748007+0000","last_undegraded":"2024-09-15T21:19:39.748007+0000","last_fullsized":"2024-09-15T21:19:39.748007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:50.363505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.743142+0000","last_change":"2024-09-15T21:19:35.736396+0000","last_active":"2024-09-15T21:19:39.743142+0000","last_peered":"2024-09-15T21:19:39.743142+0000","last_clean":"2024-09-15T21:19:39.743142+0000","last_became_active":"2024-09-15T21:19:35.736102+0000","last_became_peered":"2024-09-15T21:19:35.736102+0000","last_unstale":"2024-09-15T21:19:39.743142+0000","last_undegraded":"2024-09-15T21:19:39.743142+0000","last_fullsized":"2024-09-15T21:19:39.743142+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:44.186116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":36,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.743238+0000","last_change":"2024-09-15T21:19:28.711657+0000","last_active":"2024-09-15T21:19:39.743238+0000","last_peered":"2024-09-15T21:19:39.743238+0000","last_clean":"2024-09-15T21:19:39.743238+0000","last_became_active":"2024-09-15T21:19:26.706209+0000","last_became_peered":"2024-09-15T21:19:26.706209+0000","last_unstale":"2024-09-15T21:19:39.743238+0000","last_undegraded":"2024-09-15T21:19:39.743238+0000","last_fullsized":"2024-09-15T21:19:39.743238+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:45.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744258+0000","last_change":"2024-09-15T21:19:37.741255+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.744258+0000","last_undegraded":"2024-09-15T21:19:38.744258+0000","last_fullsized":"2024-09-15T21:19:38.744258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.743917+0000","last_change":"2024-09-15T21:19:37.743917+0000","last_active":"2024-09-15T21:19:37.743917+0000","last_peered":"2024-09-15T21:19:36.740385+0000","last_clean":"2024-09-15T21:19:36.740385+0000","last_became_active":"2024-09-15T21:19:26.707957+0000","last_became_peered":"2024-09-15T21:19:26.707957+0000","last_unstale":"2024-09-15T21:19:37.743917+0000","last_undegraded":"2024-09-15T21:19:37.743917+0000","last_fullsized":"2024-09-15T21:19:37.743917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744204+0000","last_change":"2024-09-15T21:19:37.741218+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.744204+0000","last_undegraded":"2024-09-15T21:19:38.744204+0000","last_fullsized":"2024-09-15T21:19:38.744204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745204+0000","last_change":"2024-09-15T21:19:35.740749+0000","last_active":"2024-09-15T21:19:38.745204+0000","last_peered":"2024-09-15T21:19:38.745204+0000","last_clean":"2024-09-15T21:19:38.745204+0000","last_became_active":"2024-09-15T21:19:35.740462+0000","last_became_peered":"2024-09-15T21:19:35.740462+0000","last_unstale":"2024-09-15T21:19:38.745204+0000","last_undegraded":"2024-09-15T21:19:38.745204+0000","last_fullsized":"2024-09-15T21:19:38.745204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:49.440175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744969+0000","last_change":"2024-09-15T21:19:28.712419+0000","last_active":"2024-09-15T21:19:38.744969+0000","last_peered":"2024-09-15T21:19:38.744969+0000","last_clean":"2024-09-15T21:19:38.744969+0000","last_became_active":"2024-09-15T21:19:26.705010+0000","last_became_peered":"2024-09-15T21:19:26.705010+0000","last_unstale":"2024-09-15T21:19:38.744969+0000","last_undegraded":"2024-09-15T21:19:38.744969+0000","last_fullsized":"2024-09-15T21:19:38.744969+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:58.295608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745013+0000","last_change":"2024-09-15T21:19:35.741310+0000","last_active":"2024-09-15T21:19:38.745013+0000","last_peered":"2024-09-15T21:19:38.745013+0000","last_clean":"2024-09-15T21:19:38.745013+0000","last_became_active":"2024-09-15T21:19:35.741015+0000","last_became_peered":"2024-09-15T21:19:35.741015+0000","last_unstale":"2024-09-15T21:19:38.745013+0000","last_undegraded":"2024-09-15T21:19:38.745013+0000","last_fullsized":"2024-09-15T21:19:38.745013+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:53:24.766090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.746494+0000","last_change":"2024-09-15T21:19:39.746403+0000","last_active":"2024-09-15T21:19:39.746494+0000","last_peered":"2024-09-15T21:19:39.746494+0000","last_clean":"2024-09-15T21:19:39.746494+0000","last_became_active":"2024-09-15T21:19:39.746038+0000","last_became_peered":"2024-09-15T21:19:39.746038+0000","last_unstale":"2024-09-15T21:19:39.746494+0000","last_undegraded":"2024-09-15T21:19:39.746494+0000","last_fullsized":"2024-09-15T21:19:39.746494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:44.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745447+0000","last_change":"2024-09-15T21:19:35.741013+0000","last_active":"2024-09-15T21:19:38.745447+0000","last_peered":"2024-09-15T21:19:38.745447+0000","last_clean":"2024-09-15T21:19:38.745447+0000","last_became_active":"2024-09-15T21:19:35.740206+0000","last_became_peered":"2024-09-15T21:19:35.740206+0000","last_unstale":"2024-09-15T21:19:38.745447+0000","last_undegraded":"2024-09-15T21:19:38.745447+0000","last_fullsized":"2024-09-15T21:19:38.745447+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:37.768461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745102+0000","last_change":"2024-09-15T21:19:28.712635+0000","last_active":"2024-09-15T21:19:38.745102+0000","last_peered":"2024-09-15T21:19:38.745102+0000","last_clean":"2024-09-15T21:19:38.745102+0000","last_became_active":"2024-09-15T21:19:26.705751+0000","last_became_peered":"2024-09-15T21:19:26.705751+0000","last_unstale":"2024-09-15T21:19:38.745102+0000","last_undegraded":"2024-09-15T21:19:38.745102+0000","last_fullsized":"2024-09-15T21:19:38.745102+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:12.537829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.744483+0000","last_change":"2024-09-15T21:19:37.744483+0000","last_active":"2024-09-15T21:19:37.744483+0000","last_peered":"2024-09-15T21:19:36.741546+0000","last_clean":"2024-09-15T21:19:36.741546+0000","last_became_active":"2024-09-15T21:19:26.707803+0000","last_became_peered":"2024-09-15T21:19:26.707803+0000","last_unstale":"2024-09-15T21:19:37.744483+0000","last_undegraded":"2024-09-15T21:19:37.744483+0000","last_fullsized":"2024-09-15T21:19:37.744483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.808745+0000","last_change":"2024-09-15T21:19:35.740618+0000","last_active":"2024-09-15T21:19:38.808745+0000","last_peered":"2024-09-15T21:19:38.808745+0000","last_clean":"2024-09-15T21:19:38.808745+0000","last_became_active":"2024-09-15T21:19:35.740282+0000","last_became_peered":"2024-09-15T21:19:35.740282+0000","last_unstale":"2024-09-15T21:19:38.808745+0000","last_undegraded":"2024-09-15T21:19:38.808745+0000","last_fullsized":"2024-09-15T21:19:38.808745+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:01.218654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705673,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":55585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":180224,"data_stored":52451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:46.158 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:46.439 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:46.440 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:19:46.451 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-15T21:19:44.670252+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54424,"kb_used_data":360,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689256,"statfs":{"total":193273528320,"available":193217798144,"internally_reserved":0,"allocated":368640,"data_stored":108036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.930562"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.743101+0000","last_change":"2024-09-15T21:19:35.736448+0000","last_active":"2024-09-15T21:19:39.743101+0000","last_peered":"2024-09-15T21:19:39.743101+0000","last_clean":"2024-09-15T21:19:39.743101+0000","last_became_active":"2024-09-15T21:19:35.736099+0000","last_became_peered":"2024-09-15T21:19:35.736099+0000","last_unstale":"2024-09-15T21:19:39.743101+0000","last_undegraded":"2024-09-15T21:19:39.743101+0000","last_fullsized":"2024-09-15T21:19:39.743101+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:18.340741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.747208+0000","last_change":"2024-09-15T21:19:39.747139+0000","last_active":"2024-09-15T21:19:39.747208+0000","last_peered":"2024-09-15T21:19:39.747208+0000","last_clean":"2024-09-15T21:19:39.747208+0000","last_became_active":"2024-09-15T21:19:39.746905+0000","last_became_peered":"2024-09-15T21:19:39.746905+0000","last_unstale":"2024-09-15T21:19:39.747208+0000","last_undegraded":"2024-09-15T21:19:39.747208+0000","last_fullsized":"2024-09-15T21:19:39.747208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:46.017048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.745563+0000","last_change":"2024-09-15T21:19:37.743880+0000","last_active":"2024-09-15T21:19:37.743880+0000","last_peered":"2024-09-15T21:19:36.740990+0000","last_clean":"2024-09-15T21:19:36.740990+0000","last_became_active":"2024-09-15T21:19:35.742040+0000","last_became_peered":"2024-09-15T21:19:35.742040+0000","last_unstale":"2024-09-15T21:19:38.745563+0000","last_undegraded":"2024-09-15T21:19:38.745563+0000","last_fullsized":"2024-09-15T21:19:38.745563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.748081+0000","last_change":"2024-09-15T21:19:39.747971+0000","last_active":"2024-09-15T21:19:39.748081+0000","last_peered":"2024-09-15T21:19:39.748081+0000","last_clean":"2024-09-15T21:19:39.748081+0000","last_became_active":"2024-09-15T21:19:39.747670+0000","last_became_peered":"2024-09-15T21:19:39.747670+0000","last_unstale":"2024-09-15T21:19:39.748081+0000","last_undegraded":"2024-09-15T21:19:39.748081+0000","last_fullsized":"2024-09-15T21:19:39.748081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:00.689805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.748025+0000","last_change":"2024-09-15T21:19:39.747922+0000","last_active":"2024-09-15T21:19:39.748025+0000","last_peered":"2024-09-15T21:19:39.748025+0000","last_clean":"2024-09-15T21:19:39.748025+0000","last_became_active":"2024-09-15T21:19:39.747606+0000","last_became_peered":"2024-09-15T21:19:39.747606+0000","last_unstale":"2024-09-15T21:19:39.748025+0000","last_undegraded":"2024-09-15T21:19:39.748025+0000","last_fullsized":"2024-09-15T21:19:39.748025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:28.550257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744403+0000","last_change":"2024-09-15T21:19:35.740110+0000","last_active":"2024-09-15T21:19:38.744403+0000","last_peered":"2024-09-15T21:19:38.744403+0000","last_clean":"2024-09-15T21:19:38.744403+0000","last_became_active":"2024-09-15T21:19:35.739773+0000","last_became_peered":"2024-09-15T21:19:35.739773+0000","last_unstale":"2024-09-15T21:19:38.744403+0000","last_undegraded":"2024-09-15T21:19:38.744403+0000","last_fullsized":"2024-09-15T21:19:38.744403+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:49:46.277839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744485+0000","last_change":"2024-09-15T21:19:35.740711+0000","last_active":"2024-09-15T21:19:38.744485+0000","last_peered":"2024-09-15T21:19:38.744485+0000","last_clean":"2024-09-15T21:19:38.744485+0000","last_became_active":"2024-09-15T21:19:35.740436+0000","last_became_peered":"2024-09-15T21:19:35.740436+0000","last_unstale":"2024-09-15T21:19:38.744485+0000","last_undegraded":"2024-09-15T21:19:38.744485+0000","last_fullsized":"2024-09-15T21:19:38.744485+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:56.312225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.747984+0000","last_change":"2024-09-15T21:19:39.747882+0000","last_active":"2024-09-15T21:19:39.747984+0000","last_peered":"2024-09-15T21:19:39.747984+0000","last_clean":"2024-09-15T21:19:39.747984+0000","last_became_active":"2024-09-15T21:19:39.747601+0000","last_became_peered":"2024-09-15T21:19:39.747601+0000","last_unstale":"2024-09-15T21:19:39.747984+0000","last_undegraded":"2024-09-15T21:19:39.747984+0000","last_fullsized":"2024-09-15T21:19:39.747984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:03.580034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.743874+0000","last_change":"2024-09-15T21:19:28.711561+0000","last_active":"2024-09-15T21:19:38.743874+0000","last_peered":"2024-09-15T21:19:38.743874+0000","last_clean":"2024-09-15T21:19:38.743874+0000","last_became_active":"2024-09-15T21:19:26.704834+0000","last_became_peered":"2024-09-15T21:19:26.704834+0000","last_unstale":"2024-09-15T21:19:38.743874+0000","last_undegraded":"2024-09-15T21:19:38.743874+0000","last_fullsized":"2024-09-15T21:19:38.743874+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:09.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744905+0000","last_change":"2024-09-15T21:19:37.743194+0000","last_active":"2024-09-15T21:19:37.743194+0000","last_peered":"2024-09-15T21:19:36.740191+0000","last_clean":"2024-09-15T21:19:36.740191+0000","last_became_active":"2024-09-15T21:19:35.741860+0000","last_became_peered":"2024-09-15T21:19:35.741860+0000","last_unstale":"2024-09-15T21:19:38.744905+0000","last_undegraded":"2024-09-15T21:19:38.744905+0000","last_fullsized":"2024-09-15T21:19:38.744905+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.748007+0000","last_change":"2024-09-15T21:19:39.747909+0000","last_active":"2024-09-15T21:19:39.748007+0000","last_peered":"2024-09-15T21:19:39.748007+0000","last_clean":"2024-09-15T21:19:39.748007+0000","last_became_active":"2024-09-15T21:19:39.747613+0000","last_became_peered":"2024-09-15T21:19:39.747613+0000","last_unstale":"2024-09-15T21:19:39.748007+0000","last_undegraded":"2024-09-15T21:19:39.748007+0000","last_fullsized":"2024-09-15T21:19:39.748007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:50.363505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.743142+0000","last_change":"2024-09-15T21:19:35.736396+0000","last_active":"2024-09-15T21:19:39.743142+0000","last_peered":"2024-09-15T21:19:39.743142+0000","last_clean":"2024-09-15T21:19:39.743142+0000","last_became_active":"2024-09-15T21:19:35.736102+0000","last_became_peered":"2024-09-15T21:19:35.736102+0000","last_unstale":"2024-09-15T21:19:39.743142+0000","last_undegraded":"2024-09-15T21:19:39.743142+0000","last_fullsized":"2024-09-15T21:19:39.743142+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:44.186116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":36,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.743238+0000","last_change":"2024-09-15T21:19:28.711657+0000","last_active":"2024-09-15T21:19:39.743238+0000","last_peered":"2024-09-15T21:19:39.743238+0000","last_clean":"2024-09-15T21:19:39.743238+0000","last_became_active":"2024-09-15T21:19:26.706209+0000","last_became_peered":"2024-09-15T21:19:26.706209+0000","last_unstale":"2024-09-15T21:19:39.743238+0000","last_undegraded":"2024-09-15T21:19:39.743238+0000","last_fullsized":"2024-09-15T21:19:39.743238+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:45.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744258+0000","last_change":"2024-09-15T21:19:37.741255+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.744258+0000","last_undegraded":"2024-09-15T21:19:38.744258+0000","last_fullsized":"2024-09-15T21:19:38.744258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.743917+0000","last_change":"2024-09-15T21:19:37.743917+0000","last_active":"2024-09-15T21:19:37.743917+0000","last_peered":"2024-09-15T21:19:36.740385+0000","last_clean":"2024-09-15T21:19:36.740385+0000","last_became_active":"2024-09-15T21:19:26.707957+0000","last_became_peered":"2024-09-15T21:19:26.707957+0000","last_unstale":"2024-09-15T21:19:37.743917+0000","last_undegraded":"2024-09-15T21:19:37.743917+0000","last_fullsized":"2024-09-15T21:19:37.743917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":5,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-15T21:19:38.744204+0000","last_change":"2024-09-15T21:19:37.741218+0000","last_active":"2024-09-15T21:19:34.719751+0000","last_peered":"2024-09-15T21:19:34.719751+0000","last_clean":"2024-09-15T21:19:34.719751+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-15T21:19:38.744204+0000","last_undegraded":"2024-09-15T21:19:38.744204+0000","last_fullsized":"2024-09-15T21:19:38.744204+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745204+0000","last_change":"2024-09-15T21:19:35.740749+0000","last_active":"2024-09-15T21:19:38.745204+0000","last_peered":"2024-09-15T21:19:38.745204+0000","last_clean":"2024-09-15T21:19:38.745204+0000","last_became_active":"2024-09-15T21:19:35.740462+0000","last_became_peered":"2024-09-15T21:19:35.740462+0000","last_unstale":"2024-09-15T21:19:38.745204+0000","last_undegraded":"2024-09-15T21:19:38.745204+0000","last_fullsized":"2024-09-15T21:19:38.745204+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:49.440175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.744969+0000","last_change":"2024-09-15T21:19:28.712419+0000","last_active":"2024-09-15T21:19:38.744969+0000","last_peered":"2024-09-15T21:19:38.744969+0000","last_clean":"2024-09-15T21:19:38.744969+0000","last_became_active":"2024-09-15T21:19:26.705010+0000","last_became_peered":"2024-09-15T21:19:26.705010+0000","last_unstale":"2024-09-15T21:19:38.744969+0000","last_undegraded":"2024-09-15T21:19:38.744969+0000","last_fullsized":"2024-09-15T21:19:38.744969+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:58.295608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745013+0000","last_change":"2024-09-15T21:19:35.741310+0000","last_active":"2024-09-15T21:19:38.745013+0000","last_peered":"2024-09-15T21:19:38.745013+0000","last_clean":"2024-09-15T21:19:38.745013+0000","last_became_active":"2024-09-15T21:19:35.741015+0000","last_became_peered":"2024-09-15T21:19:35.741015+0000","last_unstale":"2024-09-15T21:19:38.745013+0000","last_undegraded":"2024-09-15T21:19:38.745013+0000","last_fullsized":"2024-09-15T21:19:38.745013+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:53:24.766090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.746494+0000","last_change":"2024-09-15T21:19:39.746403+0000","last_active":"2024-09-15T21:19:39.746494+0000","last_peered":"2024-09-15T21:19:39.746494+0000","last_clean":"2024-09-15T21:19:39.746494+0000","last_became_active":"2024-09-15T21:19:39.746038+0000","last_became_peered":"2024-09-15T21:19:39.746038+0000","last_unstale":"2024-09-15T21:19:39.746494+0000","last_undegraded":"2024-09-15T21:19:39.746494+0000","last_fullsized":"2024-09-15T21:19:39.746494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:44.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745447+0000","last_change":"2024-09-15T21:19:35.741013+0000","last_active":"2024-09-15T21:19:38.745447+0000","last_peered":"2024-09-15T21:19:38.745447+0000","last_clean":"2024-09-15T21:19:38.745447+0000","last_became_active":"2024-09-15T21:19:35.740206+0000","last_became_peered":"2024-09-15T21:19:35.740206+0000","last_unstale":"2024-09-15T21:19:38.745447+0000","last_undegraded":"2024-09-15T21:19:38.745447+0000","last_fullsized":"2024-09-15T21:19:38.745447+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:37.768461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.745102+0000","last_change":"2024-09-15T21:19:28.712635+0000","last_active":"2024-09-15T21:19:38.745102+0000","last_peered":"2024-09-15T21:19:38.745102+0000","last_clean":"2024-09-15T21:19:38.745102+0000","last_became_active":"2024-09-15T21:19:26.705751+0000","last_became_peered":"2024-09-15T21:19:26.705751+0000","last_unstale":"2024-09-15T21:19:38.745102+0000","last_undegraded":"2024-09-15T21:19:38.745102+0000","last_fullsized":"2024-09-15T21:19:38.745102+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:12.537829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-15T21:19:37.744483+0000","last_change":"2024-09-15T21:19:37.744483+0000","last_active":"2024-09-15T21:19:37.744483+0000","last_peered":"2024-09-15T21:19:36.741546+0000","last_clean":"2024-09-15T21:19:36.741546+0000","last_became_active":"2024-09-15T21:19:26.707803+0000","last_became_peered":"2024-09-15T21:19:26.707803+0000","last_unstale":"2024-09-15T21:19:37.744483+0000","last_undegraded":"2024-09-15T21:19:37.744483+0000","last_fullsized":"2024-09-15T21:19:37.744483+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-15T21:19:38.808745+0000","last_change":"2024-09-15T21:19:35.740618+0000","last_active":"2024-09-15T21:19:38.808745+0000","last_peered":"2024-09-15T21:19:38.808745+0000","last_clean":"2024-09-15T21:19:38.808745+0000","last_became_active":"2024-09-15T21:19:35.740282+0000","last_became_peered":"2024-09-15T21:19:35.740282+0000","last_unstale":"2024-09-15T21:19:38.808745+0000","last_undegraded":"2024-09-15T21:19:38.808745+0000","last_fullsized":"2024-09-15T21:19:38.808745+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:01.218654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705673,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":55585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":180224,"data_stored":52451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:49.454 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:19:49.750 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:19:49.751 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:19:49.763 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-09-15T21:19:48.671004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54436,"kb_used_data":368,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689244,"statfs":{"total":193273528320,"available":193217785856,"internally_reserved":0,"allocated":376832,"data_stored":111170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.930311"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.743101+0000","last_change":"2024-09-15T21:19:35.736448+0000","last_active":"2024-09-15T21:19:39.743101+0000","last_peered":"2024-09-15T21:19:39.743101+0000","last_clean":"2024-09-15T21:19:39.743101+0000","last_became_active":"2024-09-15T21:19:35.736099+0000","last_became_peered":"2024-09-15T21:19:35.736099+0000","last_unstale":"2024-09-15T21:19:39.743101+0000","last_undegraded":"2024-09-15T21:19:39.743101+0000","last_fullsized":"2024-09-15T21:19:39.743101+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:18.340741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.747208+0000","last_change":"2024-09-15T21:19:39.747139+0000","last_active":"2024-09-15T21:19:39.747208+0000","last_peered":"2024-09-15T21:19:39.747208+0000","last_clean":"2024-09-15T21:19:39.747208+0000","last_became_active":"2024-09-15T21:19:39.746905+0000","last_became_peered":"2024-09-15T21:19:39.746905+0000","last_unstale":"2024-09-15T21:19:39.747208+0000","last_undegraded":"2024-09-15T21:19:39.747208+0000","last_fullsized":"2024-09-15T21:19:39.747208+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:46.017048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.751497+0000","last_change":"2024-09-15T21:19:39.751407+0000","last_active":"2024-09-15T21:19:39.751497+0000","last_peered":"2024-09-15T21:19:39.751497+0000","last_clean":"2024-09-15T21:19:39.751497+0000","last_became_active":"2024-09-15T21:19:39.751109+0000","last_became_peered":"2024-09-15T21:19:39.751109+0000","last_unstale":"2024-09-15T21:19:39.751497+0000","last_undegraded":"2024-09-15T21:19:39.751497+0000","last_fullsized":"2024-09-15T21:19:39.751497+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:45:01.754292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.748081+0000","last_change":"2024-09-15T21:19:39.747971+0000","last_active":"2024-09-15T21:19:39.748081+0000","last_peered":"2024-09-15T21:19:39.748081+0000","last_clean":"2024-09-15T21:19:39.748081+0000","last_became_active":"2024-09-15T21:19:39.747670+0000","last_became_peered":"2024-09-15T21:19:39.747670+0000","last_unstale":"2024-09-15T21:19:39.748081+0000","last_undegraded":"2024-09-15T21:19:39.748081+0000","last_fullsized":"2024-09-15T21:19:39.748081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:02:00.689805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.748025+0000","last_change":"2024-09-15T21:19:39.747922+0000","last_active":"2024-09-15T21:19:39.748025+0000","last_peered":"2024-09-15T21:19:39.748025+0000","last_clean":"2024-09-15T21:19:39.748025+0000","last_became_active":"2024-09-15T21:19:39.747606+0000","last_became_peered":"2024-09-15T21:19:39.747606+0000","last_unstale":"2024-09-15T21:19:39.748025+0000","last_undegraded":"2024-09-15T21:19:39.748025+0000","last_fullsized":"2024-09-15T21:19:39.748025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:39:28.550257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.744030+0000","last_change":"2024-09-15T21:19:35.740110+0000","last_active":"2024-09-15T21:19:39.744030+0000","last_peered":"2024-09-15T21:19:39.744030+0000","last_clean":"2024-09-15T21:19:39.744030+0000","last_became_active":"2024-09-15T21:19:35.739773+0000","last_became_peered":"2024-09-15T21:19:35.739773+0000","last_unstale":"2024-09-15T21:19:39.744030+0000","last_undegraded":"2024-09-15T21:19:39.744030+0000","last_fullsized":"2024-09-15T21:19:39.744030+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:49:46.277839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.742917+0000","last_change":"2024-09-15T21:19:35.740711+0000","last_active":"2024-09-15T21:19:39.742917+0000","last_peered":"2024-09-15T21:19:39.742917+0000","last_clean":"2024-09-15T21:19:39.742917+0000","last_became_active":"2024-09-15T21:19:35.740436+0000","last_became_peered":"2024-09-15T21:19:35.740436+0000","last_unstale":"2024-09-15T21:19:39.742917+0000","last_undegraded":"2024-09-15T21:19:39.742917+0000","last_fullsized":"2024-09-15T21:19:39.742917+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:58:56.312225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.747984+0000","last_change":"2024-09-15T21:19:39.747882+0000","last_active":"2024-09-15T21:19:39.747984+0000","last_peered":"2024-09-15T21:19:39.747984+0000","last_clean":"2024-09-15T21:19:39.747984+0000","last_became_active":"2024-09-15T21:19:39.747601+0000","last_became_peered":"2024-09-15T21:19:39.747601+0000","last_unstale":"2024-09-15T21:19:39.747984+0000","last_undegraded":"2024-09-15T21:19:39.747984+0000","last_fullsized":"2024-09-15T21:19:39.747984+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:03.580034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.742878+0000","last_change":"2024-09-15T21:19:28.711561+0000","last_active":"2024-09-15T21:19:39.742878+0000","last_peered":"2024-09-15T21:19:39.742878+0000","last_clean":"2024-09-15T21:19:39.742878+0000","last_became_active":"2024-09-15T21:19:26.704834+0000","last_became_peered":"2024-09-15T21:19:26.704834+0000","last_unstale":"2024-09-15T21:19:39.742878+0000","last_undegraded":"2024-09-15T21:19:39.742878+0000","last_fullsized":"2024-09-15T21:19:39.742878+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:52:09.310410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.751632+0000","last_change":"2024-09-15T21:19:39.751546+0000","last_active":"2024-09-15T21:19:39.751632+0000","last_peered":"2024-09-15T21:19:39.751632+0000","last_clean":"2024-09-15T21:19:39.751632+0000","last_became_active":"2024-09-15T21:19:39.751178+0000","last_became_peered":"2024-09-15T21:19:39.751178+0000","last_unstale":"2024-09-15T21:19:39.751632+0000","last_undegraded":"2024-09-15T21:19:39.751632+0000","last_fullsized":"2024-09-15T21:19:39.751632+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:45:35.427026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.748007+0000","last_change":"2024-09-15T21:19:39.747909+0000","last_active":"2024-09-15T21:19:39.748007+0000","last_peered":"2024-09-15T21:19:39.748007+0000","last_clean":"2024-09-15T21:19:39.748007+0000","last_became_active":"2024-09-15T21:19:39.747613+0000","last_became_peered":"2024-09-15T21:19:39.747613+0000","last_unstale":"2024-09-15T21:19:39.748007+0000","last_undegraded":"2024-09-15T21:19:39.748007+0000","last_fullsized":"2024-09-15T21:19:39.748007+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:52:50.363505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.743142+0000","last_change":"2024-09-15T21:19:35.736396+0000","last_active":"2024-09-15T21:19:39.743142+0000","last_peered":"2024-09-15T21:19:39.743142+0000","last_clean":"2024-09-15T21:19:39.743142+0000","last_became_active":"2024-09-15T21:19:35.736102+0000","last_became_peered":"2024-09-15T21:19:35.736102+0000","last_unstale":"2024-09-15T21:19:39.743142+0000","last_undegraded":"2024-09-15T21:19:39.743142+0000","last_fullsized":"2024-09-15T21:19:39.743142+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:28:44.186116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":36,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.743238+0000","last_change":"2024-09-15T21:19:28.711657+0000","last_active":"2024-09-15T21:19:39.743238+0000","last_peered":"2024-09-15T21:19:39.743238+0000","last_clean":"2024-09-15T21:19:39.743238+0000","last_became_active":"2024-09-15T21:19:26.706209+0000","last_became_peered":"2024-09-15T21:19:26.706209+0000","last_unstale":"2024-09-15T21:19:39.743238+0000","last_undegraded":"2024-09-15T21:19:39.743238+0000","last_fullsized":"2024-09-15T21:19:39.743238+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:58:45.517282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.751708+0000","last_change":"2024-09-15T21:19:39.751613+0000","last_active":"2024-09-15T21:19:39.751708+0000","last_peered":"2024-09-15T21:19:39.751708+0000","last_clean":"2024-09-15T21:19:39.751708+0000","last_became_active":"2024-09-15T21:19:39.751344+0000","last_became_peered":"2024-09-15T21:19:39.751344+0000","last_unstale":"2024-09-15T21:19:39.751708+0000","last_undegraded":"2024-09-15T21:19:39.751708+0000","last_fullsized":"2024-09-15T21:19:39.751708+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:59:26.668705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":44,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.759845+0000","last_change":"2024-09-15T21:19:39.759741+0000","last_active":"2024-09-15T21:19:39.759845+0000","last_peered":"2024-09-15T21:19:39.759845+0000","last_clean":"2024-09-15T21:19:39.759845+0000","last_became_active":"2024-09-15T21:19:39.753931+0000","last_became_peered":"2024-09-15T21:19:39.753931+0000","last_unstale":"2024-09-15T21:19:39.759845+0000","last_undegraded":"2024-09-15T21:19:39.759845+0000","last_fullsized":"2024-09-15T21:19:39.759845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:31:28.976295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.751639+0000","last_change":"2024-09-15T21:19:39.751559+0000","last_active":"2024-09-15T21:19:39.751639+0000","last_peered":"2024-09-15T21:19:39.751639+0000","last_clean":"2024-09-15T21:19:39.751639+0000","last_became_active":"2024-09-15T21:19:39.751079+0000","last_became_peered":"2024-09-15T21:19:39.751079+0000","last_unstale":"2024-09-15T21:19:39.751639+0000","last_undegraded":"2024-09-15T21:19:39.751639+0000","last_fullsized":"2024-09-15T21:19:39.751639+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:45:22.094299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.745883+0000","last_change":"2024-09-15T21:19:35.740749+0000","last_active":"2024-09-15T21:19:39.745883+0000","last_peered":"2024-09-15T21:19:39.745883+0000","last_clean":"2024-09-15T21:19:39.745883+0000","last_became_active":"2024-09-15T21:19:35.740462+0000","last_became_peered":"2024-09-15T21:19:35.740462+0000","last_unstale":"2024-09-15T21:19:39.745883+0000","last_undegraded":"2024-09-15T21:19:39.745883+0000","last_fullsized":"2024-09-15T21:19:39.745883+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:26:49.440175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.745694+0000","last_change":"2024-09-15T21:19:28.712419+0000","last_active":"2024-09-15T21:19:39.745694+0000","last_peered":"2024-09-15T21:19:39.745694+0000","last_clean":"2024-09-15T21:19:39.745694+0000","last_became_active":"2024-09-15T21:19:26.705010+0000","last_became_peered":"2024-09-15T21:19:26.705010+0000","last_unstale":"2024-09-15T21:19:39.745694+0000","last_undegraded":"2024-09-15T21:19:39.745694+0000","last_fullsized":"2024-09-15T21:19:39.745694+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:48:58.295608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.745672+0000","last_change":"2024-09-15T21:19:35.741310+0000","last_active":"2024-09-15T21:19:39.745672+0000","last_peered":"2024-09-15T21:19:39.745672+0000","last_clean":"2024-09-15T21:19:39.745672+0000","last_became_active":"2024-09-15T21:19:35.741015+0000","last_became_peered":"2024-09-15T21:19:35.741015+0000","last_unstale":"2024-09-15T21:19:39.745672+0000","last_undegraded":"2024-09-15T21:19:39.745672+0000","last_fullsized":"2024-09-15T21:19:39.745672+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:53:24.766090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.746494+0000","last_change":"2024-09-15T21:19:39.746403+0000","last_active":"2024-09-15T21:19:39.746494+0000","last_peered":"2024-09-15T21:19:39.746494+0000","last_clean":"2024-09-15T21:19:39.746494+0000","last_became_active":"2024-09-15T21:19:39.746038+0000","last_became_peered":"2024-09-15T21:19:39.746038+0000","last_unstale":"2024-09-15T21:19:39.746494+0000","last_undegraded":"2024-09-15T21:19:39.746494+0000","last_fullsized":"2024-09-15T21:19:39.746494+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:37:44.742897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.745692+0000","last_change":"2024-09-15T21:19:35.741013+0000","last_active":"2024-09-15T21:19:39.745692+0000","last_peered":"2024-09-15T21:19:39.745692+0000","last_clean":"2024-09-15T21:19:39.745692+0000","last_became_active":"2024-09-15T21:19:35.740206+0000","last_became_peered":"2024-09-15T21:19:35.740206+0000","last_unstale":"2024-09-15T21:19:39.745692+0000","last_undegraded":"2024-09-15T21:19:39.745692+0000","last_fullsized":"2024-09-15T21:19:39.745692+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:37.768461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.745763+0000","last_change":"2024-09-15T21:19:28.712635+0000","last_active":"2024-09-15T21:19:39.745763+0000","last_peered":"2024-09-15T21:19:39.745763+0000","last_clean":"2024-09-15T21:19:39.745763+0000","last_became_active":"2024-09-15T21:19:26.705751+0000","last_became_peered":"2024-09-15T21:19:26.705751+0000","last_unstale":"2024-09-15T21:19:39.745763+0000","last_undegraded":"2024-09-15T21:19:39.745763+0000","last_fullsized":"2024-09-15T21:19:39.745763+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:47:12.537829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.751553+0000","last_change":"2024-09-15T21:19:39.751463+0000","last_active":"2024-09-15T21:19:39.751553+0000","last_peered":"2024-09-15T21:19:39.751553+0000","last_clean":"2024-09-15T21:19:39.751553+0000","last_became_active":"2024-09-15T21:19:39.751163+0000","last_became_peered":"2024-09-15T21:19:39.751163+0000","last_unstale":"2024-09-15T21:19:39.751553+0000","last_undegraded":"2024-09-15T21:19:39.751553+0000","last_fullsized":"2024-09-15T21:19:39.751553+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:18:50.830204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-15T21:19:39.745589+0000","last_change":"2024-09-15T21:19:35.740618+0000","last_active":"2024-09-15T21:19:39.745589+0000","last_peered":"2024-09-15T21:19:39.745589+0000","last_clean":"2024-09-15T21:19:39.745589+0000","last_became_active":"2024-09-15T21:19:35.740282+0000","last_became_peered":"2024-09-15T21:19:35.740282+0000","last_unstale":"2024-09-15T21:19:39.745589+0000","last_undegraded":"2024-09-15T21:19:39.745589+0000","last_fullsized":"2024-09-15T21:19:39.745589+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:42:01.218654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":188416,"data_stored":55585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":188416,"data_stored":55585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:19:49.764 INFO:tasks.lost_unfound.ceph_manager:recovered! 2024-09-15T21:19:49.764 INFO:tasks.util.rados:rados -p unfound_pool put existing_1 /etc/fstab 2024-09-15T21:19:49.765 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_1 /etc/fstab 2024-09-15T21:19:49.815 INFO:tasks.util.rados:rados -p unfound_pool put existed_1 /etc/fstab 2024-09-15T21:19:49.816 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_1 /etc/fstab 2024-09-15T21:19:49.866 INFO:tasks.util.rados:rados -p unfound_pool rm existed_1 2024-09-15T21:19:49.866 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_1 2024-09-15T21:19:49.915 INFO:tasks.util.rados:rados -p unfound_pool put existing_2 /etc/fstab 2024-09-15T21:19:49.916 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_2 /etc/fstab 2024-09-15T21:19:49.966 INFO:tasks.util.rados:rados -p unfound_pool put existed_2 /etc/fstab 2024-09-15T21:19:49.966 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_2 /etc/fstab 2024-09-15T21:19:50.016 INFO:tasks.util.rados:rados -p unfound_pool rm existed_2 2024-09-15T21:19:50.016 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_2 2024-09-15T21:19:50.067 INFO:tasks.util.rados:rados -p unfound_pool put existing_3 /etc/fstab 2024-09-15T21:19:50.068 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_3 /etc/fstab 2024-09-15T21:19:50.119 INFO:tasks.util.rados:rados -p unfound_pool put existed_3 /etc/fstab 2024-09-15T21:19:50.119 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_3 /etc/fstab 2024-09-15T21:19:50.169 INFO:tasks.util.rados:rados -p unfound_pool rm existed_3 2024-09-15T21:19:50.169 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_3 2024-09-15T21:19:50.218 INFO:tasks.util.rados:rados -p unfound_pool put existing_4 /etc/fstab 2024-09-15T21:19:50.219 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_4 /etc/fstab 2024-09-15T21:19:50.268 INFO:tasks.util.rados:rados -p unfound_pool put existed_4 /etc/fstab 2024-09-15T21:19:50.269 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_4 /etc/fstab 2024-09-15T21:19:50.319 INFO:tasks.util.rados:rados -p unfound_pool rm existed_4 2024-09-15T21:19:50.319 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_4 2024-09-15T21:19:50.368 INFO:tasks.util.rados:rados -p unfound_pool put existing_5 /etc/fstab 2024-09-15T21:19:50.368 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_5 /etc/fstab 2024-09-15T21:19:50.419 INFO:tasks.util.rados:rados -p unfound_pool put existed_5 /etc/fstab 2024-09-15T21:19:50.419 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_5 /etc/fstab 2024-09-15T21:19:50.469 INFO:tasks.util.rados:rados -p unfound_pool rm existed_5 2024-09-15T21:19:50.470 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_5 2024-09-15T21:19:50.518 INFO:tasks.util.rados:rados -p unfound_pool put existing_6 /etc/fstab 2024-09-15T21:19:50.518 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_6 /etc/fstab 2024-09-15T21:19:50.568 INFO:tasks.util.rados:rados -p unfound_pool put existed_6 /etc/fstab 2024-09-15T21:19:50.568 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_6 /etc/fstab 2024-09-15T21:19:50.619 INFO:tasks.util.rados:rados -p unfound_pool rm existed_6 2024-09-15T21:19:50.620 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_6 2024-09-15T21:19:50.668 INFO:tasks.util.rados:rados -p unfound_pool put existing_7 /etc/fstab 2024-09-15T21:19:50.668 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_7 /etc/fstab 2024-09-15T21:19:50.719 INFO:tasks.util.rados:rados -p unfound_pool put existed_7 /etc/fstab 2024-09-15T21:19:50.719 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_7 /etc/fstab 2024-09-15T21:19:50.769 INFO:tasks.util.rados:rados -p unfound_pool rm existed_7 2024-09-15T21:19:50.769 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_7 2024-09-15T21:19:50.819 INFO:tasks.util.rados:rados -p unfound_pool put existing_8 /etc/fstab 2024-09-15T21:19:50.819 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_8 /etc/fstab 2024-09-15T21:19:50.869 INFO:tasks.util.rados:rados -p unfound_pool put existed_8 /etc/fstab 2024-09-15T21:19:50.869 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_8 /etc/fstab 2024-09-15T21:19:50.919 INFO:tasks.util.rados:rados -p unfound_pool rm existed_8 2024-09-15T21:19:50.919 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_8 2024-09-15T21:19:50.968 INFO:tasks.util.rados:rados -p unfound_pool put existing_9 /etc/fstab 2024-09-15T21:19:50.968 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_9 /etc/fstab 2024-09-15T21:19:51.018 INFO:tasks.util.rados:rados -p unfound_pool put existed_9 /etc/fstab 2024-09-15T21:19:51.018 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_9 /etc/fstab 2024-09-15T21:19:51.068 INFO:tasks.util.rados:rados -p unfound_pool rm existed_9 2024-09-15T21:19:51.068 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool rm existed_9 2024-09-15T21:19:51.117 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 injectargs '--osd-recovery-delay-start 1000 --osd-min-pg-log-entries 100000000' 2024-09-15T21:19:51.237 INFO:teuthology.orchestra.run.smithi073.stdout:{} 2024-09-15T21:19:51.237 INFO:teuthology.orchestra.run.smithi073.stderr:osd_min_pg_log_entries = '' (not observed, change may require restart) osd_recovery_delay_start = '' 2024-09-15T21:19:51.247 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T21:19:51.247 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:19:51.278 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T21:19:51.278 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-15T21:19:51.610 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-15T21:19:51.607+0000 7f8e1ee55640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-09-15T21:19:51.790 INFO:teuthology.orchestra.run.smithi073.stderr:marked down osd.0. 2024-09-15T21:19:51.805 INFO:tasks.util.rados:rados -p unfound_pool put new_1 /etc/fstab 2024-09-15T21:19:51.805 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_1 /etc/fstab 2024-09-15T21:19:52.810 INFO:tasks.util.rados:rados -p unfound_pool put existed_1 /etc/fstab 2024-09-15T21:19:52.810 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_1 /etc/fstab 2024-09-15T21:19:52.865 INFO:tasks.util.rados:rados -p unfound_pool put existing_1 /etc/fstab 2024-09-15T21:19:52.866 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_1 /etc/fstab 2024-09-15T21:19:52.917 INFO:tasks.util.rados:rados -p unfound_pool put new_2 /etc/fstab 2024-09-15T21:19:52.917 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_2 /etc/fstab 2024-09-15T21:19:52.967 INFO:tasks.util.rados:rados -p unfound_pool put existed_2 /etc/fstab 2024-09-15T21:19:52.967 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_2 /etc/fstab 2024-09-15T21:19:53.017 INFO:tasks.util.rados:rados -p unfound_pool put existing_2 /etc/fstab 2024-09-15T21:19:53.017 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_2 /etc/fstab 2024-09-15T21:19:53.067 INFO:tasks.util.rados:rados -p unfound_pool put new_3 /etc/fstab 2024-09-15T21:19:53.067 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_3 /etc/fstab 2024-09-15T21:19:53.117 INFO:tasks.util.rados:rados -p unfound_pool put existed_3 /etc/fstab 2024-09-15T21:19:53.117 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_3 /etc/fstab 2024-09-15T21:19:53.166 INFO:tasks.util.rados:rados -p unfound_pool put existing_3 /etc/fstab 2024-09-15T21:19:53.167 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_3 /etc/fstab 2024-09-15T21:19:53.216 INFO:tasks.util.rados:rados -p unfound_pool put new_4 /etc/fstab 2024-09-15T21:19:53.217 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_4 /etc/fstab 2024-09-15T21:19:53.266 INFO:tasks.util.rados:rados -p unfound_pool put existed_4 /etc/fstab 2024-09-15T21:19:53.267 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_4 /etc/fstab 2024-09-15T21:19:53.316 INFO:tasks.util.rados:rados -p unfound_pool put existing_4 /etc/fstab 2024-09-15T21:19:53.316 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_4 /etc/fstab 2024-09-15T21:19:53.366 INFO:tasks.util.rados:rados -p unfound_pool put new_5 /etc/fstab 2024-09-15T21:19:53.366 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_5 /etc/fstab 2024-09-15T21:19:53.416 INFO:tasks.util.rados:rados -p unfound_pool put existed_5 /etc/fstab 2024-09-15T21:19:53.416 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_5 /etc/fstab 2024-09-15T21:19:53.466 INFO:tasks.util.rados:rados -p unfound_pool put existing_5 /etc/fstab 2024-09-15T21:19:53.466 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_5 /etc/fstab 2024-09-15T21:19:53.517 INFO:tasks.util.rados:rados -p unfound_pool put new_6 /etc/fstab 2024-09-15T21:19:53.517 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_6 /etc/fstab 2024-09-15T21:19:53.567 INFO:tasks.util.rados:rados -p unfound_pool put existed_6 /etc/fstab 2024-09-15T21:19:53.567 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_6 /etc/fstab 2024-09-15T21:19:53.617 INFO:tasks.util.rados:rados -p unfound_pool put existing_6 /etc/fstab 2024-09-15T21:19:53.617 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_6 /etc/fstab 2024-09-15T21:19:53.667 INFO:tasks.util.rados:rados -p unfound_pool put new_7 /etc/fstab 2024-09-15T21:19:53.667 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_7 /etc/fstab 2024-09-15T21:19:53.717 INFO:tasks.util.rados:rados -p unfound_pool put existed_7 /etc/fstab 2024-09-15T21:19:53.718 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_7 /etc/fstab 2024-09-15T21:19:53.769 INFO:tasks.util.rados:rados -p unfound_pool put existing_7 /etc/fstab 2024-09-15T21:19:53.769 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_7 /etc/fstab 2024-09-15T21:19:53.820 INFO:tasks.util.rados:rados -p unfound_pool put new_8 /etc/fstab 2024-09-15T21:19:53.820 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_8 /etc/fstab 2024-09-15T21:19:53.870 INFO:tasks.util.rados:rados -p unfound_pool put existed_8 /etc/fstab 2024-09-15T21:19:53.870 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_8 /etc/fstab 2024-09-15T21:19:53.920 INFO:tasks.util.rados:rados -p unfound_pool put existing_8 /etc/fstab 2024-09-15T21:19:53.920 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_8 /etc/fstab 2024-09-15T21:19:53.971 INFO:tasks.util.rados:rados -p unfound_pool put new_9 /etc/fstab 2024-09-15T21:19:53.971 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put new_9 /etc/fstab 2024-09-15T21:19:54.021 INFO:tasks.util.rados:rados -p unfound_pool put existed_9 /etc/fstab 2024-09-15T21:19:54.021 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existed_9 /etc/fstab 2024-09-15T21:19:54.071 INFO:tasks.util.rados:rados -p unfound_pool put existing_9 /etc/fstab 2024-09-15T21:19:54.071 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool put existing_9 /etc/fstab 2024-09-15T21:19:54.121 INFO:tasks.lost_unfound:osd.0 command_args is foo 2024-09-15T21:19:54.121 INFO:tasks.lost_unfound:() 2024-09-15T21:19:54.121 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-15T21:19:54.122 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 --osd-recovery-delay-start 1000 2024-09-15T21:19:54.124 INFO:tasks.ceph.osd.0:Started 2024-09-15T21:19:54.124 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:19:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-15T21:19:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-15T21:19:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-15T21:19:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-15T21:19:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-15T21:19:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-15T21:19:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:config diff 2024-09-15T21:19:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:config diff get 2024-09-15T21:19:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:config get 2024-09-15T21:19:54.224 INFO:teuthology.orchestra.run.smithi073.stderr:config help [] 2024-09-15T21:19:54.224 INFO:teuthology.orchestra.run.smithi073.stderr:config set ... 2024-09-15T21:19:54.224 INFO:teuthology.orchestra.run.smithi073.stderr:admin_socket: invalid command 2024-09-15T21:19:54.225 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:19:54.225 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-15T21:19:55.291 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:19:55.288+0000 7fa55340b640 -1 Falling back to public interface 2024-09-15T21:19:59.083 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:19:59.080+0000 7fa55340b640 -1 osd.0 21 log_to_monitors true 2024-09-15T21:19:59.226 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:19:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-15T21:19:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-15T21:19:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-15T21:19:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-15T21:19:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-15T21:19:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-15T21:19:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-15T21:19:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-15T21:19:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-15T21:19:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-15T21:19:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator fragmentation block 2024-09-15T21:19:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:admin_socket: invalid command 2024-09-15T21:19:59.335 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:19:59.336 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-15T21:19:59.879 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:19:59.876+0000 7fa54aabe640 -1 osd.0 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:20:04.337 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-15T21:20:04.455 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:04.456 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-15T21:20:04.456 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-15T21:20:04.456 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:04.465 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-15T21:20:04.763 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-15T21:20:04.760+0000 7f8e1ee55640 -1 mon.a@0(leader).osd e26 definitely_dead 0 2024-09-15T21:20:04.764 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0 is already in. 2024-09-15T21:20:04.776 INFO:tasks.lost_unfound.ceph_manager:waiting for osd.0 to be up 2024-09-15T21:20:04.776 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:20:05.064 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:05.064 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","created":"2024-09-15T21:19:07.254022+0000","modified":"2024-09-15T21:20:01.865143+0000","last_up_change":"2024-09-15T21:20:00.856689+0000","last_in_change":"2024-09-15T21:19:38.738398+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:19:25.465189+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-15T21:19:34.551635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c55f097a-59ed-4ce1-9803-4c6019429ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":25,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6801","nonce":173704734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6803","nonce":173704734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6807","nonce":173704734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6805","nonce":173704734}]},"public_addr":"172.21.15.73:6801/173704734","cluster_addr":"172.21.15.73:6803/173704734","heartbeat_back_addr":"172.21.15.73:6807/173704734","heartbeat_front_addr":"172.21.15.73:6805/173704734","state":["exists","up"]},{"osd":1,"uuid":"7daff6cd-a09c-4c15-a282-0859e8372dd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6809","nonce":1755138815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6811","nonce":1755138815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6815","nonce":1755138815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6813","nonce":1755138815}]},"public_addr":"172.21.15.73:6809/1755138815","cluster_addr":"172.21.15.73:6811/1755138815","heartbeat_back_addr":"172.21.15.73:6815/1755138815","heartbeat_front_addr":"172.21.15.73:6813/1755138815","state":["exists","up"]},{"osd":2,"uuid":"9abad4dd-4a88-4080-9ab4-5c4203d7fd51","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6817","nonce":1392278833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6819","nonce":1392278833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6823","nonce":1392278833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":1392278833},{"type":"v1","addr":"172.21.15.73:6821","nonce":1392278833}]},"public_addr":"172.21.15.73:6817/1392278833","cluster_addr":"172.21.15.73:6819/1392278833","heartbeat_back_addr":"172.21.15.73:6823/1392278833","heartbeat_front_addr":"172.21.15.73:6821/1392278833","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.867529+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.608114+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-15T21:19:17.891920+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:20:05.077 INFO:tasks.lost_unfound.ceph_manager:osd.0 is up 2024-09-15T21:20:05.078 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-15T21:20:05.078 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:20:05.214 INFO:teuthology.orchestra.run.smithi073.stdout:34359738383 2024-09-15T21:20:05.215 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:20:05.216 INFO:teuthology.orchestra.run.smithi073.stdout:107374182403 2024-09-15T21:20:05.216 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:20:05.541 INFO:teuthology.orchestra.run.smithi073.stdout:34359738381 2024-09-15T21:20:05.544 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-09-15T21:20:05.554 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738383 got 34359738381 for osd.1 2024-09-15T21:20:05.557 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182403 got 1 for osd.0 2024-09-15T21:20:06.555 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:20:06.558 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:20:06.863 INFO:teuthology.orchestra.run.smithi073.stdout:34359738381 2024-09-15T21:20:06.864 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-09-15T21:20:06.877 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738383 got 34359738381 for osd.1 2024-09-15T21:20:06.878 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182403 got 1 for osd.0 2024-09-15T21:20:07.877 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-15T21:20:07.878 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:20:08.184 INFO:teuthology.orchestra.run.smithi073.stdout:34359738383 2024-09-15T21:20:08.187 INFO:teuthology.orchestra.run.smithi073.stdout:107374182403 2024-09-15T21:20:08.197 INFO:tasks.lost_unfound.ceph_manager:need seq 34359738383 got 34359738383 for osd.1 2024-09-15T21:20:08.198 DEBUG:teuthology.parallel:result is None 2024-09-15T21:20:08.200 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182403 got 107374182403 for osd.0 2024-09-15T21:20:08.200 DEBUG:teuthology.parallel:result is None 2024-09-15T21:20:08.200 INFO:tasks.lost_unfound.ceph_manager:waiting till active 2024-09-15T21:20:08.200 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:08.519 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:08.520 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:20:08.531 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-09-15T21:20:06.675164+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55104,"kb_used_data":592,"kb_used_omap":5,"kb_used_meta":54458,"kb_avail":188688576,"statfs":{"total":193273528320,"available":193217101824,"internally_reserved":0,"allocated":606208,"data_stored":163976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6032,"internal_metadata":55765104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.812989"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":23,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.886006+0000","last_change":"2024-09-15T21:20:01.885927+0000","last_active":"2024-09-15T21:20:01.886006+0000","last_peered":"2024-09-15T21:20:01.886006+0000","last_clean":"2024-09-15T21:20:01.886006+0000","last_became_active":"2024-09-15T21:20:01.885642+0000","last_became_peered":"2024-09-15T21:20:01.885642+0000","last_unstale":"2024-09-15T21:20:01.886006+0000","last_undegraded":"2024-09-15T21:20:01.886006+0000","last_fullsized":"2024-09-15T21:20:01.886006+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:51:57.442938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":26,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-15T21:20:01.887405+0000","last_change":"2024-09-15T21:20:01.888724+0000","last_active":"2024-09-15T21:20:01.887405+0000","last_peered":"2024-09-15T21:20:01.887405+0000","last_clean":"2024-09-15T21:19:50.263852+0000","last_became_active":"2024-09-15T21:20:01.887143+0000","last_became_peered":"2024-09-15T21:20:01.887143+0000","last_unstale":"2024-09-15T21:20:01.887405+0000","last_undegraded":"2024-09-15T21:20:01.880415+0000","last_fullsized":"2024-09-15T21:20:01.887405+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":45,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-15T21:20:01.883301+0000","last_change":"2024-09-15T21:20:01.884274+0000","last_active":"2024-09-15T21:20:01.883301+0000","last_peered":"2024-09-15T21:20:01.883301+0000","last_clean":"2024-09-15T21:19:50.814098+0000","last_became_active":"2024-09-15T21:20:01.883046+0000","last_became_peered":"2024-09-15T21:20:01.883046+0000","last_unstale":"2024-09-15T21:20:01.883301+0000","last_undegraded":"2024-09-15T21:20:01.873048+0000","last_fullsized":"2024-09-15T21:20:01.883301+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.888406+0000","last_change":"2024-09-15T21:20:01.887734+0000","last_active":"2024-09-15T21:20:01.888406+0000","last_peered":"2024-09-15T21:20:01.888406+0000","last_clean":"2024-09-15T21:20:01.888406+0000","last_became_active":"2024-09-15T21:20:01.886910+0000","last_became_peered":"2024-09-15T21:20:01.886910+0000","last_unstale":"2024-09-15T21:20:01.888406+0000","last_undegraded":"2024-09-15T21:20:01.888406+0000","last_fullsized":"2024-09-15T21:20:01.888406+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:33.977199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":36,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:01.887321+0000","last_change":"2024-09-15T21:20:01.887494+0000","last_active":"2024-09-15T21:20:01.887321+0000","last_peered":"2024-09-15T21:20:01.887321+0000","last_clean":"2024-09-15T21:19:50.164448+0000","last_became_active":"2024-09-15T21:20:01.887211+0000","last_became_peered":"2024-09-15T21:20:01.887211+0000","last_unstale":"2024-09-15T21:20:01.887321+0000","last_undegraded":"2024-09-15T21:20:01.880521+0000","last_fullsized":"2024-09-15T21:20:01.887321+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":39,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:01.883364+0000","last_change":"2024-09-15T21:20:01.883526+0000","last_active":"2024-09-15T21:20:01.883364+0000","last_peered":"2024-09-15T21:20:01.883364+0000","last_clean":"2024-09-15T21:19:51.013140+0000","last_became_active":"2024-09-15T21:20:01.883274+0000","last_became_peered":"2024-09-15T21:20:01.883274+0000","last_unstale":"2024-09-15T21:20:01.883364+0000","last_undegraded":"2024-09-15T21:20:01.871699+0000","last_fullsized":"2024-09-15T21:20:01.883364+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":40,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:01.883152+0000","last_change":"2024-09-15T21:20:01.883360+0000","last_active":"2024-09-15T21:20:01.883152+0000","last_peered":"2024-09-15T21:20:01.883152+0000","last_clean":"2024-09-15T21:19:50.413489+0000","last_became_active":"2024-09-15T21:20:01.883061+0000","last_became_peered":"2024-09-15T21:20:01.883061+0000","last_unstale":"2024-09-15T21:20:01.883152+0000","last_undegraded":"2024-09-15T21:20:01.871027+0000","last_fullsized":"2024-09-15T21:20:01.883152+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":33,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:01.887000+0000","last_change":"2024-09-15T21:20:01.887604+0000","last_active":"2024-09-15T21:20:01.887000+0000","last_peered":"2024-09-15T21:20:01.887000+0000","last_clean":"2024-09-15T21:19:36.740313+0000","last_became_active":"2024-09-15T21:20:01.886915+0000","last_became_peered":"2024-09-15T21:20:01.886915+0000","last_unstale":"2024-09-15T21:20:01.887000+0000","last_undegraded":"2024-09-15T21:20:01.880290+0000","last_fullsized":"2024-09-15T21:20:01.887000+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.883547+0000","last_change":"2024-09-15T21:20:01.883430+0000","last_active":"2024-09-15T21:20:01.883547+0000","last_peered":"2024-09-15T21:20:01.883547+0000","last_clean":"2024-09-15T21:20:01.883547+0000","last_became_active":"2024-09-15T21:20:01.883105+0000","last_became_peered":"2024-09-15T21:20:01.883105+0000","last_unstale":"2024-09-15T21:20:01.883547+0000","last_undegraded":"2024-09-15T21:20:01.883547+0000","last_fullsized":"2024-09-15T21:20:01.883547+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:09.125353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":46,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:01.883615+0000","last_change":"2024-09-15T21:20:01.883768+0000","last_active":"2024-09-15T21:20:01.883615+0000","last_peered":"2024-09-15T21:20:01.883615+0000","last_clean":"2024-09-15T21:19:50.864243+0000","last_became_active":"2024-09-15T21:20:01.883526+0000","last_became_peered":"2024-09-15T21:20:01.883526+0000","last_unstale":"2024-09-15T21:20:01.883615+0000","last_undegraded":"2024-09-15T21:20:01.872554+0000","last_fullsized":"2024-09-15T21:20:01.883615+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.884102+0000","last_change":"2024-09-15T21:20:01.884003+0000","last_active":"2024-09-15T21:20:01.884102+0000","last_peered":"2024-09-15T21:20:01.884102+0000","last_clean":"2024-09-15T21:20:01.884102+0000","last_became_active":"2024-09-15T21:20:01.883680+0000","last_became_peered":"2024-09-15T21:20:01.883680+0000","last_unstale":"2024-09-15T21:20:01.884102+0000","last_undegraded":"2024-09-15T21:20:01.884102+0000","last_fullsized":"2024-09-15T21:20:01.884102+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:51:46.022176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":32,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:01.887279+0000","last_change":"2024-09-15T21:20:01.887463+0000","last_active":"2024-09-15T21:20:01.887279+0000","last_peered":"2024-09-15T21:20:01.887279+0000","last_clean":"2024-09-15T21:19:51.063279+0000","last_became_active":"2024-09-15T21:20:01.887184+0000","last_became_peered":"2024-09-15T21:20:01.887184+0000","last_unstale":"2024-09-15T21:20:01.887279+0000","last_undegraded":"2024-09-15T21:20:01.880331+0000","last_fullsized":"2024-09-15T21:20:01.887279+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.886799+0000","last_change":"2024-09-15T21:20:01.886694+0000","last_active":"2024-09-15T21:20:01.886799+0000","last_peered":"2024-09-15T21:20:01.886799+0000","last_clean":"2024-09-15T21:20:01.886799+0000","last_became_active":"2024-09-15T21:20:01.886134+0000","last_became_peered":"2024-09-15T21:20:01.886134+0000","last_unstale":"2024-09-15T21:20:01.886799+0000","last_undegraded":"2024-09-15T21:20:01.886799+0000","last_fullsized":"2024-09-15T21:20:01.886799+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:06:26.531458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":34,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:01.885169+0000","last_change":"2024-09-15T21:20:01.885461+0000","last_active":"2024-09-15T21:20:01.885169+0000","last_peered":"2024-09-15T21:20:01.885169+0000","last_clean":"2024-09-15T21:19:50.113984+0000","last_became_active":"2024-09-15T21:20:01.885076+0000","last_became_peered":"2024-09-15T21:20:01.885076+0000","last_unstale":"2024-09-15T21:20:01.885169+0000","last_undegraded":"2024-09-15T21:20:01.871231+0000","last_fullsized":"2024-09-15T21:20:01.885169+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":63,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.885742+0000","last_change":"2024-09-15T21:20:01.885650+0000","last_active":"2024-09-15T21:20:01.885742+0000","last_peered":"2024-09-15T21:20:01.885742+0000","last_clean":"2024-09-15T21:20:01.885742+0000","last_became_active":"2024-09-15T21:20:01.885016+0000","last_became_peered":"2024-09-15T21:20:01.885016+0000","last_unstale":"2024-09-15T21:20:01.885742+0000","last_undegraded":"2024-09-15T21:20:01.885742+0000","last_fullsized":"2024-09-15T21:20:01.885742+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:04:10.959799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":35,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:01.883370+0000","last_change":"2024-09-15T21:20:01.883653+0000","last_active":"2024-09-15T21:20:01.883370+0000","last_peered":"2024-09-15T21:20:01.883370+0000","last_clean":"2024-09-15T21:19:50.513634+0000","last_became_active":"2024-09-15T21:20:01.883288+0000","last_became_peered":"2024-09-15T21:20:01.883288+0000","last_unstale":"2024-09-15T21:20:01.883370+0000","last_undegraded":"2024-09-15T21:20:01.869738+0000","last_fullsized":"2024-09-15T21:20:01.883370+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":36,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:01.885291+0000","last_change":"2024-09-15T21:20:01.885456+0000","last_active":"2024-09-15T21:20:01.885291+0000","last_peered":"2024-09-15T21:20:01.885291+0000","last_clean":"2024-09-15T21:19:39.745933+0000","last_became_active":"2024-09-15T21:20:01.885214+0000","last_became_peered":"2024-09-15T21:20:01.885214+0000","last_unstale":"2024-09-15T21:20:01.885291+0000","last_undegraded":"2024-09-15T21:20:01.874206+0000","last_fullsized":"2024-09-15T21:20:01.885291+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.887788+0000","last_change":"2024-09-15T21:20:01.887699+0000","last_active":"2024-09-15T21:20:01.887788+0000","last_peered":"2024-09-15T21:20:01.887788+0000","last_clean":"2024-09-15T21:20:01.887788+0000","last_became_active":"2024-09-15T21:20:01.887331+0000","last_became_peered":"2024-09-15T21:20:01.887331+0000","last_unstale":"2024-09-15T21:20:01.887788+0000","last_undegraded":"2024-09-15T21:20:01.887788+0000","last_fullsized":"2024-09-15T21:20:01.887788+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:36:09.018870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":41,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:01.885160+0000","last_change":"2024-09-15T21:20:01.885342+0000","last_active":"2024-09-15T21:20:01.885160+0000","last_peered":"2024-09-15T21:20:01.885160+0000","last_clean":"2024-09-15T21:19:50.963035+0000","last_became_active":"2024-09-15T21:20:01.885073+0000","last_became_peered":"2024-09-15T21:20:01.885073+0000","last_unstale":"2024-09-15T21:20:01.885160+0000","last_undegraded":"2024-09-15T21:20:01.873279+0000","last_fullsized":"2024-09-15T21:20:01.885160+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.885962+0000","last_change":"2024-09-15T21:20:01.885871+0000","last_active":"2024-09-15T21:20:01.885962+0000","last_peered":"2024-09-15T21:20:01.885962+0000","last_clean":"2024-09-15T21:20:01.885962+0000","last_became_active":"2024-09-15T21:20:01.885652+0000","last_became_peered":"2024-09-15T21:20:01.885652+0000","last_unstale":"2024-09-15T21:20:01.885962+0000","last_undegraded":"2024-09-15T21:20:01.885962+0000","last_fullsized":"2024-09-15T21:20:01.885962+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:44:58.056893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":37,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:01.885277+0000","last_change":"2024-09-15T21:20:01.885464+0000","last_active":"2024-09-15T21:20:01.885277+0000","last_peered":"2024-09-15T21:20:01.885277+0000","last_clean":"2024-09-15T21:19:39.745731+0000","last_became_active":"2024-09-15T21:20:01.885190+0000","last_became_peered":"2024-09-15T21:20:01.885190+0000","last_unstale":"2024-09-15T21:20:01.885277+0000","last_undegraded":"2024-09-15T21:20:01.873454+0000","last_fullsized":"2024-09-15T21:20:01.885277+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.885318+0000","last_change":"2024-09-15T21:20:01.885206+0000","last_active":"2024-09-15T21:20:01.885318+0000","last_peered":"2024-09-15T21:20:01.885318+0000","last_clean":"2024-09-15T21:20:01.885318+0000","last_became_active":"2024-09-15T21:20:01.884904+0000","last_became_peered":"2024-09-15T21:20:01.884904+0000","last_unstale":"2024-09-15T21:20:01.885318+0000","last_undegraded":"2024-09-15T21:20:01.885318+0000","last_fullsized":"2024-09-15T21:20:01.885318+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:54:45.233262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":58,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-15T21:20:01.886045+0000","last_change":"2024-09-15T21:20:01.885960+0000","last_active":"2024-09-15T21:20:01.886045+0000","last_peered":"2024-09-15T21:20:01.886045+0000","last_clean":"2024-09-15T21:20:01.886045+0000","last_became_active":"2024-09-15T21:20:01.885635+0000","last_became_peered":"2024-09-15T21:20:01.885635+0000","last_unstale":"2024-09-15T21:20:01.886045+0000","last_undegraded":"2024-09-15T21:20:01.886045+0000","last_fullsized":"2024-09-15T21:20:01.886045+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:26.133017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":45,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:01.887532+0000","last_change":"2024-09-15T21:20:01.887790+0000","last_active":"2024-09-15T21:20:01.887532+0000","last_peered":"2024-09-15T21:20:01.887532+0000","last_clean":"2024-09-15T21:19:50.713871+0000","last_became_active":"2024-09-15T21:20:01.887434+0000","last_became_peered":"2024-09-15T21:20:01.887434+0000","last_unstale":"2024-09-15T21:20:01.887532+0000","last_undegraded":"2024-09-15T21:20:01.872552+0000","last_fullsized":"2024-09-15T21:20:01.887532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"0,1","objects":1},{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182403,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":260,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":266240,"data_stored":77641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28110507},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738383,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":339968,"data_stored":86335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:08.533 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:08.882 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:08.883 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":60,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":26,"num_osds":3,"num_up_osds":2,"osd_up_since":1726435200,"num_in_osds":2,"osd_in_since":1726435178,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+degraded","count":12},{"state_name":"active+clean","count":10},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56426496,"bytes_avail":193217101824,"bytes_total":193273528320,"degraded_objects":27,"degraded_total":60,"degraded_ratio":0.45000000000000001},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-15T21:19:07.254830+0000","services":{}},"progress_events":{}} 2024-09-15T21:20:08.895 INFO:tasks.lost_unfound.ceph_manager:{'fsid': '1d60fae4-b8c3-4364-9331-b0445493a043', 'health': {'status': 'HEALTH_WARN', 'checks': {'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded', 'count': 14}, 'muted': False}}, 'mutes': []}, 'election_epoch': 10, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 60, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 26, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726435200, 'num_in_osds': 2, 'osd_in_since': 1726435178, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+recovery_wait+degraded', 'count': 12}, {'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovering+degraded', 'count': 2}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56426496, 'bytes_avail': 193217101824, 'bytes_total': 193273528320, 'degraded_objects': 27, 'degraded_total': 60, 'degraded_ratio': 0.45}, 'fsmap': {'epoch': 1, 'btime': '2024-09-15T21:19:07:253728+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 1, 'modified': '2024-09-15T21:19:07.254830+0000', 'services': {}}, 'progress_events': {}} 2024-09-15T21:20:08.895 INFO:tasks.lost_unfound.ceph_manager:active! 2024-09-15T21:20:08.896 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T21:20:08.896 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:20:08.920 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T21:20:08.921 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-15T21:20:09.251 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-15T21:20:09.248+0000 7f8e1ee55640 -1 mon.a@0(leader).osd e26 definitely_dead 0 2024-09-15T21:20:09.909 INFO:teuthology.orchestra.run.smithi073.stderr:marked down osd.1. 2024-09-15T21:20:09.926 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-15T21:20:10.220 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-15T21:20:10.217+0000 7f8e1ee55640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-15T21:20:10.914 INFO:teuthology.orchestra.run.smithi073.stderr:marked out osd.1. 2024-09-15T21:20:10.930 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd lost 1 --yes-i-really-mean-it 2024-09-15T21:20:11.912 INFO:teuthology.orchestra.run.smithi073.stderr:marked osd lost in epoch 27 2024-09-15T21:20:11.916 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.910+0000 7fa5351e0640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-15T21:20:11.916 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.910+0000 7fa5349df640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-15T21:20:11.916 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.910+0000 7fa5361e2640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-15T21:20:11.916 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.910+0000 7fa5329db640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-15T21:20:11.916 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.911+0000 7fa5389e7640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-15T21:20:11.916 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.911+0000 7fa5329db640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-15T21:20:11.916 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.911+0000 7fa5351e0640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-15T21:20:11.916 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.911+0000 7fa5339dd640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-15T21:20:11.916 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.911+0000 7fa5341de640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-15T21:20:11.916 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.911+0000 7fa5379e5640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-15T21:20:11.916 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.911+0000 7fa5381e6640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-15T21:20:11.917 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.911+0000 7fa5371e4640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-15T21:20:11.917 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.911+0000 7fa5321da640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-15T21:20:11.917 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:11.912+0000 7fa5359e1640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-15T21:20:11.926 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-15T21:20:11.926 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-15T21:20:11.928 INFO:tasks.ceph.osd.2:Started 2024-09-15T21:20:11.928 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:20:12.025 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-15T21:20:12.025 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-15T21:20:12.026 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-15T21:20:12.026 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-15T21:20:12.026 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-15T21:20:12.026 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-15T21:20:12.026 INFO:teuthology.orchestra.run.smithi073.stderr:config diff 2024-09-15T21:20:12.026 INFO:teuthology.orchestra.run.smithi073.stderr:config diff get 2024-09-15T21:20:12.026 INFO:teuthology.orchestra.run.smithi073.stderr:config get 2024-09-15T21:20:12.026 INFO:teuthology.orchestra.run.smithi073.stderr:config help [] 2024-09-15T21:20:12.026 INFO:teuthology.orchestra.run.smithi073.stderr:config set ... 2024-09-15T21:20:12.026 INFO:teuthology.orchestra.run.smithi073.stderr:admin_socket: invalid command 2024-09-15T21:20:12.028 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:20:12.028 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T21:20:13.101 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-15T21:20:13.097+0000 7f9ca4134640 -1 Falling back to public interface 2024-09-15T21:20:17.029 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:20:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-15T21:20:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-15T21:20:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-15T21:20:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-15T21:20:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-15T21:20:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-15T21:20:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-15T21:20:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-15T21:20:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-15T21:20:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-15T21:20:17.142 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator fragmentation block 2024-09-15T21:20:17.143 INFO:teuthology.orchestra.run.smithi073.stderr:admin_socket: invalid command 2024-09-15T21:20:17.144 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:20:17.144 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-15T21:20:17.152 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-15T21:20:17.149+0000 7f9ca4134640 -1 osd.2 17 log_to_monitors true 2024-09-15T21:20:17.981 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.977+0000 7fa5351e0640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-15T21:20:17.981 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.977+0000 7fa5349df640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-15T21:20:17.981 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.977+0000 7fa5329db640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-15T21:20:17.981 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.977+0000 7fa5351e0640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-15T21:20:17.981 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.977+0000 7fa5379e5640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-15T21:20:17.982 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.977+0000 7fa5319d9640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-15T21:20:17.984 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.978+0000 7fa5329db640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-15T21:20:17.984 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.978+0000 7fa5349df640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-15T21:20:17.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.978+0000 7fa5331dc640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-15T21:20:17.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.978+0000 7fa5381e6640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-15T21:20:17.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.978+0000 7fa5321da640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-15T21:20:17.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.978+0000 7fa5361e2640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-15T21:20:17.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.978+0000 7fa5339dd640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-15T21:20:17.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:17.978+0000 7fa5341de640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-15T21:20:17.994 INFO:tasks.ceph.osd.2.smithi073.stderr:2024-09-15T21:20:17.990+0000 7f9c9beea640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:20:18.984 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.979+0000 7fa5331dc640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-15T21:20:18.984 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.979+0000 7fa5361e2640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-15T21:20:18.984 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.979+0000 7fa5329db640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-15T21:20:18.984 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.979+0000 7fa5381e6640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-15T21:20:18.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.979+0000 7fa5329db640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-15T21:20:18.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.979+0000 7fa5381e6640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-15T21:20:18.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.979+0000 7fa5361e2640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-15T21:20:18.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.980+0000 7fa5389e7640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-15T21:20:18.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.980+0000 7fa5339dd640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-15T21:20:18.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.980+0000 7fa5389e7640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-15T21:20:18.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.980+0000 7fa5319d9640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-15T21:20:18.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.980+0000 7fa5339dd640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-15T21:20:18.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.981+0000 7fa5391e8640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-15T21:20:18.985 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:18.981+0000 7fa5351e0640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-15T21:20:19.997 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:19.992+0000 7fa5341de640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-15T21:20:19.997 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:19.992+0000 7fa5381e6640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-15T21:20:19.998 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:19.992+0000 7fa5331dc640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-15T21:20:19.998 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:19.993+0000 7fa5349df640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-15T21:20:19.998 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:19.993+0000 7fa5359e1640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-15T21:20:19.998 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:19.993+0000 7fa5369e3640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-15T21:20:19.999 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:19.995+0000 7fa5321da640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-15T21:20:21.005 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:21.000+0000 7fa5341de640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-15T21:20:21.005 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:21.000+0000 7fa5341de640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-15T21:20:21.006 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:21.001+0000 7fa5331dc640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-15T21:20:21.006 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:21.001+0000 7fa5329db640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-15T21:20:21.006 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:21.001+0000 7fa5359e1640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-15T21:20:21.007 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:21.002+0000 7fa5349df640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-15T21:20:21.007 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:21.003+0000 7fa5361e2640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-15T21:20:22.146 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-15T21:20:22.262 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:22.262 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-09-15T21:20:22.262 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-09-15T21:20:22.262 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:22.271 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-15T21:20:22.562 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-15T21:20:22.558+0000 7f8e1ee55640 -1 mon.a@0(leader).osd e33 definitely_dead 0 2024-09-15T21:20:23.020 INFO:teuthology.orchestra.run.smithi073.stderr:marked in osd.2. 2024-09-15T21:20:23.035 INFO:tasks.lost_unfound.ceph_manager:waiting for osd.2 to be up 2024-09-15T21:20:23.036 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:20:23.318 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:23.319 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","created":"2024-09-15T21:19:07.254022+0000","modified":"2024-09-15T21:20:23.000148+0000","last_up_change":"2024-09-15T21:20:18.975075+0000","last_in_change":"2024-09-15T21:20:23.000148+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:19:25.465189+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-15T21:19:34.551635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c55f097a-59ed-4ce1-9803-4c6019429ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":32,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6801","nonce":173704734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6803","nonce":173704734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6807","nonce":173704734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6805","nonce":173704734}]},"public_addr":"172.21.15.73:6801/173704734","cluster_addr":"172.21.15.73:6803/173704734","heartbeat_back_addr":"172.21.15.73:6807/173704734","heartbeat_front_addr":"172.21.15.73:6805/173704734","state":["exists","up"]},{"osd":1,"uuid":"7daff6cd-a09c-4c15-a282-0859e8372dd3","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6809","nonce":1755138815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6811","nonce":1755138815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6815","nonce":1755138815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1755138815},{"type":"v1","addr":"172.21.15.73:6813","nonce":1755138815}]},"public_addr":"172.21.15.73:6809/1755138815","cluster_addr":"172.21.15.73:6811/1755138815","heartbeat_back_addr":"172.21.15.73:6815/1755138815","heartbeat_front_addr":"172.21.15.73:6813/1755138815","state":["exists"]},{"osd":2,"uuid":"9abad4dd-4a88-4080-9ab4-5c4203d7fd51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":17,"up_from":31,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":508991462},{"type":"v1","addr":"172.21.15.73:6809","nonce":508991462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":508991462},{"type":"v1","addr":"172.21.15.73:6811","nonce":508991462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":508991462},{"type":"v1","addr":"172.21.15.73:6815","nonce":508991462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":508991462},{"type":"v1","addr":"172.21.15.73:6813","nonce":508991462}]},"public_addr":"172.21.15.73:6809/508991462","cluster_addr":"172.21.15.73:6811/508991462","heartbeat_back_addr":"172.21.15.73:6815/508991462","heartbeat_front_addr":"172.21.15.73:6813/508991462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.867529+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-15T21:19:17.608114+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.891920+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.1","osds":[0,2]},{"pgid":"1.3","osds":[0,2]},{"pgid":"1.5","osds":[0,2]},{"pgid":"1.7","osds":[0,2]},{"pgid":"2.0","osds":[0,2]},{"pgid":"2.1","osds":[0,2]},{"pgid":"2.3","osds":[0,2]},{"pgid":"2.c","osds":[0,2]},{"pgid":"2.d","osds":[0,2]},{"pgid":"2.e","osds":[0,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:20:23.332 INFO:tasks.lost_unfound.ceph_manager:osd.2 is up 2024-09-15T21:20:23.332 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:20:23.333 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:20:23.483 INFO:teuthology.orchestra.run.smithi073.stdout:133143986179 2024-09-15T21:20:23.483 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:20:23.484 INFO:teuthology.orchestra.run.smithi073.stdout:107374182407 2024-09-15T21:20:23.485 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:20:23.815 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-09-15T21:20:23.818 INFO:teuthology.orchestra.run.smithi073.stdout:107374182406 2024-09-15T21:20:23.829 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986179 got 1 for osd.2 2024-09-15T21:20:23.831 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182407 got 107374182406 for osd.0 2024-09-15T21:20:24.829 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:20:24.832 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:20:25.036 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:25.032+0000 7fa5391e8640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-15T21:20:25.037 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:25.032+0000 7fa5329db640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-15T21:20:25.037 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:25.032+0000 7fa5321da640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-15T21:20:25.037 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:25.032+0000 7fa5389e7640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-15T21:20:25.037 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:25.033+0000 7fa5359e1640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-15T21:20:25.037 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:25.033+0000 7fa5371e4640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-15T21:20:25.037 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:25.033+0000 7fa5379e5640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-15T21:20:25.038 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:25.033+0000 7fa5381e6640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-15T21:20:25.038 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-09-15T21:20:25.033+0000 7fa5381e6640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-15T21:20:25.156 INFO:teuthology.orchestra.run.smithi073.stdout:107374182407 2024-09-15T21:20:25.157 INFO:teuthology.orchestra.run.smithi073.stdout:133143986179 2024-09-15T21:20:25.170 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182407 got 107374182407 for osd.0 2024-09-15T21:20:25.170 DEBUG:teuthology.parallel:result is None 2024-09-15T21:20:25.171 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986179 got 133143986179 for osd.2 2024-09-15T21:20:25.171 DEBUG:teuthology.parallel:result is None 2024-09-15T21:20:25.171 INFO:tasks.lost_unfound.ceph_manager:waiting till active 2024-09-15T21:20:25.171 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:25.452 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:25.452 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:20:25.464 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-15T21:20:25.029273+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":18,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55616,"kb_used_data":596,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688064,"statfs":{"total":193273528320,"available":193216577536,"internally_reserved":0,"allocated":610304,"data_stored":203295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":56286548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-25,"num_objects_misplaced":0,"num_objects_unfound":-9,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.351109"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":50,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-15T21:20:24.031808+0000","last_change":"2024-09-15T21:20:24.031714+0000","last_active":"2024-09-15T21:20:24.031808+0000","last_peered":"2024-09-15T21:20:24.031808+0000","last_clean":"2024-09-15T21:20:24.031808+0000","last_became_active":"2024-09-15T21:20:24.031439+0000","last_became_peered":"2024-09-15T21:20:24.031439+0000","last_unstale":"2024-09-15T21:20:24.031808+0000","last_undegraded":"2024-09-15T21:20:24.031808+0000","last_fullsized":"2024-09-15T21:20:24.031808+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:51.235292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":46,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-15T21:20:21.008632+0000","last_change":"2024-09-15T21:20:23.016734+0000","last_active":"2024-09-15T21:20:23.016734+0000","last_peered":"2024-09-15T21:20:21.008632+0000","last_clean":"2024-09-15T21:19:50.263852+0000","last_became_active":"2024-09-15T21:20:21.008202+0000","last_became_peered":"2024-09-15T21:20:21.008202+0000","last_unstale":"2024-09-15T21:20:21.008632+0000","last_undegraded":"2024-09-15T21:20:21.005446+0000","last_fullsized":"2024-09-15T21:20:21.008632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":60,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-15T21:20:21.009417+0000","last_change":"2024-09-15T21:20:23.017859+0000","last_active":"2024-09-15T21:20:23.017859+0000","last_peered":"2024-09-15T21:20:21.009417+0000","last_clean":"2024-09-15T21:19:50.814098+0000","last_became_active":"2024-09-15T21:20:21.009047+0000","last_became_peered":"2024-09-15T21:20:21.009047+0000","last_unstale":"2024-09-15T21:20:21.009417+0000","last_undegraded":"2024-09-15T21:20:21.002434+0000","last_fullsized":"2024-09-15T21:20:21.009417+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":50,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-15T21:20:21.013129+0000","last_change":"2024-09-15T21:20:23.018657+0000","last_active":"2024-09-15T21:20:23.018657+0000","last_peered":"2024-09-15T21:20:21.013129+0000","last_clean":"2024-09-15T21:20:21.013129+0000","last_became_active":"2024-09-15T21:20:21.008016+0000","last_became_peered":"2024-09-15T21:20:21.008016+0000","last_unstale":"2024-09-15T21:20:21.013129+0000","last_undegraded":"2024-09-15T21:20:21.013129+0000","last_fullsized":"2024-09-15T21:20:21.013129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":63,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.035961+0000","last_change":"2024-09-15T21:20:24.036146+0000","last_active":"2024-09-15T21:20:24.035961+0000","last_peered":"2024-09-15T21:20:24.035961+0000","last_clean":"2024-09-15T21:19:50.164448+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:20:24.035961+0000","last_undegraded":"2024-09-15T21:20:24.028206+0000","last_fullsized":"2024-09-15T21:20:24.035961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":59,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.035787+0000","last_change":"2024-09-15T21:20:24.035954+0000","last_active":"2024-09-15T21:20:24.035787+0000","last_peered":"2024-09-15T21:20:24.035787+0000","last_clean":"2024-09-15T21:19:51.013140+0000","last_became_active":"2024-09-15T21:20:24.035700+0000","last_became_peered":"2024-09-15T21:20:24.035700+0000","last_unstale":"2024-09-15T21:20:24.035787+0000","last_undegraded":"2024-09-15T21:20:24.025973+0000","last_fullsized":"2024-09-15T21:20:24.035787+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":59,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.036470+0000","last_change":"2024-09-15T21:20:24.036588+0000","last_active":"2024-09-15T21:20:24.036470+0000","last_peered":"2024-09-15T21:20:24.036470+0000","last_clean":"2024-09-15T21:19:50.413489+0000","last_became_active":"2024-09-15T21:20:24.036417+0000","last_became_peered":"2024-09-15T21:20:24.036417+0000","last_unstale":"2024-09-15T21:20:24.036470+0000","last_undegraded":"2024-09-15T21:20:24.028324+0000","last_fullsized":"2024-09-15T21:20:24.036470+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":60,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.032281+0000","last_change":"2024-09-15T21:20:24.032808+0000","last_active":"2024-09-15T21:20:24.032281+0000","last_peered":"2024-09-15T21:20:24.032281+0000","last_clean":"2024-09-15T21:19:36.740313+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:20:24.032281+0000","last_undegraded":"2024-09-15T21:20:24.026851+0000","last_fullsized":"2024-09-15T21:20:24.032281+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-15T21:20:24.032969+0000","last_change":"2024-09-15T21:20:24.032881+0000","last_active":"2024-09-15T21:20:24.032969+0000","last_peered":"2024-09-15T21:20:24.032969+0000","last_clean":"2024-09-15T21:20:24.032969+0000","last_became_active":"2024-09-15T21:20:24.032040+0000","last_became_peered":"2024-09-15T21:20:24.032040+0000","last_unstale":"2024-09-15T21:20:24.032969+0000","last_undegraded":"2024-09-15T21:20:24.032969+0000","last_fullsized":"2024-09-15T21:20:24.032969+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:46:45.703820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":62,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-15T21:20:21.009061+0000","last_change":"2024-09-15T21:20:23.017953+0000","last_active":"2024-09-15T21:20:23.017953+0000","last_peered":"2024-09-15T21:20:21.009061+0000","last_clean":"2024-09-15T21:19:50.864243+0000","last_became_active":"2024-09-15T21:20:21.008498+0000","last_became_peered":"2024-09-15T21:20:21.008498+0000","last_unstale":"2024-09-15T21:20:21.009061+0000","last_undegraded":"2024-09-15T21:20:21.003604+0000","last_fullsized":"2024-09-15T21:20:21.009061+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-15T21:20:21.013272+0000","last_change":"2024-09-15T21:20:23.016074+0000","last_active":"2024-09-15T21:20:23.016074+0000","last_peered":"2024-09-15T21:20:21.013272+0000","last_clean":"2024-09-15T21:20:21.013272+0000","last_became_active":"2024-09-15T21:20:21.008902+0000","last_became_peered":"2024-09-15T21:20:21.008902+0000","last_unstale":"2024-09-15T21:20:21.013272+0000","last_undegraded":"2024-09-15T21:20:21.013272+0000","last_fullsized":"2024-09-15T21:20:21.013272+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":55,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.035810+0000","last_change":"2024-09-15T21:20:24.035959+0000","last_active":"2024-09-15T21:20:24.035810+0000","last_peered":"2024-09-15T21:20:24.035810+0000","last_clean":"2024-09-15T21:19:51.063279+0000","last_became_active":"2024-09-15T21:20:24.035738+0000","last_became_peered":"2024-09-15T21:20:24.035738+0000","last_unstale":"2024-09-15T21:20:24.035810+0000","last_undegraded":"2024-09-15T21:20:24.026174+0000","last_fullsized":"2024-09-15T21:20:24.035810+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":62,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.032123+0000","last_change":"2024-09-15T21:20:24.032266+0000","last_active":"2024-09-15T21:20:24.032123+0000","last_peered":"2024-09-15T21:20:24.032123+0000","last_clean":"2024-09-15T21:20:01.890484+0000","last_became_active":"2024-09-15T21:20:24.032032+0000","last_became_peered":"2024-09-15T21:20:24.032032+0000","last_unstale":"2024-09-15T21:20:24.032123+0000","last_undegraded":"2024-09-15T21:20:24.025362+0000","last_fullsized":"2024-09-15T21:20:24.032123+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":50,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-15T21:20:21.007674+0000","last_change":"2024-09-15T21:20:23.017434+0000","last_active":"2024-09-15T21:20:23.017434+0000","last_peered":"2024-09-15T21:20:21.007674+0000","last_clean":"2024-09-15T21:19:50.113984+0000","last_became_active":"2024-09-15T21:20:21.007378+0000","last_became_peered":"2024-09-15T21:20:21.007378+0000","last_unstale":"2024-09-15T21:20:21.007674+0000","last_undegraded":"2024-09-15T21:20:21.002842+0000","last_fullsized":"2024-09-15T21:20:21.007674+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":57,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-15T21:20:21.013346+0000","last_change":"2024-09-15T21:20:23.018745+0000","last_active":"2024-09-15T21:20:23.018745+0000","last_peered":"2024-09-15T21:20:21.013346+0000","last_clean":"2024-09-15T21:20:21.013346+0000","last_became_active":"2024-09-15T21:20:21.008760+0000","last_became_peered":"2024-09-15T21:20:21.008760+0000","last_unstale":"2024-09-15T21:20:21.013346+0000","last_undegraded":"2024-09-15T21:20:21.013346+0000","last_fullsized":"2024-09-15T21:20:21.013346+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":51,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-15T21:20:21.009369+0000","last_change":"2024-09-15T21:20:23.016251+0000","last_active":"2024-09-15T21:20:23.016251+0000","last_peered":"2024-09-15T21:20:21.009369+0000","last_clean":"2024-09-15T21:19:50.513634+0000","last_became_active":"2024-09-15T21:20:21.008044+0000","last_became_peered":"2024-09-15T21:20:21.008044+0000","last_unstale":"2024-09-15T21:20:21.009369+0000","last_undegraded":"2024-09-15T21:20:21.003586+0000","last_fullsized":"2024-09-15T21:20:21.009369+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":55,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.032282+0000","last_change":"2024-09-15T21:20:24.032431+0000","last_active":"2024-09-15T21:20:24.032282+0000","last_peered":"2024-09-15T21:20:24.032282+0000","last_clean":"2024-09-15T21:19:39.745933+0000","last_became_active":"2024-09-15T21:20:24.032201+0000","last_became_peered":"2024-09-15T21:20:24.032201+0000","last_unstale":"2024-09-15T21:20:24.032282+0000","last_undegraded":"2024-09-15T21:20:24.024923+0000","last_fullsized":"2024-09-15T21:20:24.032282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-15T21:20:24.032816+0000","last_change":"2024-09-15T21:20:24.032468+0000","last_active":"2024-09-15T21:20:24.032816+0000","last_peered":"2024-09-15T21:20:24.032816+0000","last_clean":"2024-09-15T21:20:24.032816+0000","last_became_active":"2024-09-15T21:20:24.032206+0000","last_became_peered":"2024-09-15T21:20:24.032206+0000","last_unstale":"2024-09-15T21:20:24.032816+0000","last_undegraded":"2024-09-15T21:20:24.032816+0000","last_fullsized":"2024-09-15T21:20:24.032816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:48.044077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":61,"reported_epoch":35,"state":"active+recovering+degraded","last_fresh":"2024-09-15T21:20:24.031849+0000","last_change":"2024-09-15T21:20:24.032926+0000","last_active":"2024-09-15T21:20:24.031849+0000","last_peered":"2024-09-15T21:20:24.031849+0000","last_clean":"2024-09-15T21:19:50.963035+0000","last_became_active":"2024-09-15T21:20:24.031529+0000","last_became_peered":"2024-09-15T21:20:24.031529+0000","last_unstale":"2024-09-15T21:20:24.031849+0000","last_undegraded":"2024-09-15T21:20:24.026166+0000","last_fullsized":"2024-09-15T21:20:24.031849+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-15T21:20:21.013221+0000","last_change":"2024-09-15T21:20:23.017283+0000","last_active":"2024-09-15T21:20:23.017283+0000","last_peered":"2024-09-15T21:20:21.013221+0000","last_clean":"2024-09-15T21:20:21.013221+0000","last_became_active":"2024-09-15T21:20:21.008131+0000","last_became_peered":"2024-09-15T21:20:21.008131+0000","last_unstale":"2024-09-15T21:20:21.013221+0000","last_undegraded":"2024-09-15T21:20:21.013221+0000","last_fullsized":"2024-09-15T21:20:21.013221+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":56,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.035999+0000","last_change":"2024-09-15T21:20:24.036145+0000","last_active":"2024-09-15T21:20:24.035999+0000","last_peered":"2024-09-15T21:20:24.035999+0000","last_clean":"2024-09-15T21:19:39.745731+0000","last_became_active":"2024-09-15T21:20:24.035699+0000","last_became_peered":"2024-09-15T21:20:24.035699+0000","last_unstale":"2024-09-15T21:20:24.035999+0000","last_undegraded":"2024-09-15T21:20:24.026554+0000","last_fullsized":"2024-09-15T21:20:24.035999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-15T21:20:24.031827+0000","last_change":"2024-09-15T21:20:24.031742+0000","last_active":"2024-09-15T21:20:24.031827+0000","last_peered":"2024-09-15T21:20:24.031827+0000","last_clean":"2024-09-15T21:20:24.031827+0000","last_became_active":"2024-09-15T21:20:24.031400+0000","last_became_peered":"2024-09-15T21:20:24.031400+0000","last_unstale":"2024-09-15T21:20:24.031827+0000","last_undegraded":"2024-09-15T21:20:24.031827+0000","last_fullsized":"2024-09-15T21:20:24.031827+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:36:52.618460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-15T21:20:21.013201+0000","last_change":"2024-09-15T21:20:23.015516+0000","last_active":"2024-09-15T21:20:23.015516+0000","last_peered":"2024-09-15T21:20:21.013201+0000","last_clean":"2024-09-15T21:20:21.013201+0000","last_became_active":"2024-09-15T21:20:21.008245+0000","last_became_peered":"2024-09-15T21:20:21.008245+0000","last_unstale":"2024-09-15T21:20:21.013201+0000","last_undegraded":"2024-09-15T21:20:21.013201+0000","last_fullsized":"2024-09-15T21:20:21.013201+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":64,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.035746+0000","last_change":"2024-09-15T21:20:24.035912+0000","last_active":"2024-09-15T21:20:24.035746+0000","last_peered":"2024-09-15T21:20:24.035746+0000","last_clean":"2024-09-15T21:19:50.713871+0000","last_became_active":"2024-09-15T21:20:24.035656+0000","last_became_peered":"2024-09-15T21:20:24.035656+0000","last_unstale":"2024-09-15T21:20:24.035746+0000","last_undegraded":"2024-09-15T21:20:24.026450+0000","last_fullsized":"2024-09-15T21:20:24.035746+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":18,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":278528,"data_stored":97343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":28176041},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182408,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":324,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":331776,"data_stored":105952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28110507},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:25.466 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:25.814 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:25.814 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":77,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726435218,"num_in_osds":2,"osd_in_since":1726435223,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":14},{"state_name":"remapped+peering","count":10}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56938496,"bytes_avail":193216589824,"bytes_total":193273528320,"inactive_pgs_ratio":1},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-15T21:20:20.678211+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:20:25.827 INFO:tasks.lost_unfound.ceph_manager:{'fsid': '1d60fae4-b8c3-4364-9331-b0445493a043', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 10, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 77, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726435218, 'num_in_osds': 2, 'osd_in_since': 1726435223, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 14}, {'state_name': 'remapped+peering', 'count': 10}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56938496, 'bytes_avail': 193216589824, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 1}, 'fsmap': {'epoch': 1, 'btime': '2024-09-15T21:19:07:253728+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-15T21:20:20.678211+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-15T21:20:28.828 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:29.033 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-15T21:20:29.029+0000 7f8e2165a640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 5 pgs recovery_unfound (PG_DAMAGED) 2024-09-15T21:20:29.123 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:29.124 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:20:29.135 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":69,"stamp":"2024-09-15T21:20:28.680078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13543,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":50,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55644,"kb_used_data":620,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688036,"statfs":{"total":193273528320,"available":193216548864,"internally_reserved":0,"allocated":634880,"data_stored":213517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":56286548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001464"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":50,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-15T21:20:24.031808+0000","last_change":"2024-09-15T21:20:24.031714+0000","last_active":"2024-09-15T21:20:24.031808+0000","last_peered":"2024-09-15T21:20:24.031808+0000","last_clean":"2024-09-15T21:20:24.031808+0000","last_became_active":"2024-09-15T21:20:24.031439+0000","last_became_peered":"2024-09-15T21:20:24.031439+0000","last_unstale":"2024-09-15T21:20:24.031808+0000","last_undegraded":"2024-09-15T21:20:24.031808+0000","last_fullsized":"2024-09-15T21:20:24.031808+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:51.235292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":50,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-15T21:20:25.038324+0000","last_change":"2024-09-15T21:20:25.038606+0000","last_active":"2024-09-15T21:20:25.038324+0000","last_peered":"2024-09-15T21:20:25.038324+0000","last_clean":"2024-09-15T21:19:50.263852+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:20:25.038324+0000","last_undegraded":"2024-09-15T21:20:25.031302+0000","last_fullsized":"2024-09-15T21:20:25.038324+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":65,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:20:25.040133+0000","last_change":"2024-09-15T21:20:27.983758+0000","last_active":"2024-09-15T21:20:25.040133+0000","last_peered":"2024-09-15T21:20:25.040133+0000","last_clean":"2024-09-15T21:19:50.814098+0000","last_became_active":"2024-09-15T21:20:25.036309+0000","last_became_peered":"2024-09-15T21:20:25.036309+0000","last_unstale":"2024-09-15T21:20:25.040133+0000","last_undegraded":"2024-09-15T21:20:25.040133+0000","last_fullsized":"2024-09-15T21:20:25.040133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":51,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.036570+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:20:25.036570+0000","last_peered":"2024-09-15T21:20:25.036570+0000","last_clean":"2024-09-15T21:20:25.036570+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:20:25.036570+0000","last_undegraded":"2024-09-15T21:20:25.036570+0000","last_fullsized":"2024-09-15T21:20:25.036570+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":63,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.035961+0000","last_change":"2024-09-15T21:20:24.036146+0000","last_active":"2024-09-15T21:20:24.035961+0000","last_peered":"2024-09-15T21:20:24.035961+0000","last_clean":"2024-09-15T21:19:50.164448+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:20:24.035961+0000","last_undegraded":"2024-09-15T21:20:24.028206+0000","last_fullsized":"2024-09-15T21:20:24.035961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":59,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.035787+0000","last_change":"2024-09-15T21:20:24.035954+0000","last_active":"2024-09-15T21:20:24.035787+0000","last_peered":"2024-09-15T21:20:24.035787+0000","last_clean":"2024-09-15T21:19:51.013140+0000","last_became_active":"2024-09-15T21:20:24.035700+0000","last_became_peered":"2024-09-15T21:20:24.035700+0000","last_unstale":"2024-09-15T21:20:24.035787+0000","last_undegraded":"2024-09-15T21:20:24.025973+0000","last_fullsized":"2024-09-15T21:20:24.035787+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":59,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.036470+0000","last_change":"2024-09-15T21:20:24.036588+0000","last_active":"2024-09-15T21:20:24.036470+0000","last_peered":"2024-09-15T21:20:24.036470+0000","last_clean":"2024-09-15T21:19:50.413489+0000","last_became_active":"2024-09-15T21:20:24.036417+0000","last_became_peered":"2024-09-15T21:20:24.036417+0000","last_unstale":"2024-09-15T21:20:24.036470+0000","last_undegraded":"2024-09-15T21:20:24.028324+0000","last_fullsized":"2024-09-15T21:20:24.036470+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":60,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.032281+0000","last_change":"2024-09-15T21:20:24.032808+0000","last_active":"2024-09-15T21:20:24.032281+0000","last_peered":"2024-09-15T21:20:24.032281+0000","last_clean":"2024-09-15T21:19:36.740313+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:20:24.032281+0000","last_undegraded":"2024-09-15T21:20:24.026851+0000","last_fullsized":"2024-09-15T21:20:24.032281+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-15T21:20:24.032969+0000","last_change":"2024-09-15T21:20:24.032881+0000","last_active":"2024-09-15T21:20:24.032969+0000","last_peered":"2024-09-15T21:20:24.032969+0000","last_clean":"2024-09-15T21:20:24.032969+0000","last_became_active":"2024-09-15T21:20:24.032040+0000","last_became_peered":"2024-09-15T21:20:24.032040+0000","last_unstale":"2024-09-15T21:20:24.032969+0000","last_undegraded":"2024-09-15T21:20:24.032969+0000","last_fullsized":"2024-09-15T21:20:24.032969+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:46:45.703820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":67,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:20:25.040628+0000","last_change":"2024-09-15T21:20:27.983707+0000","last_active":"2024-09-15T21:20:25.040628+0000","last_peered":"2024-09-15T21:20:25.040628+0000","last_clean":"2024-09-15T21:19:50.864243+0000","last_became_active":"2024-09-15T21:20:25.036224+0000","last_became_peered":"2024-09-15T21:20:25.036224+0000","last_unstale":"2024-09-15T21:20:25.040628+0000","last_undegraded":"2024-09-15T21:20:25.040628+0000","last_fullsized":"2024-09-15T21:20:25.040628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037255+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:20:25.037255+0000","last_peered":"2024-09-15T21:20:25.037255+0000","last_clean":"2024-09-15T21:20:25.037255+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:20:25.037255+0000","last_undegraded":"2024-09-15T21:20:25.037255+0000","last_fullsized":"2024-09-15T21:20:25.037255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":55,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.035810+0000","last_change":"2024-09-15T21:20:24.035959+0000","last_active":"2024-09-15T21:20:24.035810+0000","last_peered":"2024-09-15T21:20:24.035810+0000","last_clean":"2024-09-15T21:19:51.063279+0000","last_became_active":"2024-09-15T21:20:24.035738+0000","last_became_peered":"2024-09-15T21:20:24.035738+0000","last_unstale":"2024-09-15T21:20:24.035810+0000","last_undegraded":"2024-09-15T21:20:24.026174+0000","last_fullsized":"2024-09-15T21:20:24.035810+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":62,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.032123+0000","last_change":"2024-09-15T21:20:24.032266+0000","last_active":"2024-09-15T21:20:24.032123+0000","last_peered":"2024-09-15T21:20:24.032123+0000","last_clean":"2024-09-15T21:20:01.890484+0000","last_became_active":"2024-09-15T21:20:24.032032+0000","last_became_peered":"2024-09-15T21:20:24.032032+0000","last_unstale":"2024-09-15T21:20:24.032123+0000","last_undegraded":"2024-09-15T21:20:24.025362+0000","last_fullsized":"2024-09-15T21:20:24.032123+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":54,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-15T21:20:25.039393+0000","last_change":"2024-09-15T21:20:25.039589+0000","last_active":"2024-09-15T21:20:25.039393+0000","last_peered":"2024-09-15T21:20:25.039393+0000","last_clean":"2024-09-15T21:19:50.113984+0000","last_became_active":"2024-09-15T21:20:25.035785+0000","last_became_peered":"2024-09-15T21:20:25.035785+0000","last_unstale":"2024-09-15T21:20:25.039393+0000","last_undegraded":"2024-09-15T21:20:25.031877+0000","last_fullsized":"2024-09-15T21:20:25.039393+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.040332+0000","last_change":"2024-09-15T21:20:25.037062+0000","last_active":"2024-09-15T21:20:25.040332+0000","last_peered":"2024-09-15T21:20:25.040332+0000","last_clean":"2024-09-15T21:20:25.040332+0000","last_became_active":"2024-09-15T21:20:25.036685+0000","last_became_peered":"2024-09-15T21:20:25.036685+0000","last_unstale":"2024-09-15T21:20:25.040332+0000","last_undegraded":"2024-09-15T21:20:25.040332+0000","last_fullsized":"2024-09-15T21:20:25.040332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:47:47.150501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":55,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-15T21:20:25.038873+0000","last_change":"2024-09-15T21:20:25.039099+0000","last_active":"2024-09-15T21:20:25.038873+0000","last_peered":"2024-09-15T21:20:25.038873+0000","last_clean":"2024-09-15T21:19:50.513634+0000","last_became_active":"2024-09-15T21:20:25.035723+0000","last_became_peered":"2024-09-15T21:20:25.035723+0000","last_unstale":"2024-09-15T21:20:25.038873+0000","last_undegraded":"2024-09-15T21:20:25.030642+0000","last_fullsized":"2024-09-15T21:20:25.038873+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":55,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.032282+0000","last_change":"2024-09-15T21:20:24.032431+0000","last_active":"2024-09-15T21:20:24.032282+0000","last_peered":"2024-09-15T21:20:24.032282+0000","last_clean":"2024-09-15T21:19:39.745933+0000","last_became_active":"2024-09-15T21:20:24.032201+0000","last_became_peered":"2024-09-15T21:20:24.032201+0000","last_unstale":"2024-09-15T21:20:24.032282+0000","last_undegraded":"2024-09-15T21:20:24.024923+0000","last_fullsized":"2024-09-15T21:20:24.032282+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-15T21:20:24.032816+0000","last_change":"2024-09-15T21:20:24.032468+0000","last_active":"2024-09-15T21:20:24.032816+0000","last_peered":"2024-09-15T21:20:24.032816+0000","last_clean":"2024-09-15T21:20:24.032816+0000","last_became_active":"2024-09-15T21:20:24.032206+0000","last_became_peered":"2024-09-15T21:20:24.032206+0000","last_unstale":"2024-09-15T21:20:24.032816+0000","last_undegraded":"2024-09-15T21:20:24.032816+0000","last_fullsized":"2024-09-15T21:20:24.032816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:48.044077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":61,"reported_epoch":35,"state":"active+recovering+degraded","last_fresh":"2024-09-15T21:20:24.031849+0000","last_change":"2024-09-15T21:20:24.032926+0000","last_active":"2024-09-15T21:20:24.031849+0000","last_peered":"2024-09-15T21:20:24.031849+0000","last_clean":"2024-09-15T21:19:50.963035+0000","last_became_active":"2024-09-15T21:20:24.031529+0000","last_became_peered":"2024-09-15T21:20:24.031529+0000","last_unstale":"2024-09-15T21:20:24.031849+0000","last_undegraded":"2024-09-15T21:20:24.026166+0000","last_fullsized":"2024-09-15T21:20:24.031849+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.036378+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:20:25.036378+0000","last_peered":"2024-09-15T21:20:25.036378+0000","last_clean":"2024-09-15T21:20:25.036378+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:20:25.036378+0000","last_undegraded":"2024-09-15T21:20:25.036378+0000","last_fullsized":"2024-09-15T21:20:25.036378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":56,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.035999+0000","last_change":"2024-09-15T21:20:24.036145+0000","last_active":"2024-09-15T21:20:24.035999+0000","last_peered":"2024-09-15T21:20:24.035999+0000","last_clean":"2024-09-15T21:19:39.745731+0000","last_became_active":"2024-09-15T21:20:24.035699+0000","last_became_peered":"2024-09-15T21:20:24.035699+0000","last_unstale":"2024-09-15T21:20:24.035999+0000","last_undegraded":"2024-09-15T21:20:24.026554+0000","last_fullsized":"2024-09-15T21:20:24.035999+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-15T21:20:24.031827+0000","last_change":"2024-09-15T21:20:24.031742+0000","last_active":"2024-09-15T21:20:24.031827+0000","last_peered":"2024-09-15T21:20:24.031827+0000","last_clean":"2024-09-15T21:20:24.031827+0000","last_became_active":"2024-09-15T21:20:24.031400+0000","last_became_peered":"2024-09-15T21:20:24.031400+0000","last_unstale":"2024-09-15T21:20:24.031827+0000","last_undegraded":"2024-09-15T21:20:24.031827+0000","last_fullsized":"2024-09-15T21:20:24.031827+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:36:52.618460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037955+0000","last_change":"2024-09-15T21:20:25.035926+0000","last_active":"2024-09-15T21:20:25.037955+0000","last_peered":"2024-09-15T21:20:25.037955+0000","last_clean":"2024-09-15T21:20:25.037955+0000","last_became_active":"2024-09-15T21:20:25.035642+0000","last_became_peered":"2024-09-15T21:20:25.035642+0000","last_unstale":"2024-09-15T21:20:25.037955+0000","last_undegraded":"2024-09-15T21:20:25.037955+0000","last_fullsized":"2024-09-15T21:20:25.037955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:40:43.901095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":64,"reported_epoch":35,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:24.035746+0000","last_change":"2024-09-15T21:20:24.035912+0000","last_active":"2024-09-15T21:20:24.035746+0000","last_peered":"2024-09-15T21:20:24.035746+0000","last_clean":"2024-09-15T21:19:50.713871+0000","last_became_active":"2024-09-15T21:20:24.035656+0000","last_became_peered":"2024-09-15T21:20:24.035656+0000","last_unstale":"2024-09-15T21:20:24.035746+0000","last_undegraded":"2024-09-15T21:20:24.026450+0000","last_fullsized":"2024-09-15T21:20:24.035746+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":49,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986180,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":296,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":303104,"data_stored":107565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":28176041},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182408,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":324,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":331776,"data_stored":105952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28110507},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:29.137 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:29.488 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:29.488 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 50/60 objects degraded (83.333%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":81,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726435218,"num_in_osds":2,"osd_in_since":1726435223,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56979456,"bytes_avail":193216548864,"bytes_total":193273528320,"degraded_objects":50,"degraded_total":60,"degraded_ratio":0.83333333333333348,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-15T21:20:20.678211+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:20:29.500 INFO:tasks.lost_unfound.ceph_manager:{'fsid': '1d60fae4-b8c3-4364-9331-b0445493a043', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 50/60 objects degraded (83.333%), 12 pgs degraded', 'count': 12}, 'muted': False}}, 'mutes': []}, 'election_epoch': 10, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 81, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726435218, 'num_in_osds': 2, 'osd_in_since': 1726435223, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering+degraded', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56979456, 'bytes_avail': 193216548864, 'bytes_total': 193273528320, 'degraded_objects': 50, 'degraded_total': 60, 'degraded_ratio': 0.8333333333333335, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-15T21:19:07:253728+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-15T21:20:20.678211+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-15T21:20:29.501 INFO:tasks.lost_unfound.ceph_manager:active! 2024-09-15T21:20:29.501 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:20:29.501 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:20:29.634 INFO:teuthology.orchestra.run.smithi073.stdout:107374182409 2024-09-15T21:20:29.634 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:20:29.635 INFO:teuthology.orchestra.run.smithi073.stdout:133143986181 2024-09-15T21:20:29.636 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:20:29.989 INFO:teuthology.orchestra.run.smithi073.stdout:107374182408 2024-09-15T21:20:29.990 INFO:teuthology.orchestra.run.smithi073.stdout:133143986180 2024-09-15T21:20:30.003 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986181 got 133143986180 for osd.2 2024-09-15T21:20:30.004 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182409 got 107374182408 for osd.0 2024-09-15T21:20:31.005 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:20:31.005 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:20:31.303 INFO:teuthology.orchestra.run.smithi073.stdout:107374182410 2024-09-15T21:20:31.304 INFO:teuthology.orchestra.run.smithi073.stdout:133143986181 2024-09-15T21:20:31.318 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986181 got 133143986181 for osd.2 2024-09-15T21:20:31.318 DEBUG:teuthology.parallel:result is None 2024-09-15T21:20:31.318 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182409 got 107374182410 for osd.0 2024-09-15T21:20:31.319 DEBUG:teuthology.parallel:result is None 2024-09-15T21:20:31.319 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:20:31.665 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:31.665 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 50/60 objects degraded (83.333%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":83,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726435218,"num_in_osds":2,"osd_in_since":1726435223,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":13543,"bytes_used":56995840,"bytes_avail":193216532480,"bytes_total":193273528320,"degraded_objects":50,"degraded_total":60,"degraded_ratio":0.83333333333333348,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-15T21:20:20.678211+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:20:31.678 INFO:tasks.lost_unfound.ceph_manager:{'fsid': '1d60fae4-b8c3-4364-9331-b0445493a043', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 50/60 objects degraded (83.333%), 11 pgs degraded', 'count': 11}, 'muted': False}}, 'mutes': []}, 'election_epoch': 10, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 83, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726435218, 'num_in_osds': 2, 'osd_in_since': 1726435223, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 13543, 'bytes_used': 56995840, 'bytes_avail': 193216532480, 'bytes_total': 193273528320, 'degraded_objects': 50, 'degraded_total': 60, 'degraded_ratio': 0.8333333333333335, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-15T21:19:07:253728+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-15T21:20:20.678211+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-15T21:20:31.678 INFO:tasks.lost_unfound:there are 27 unfound objects 2024-09-15T21:20:31.679 DEBUG:teuthology.orchestra.run.smithi073:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.admin -b 4096 -p unfound_pool -t 20 bench 240 write' 2024-09-15T21:20:31.730 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:hints = 1 2024-09-15T21:20:31.730 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Maintaining 20 concurrent writes of 4096 bytes to objects of size 4096 for up to 240 seconds or 0 objects 2024-09-15T21:20:31.730 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Object prefix: benchmark_data_smithi073_42248 2024-09-15T21:20:31.730 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:20:31.730 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 0 0 0 0 0 0 - 0 2024-09-15T21:20:32.730 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 1 20 3056 3036 11.8587 11.8594 0.0073625 0.00654575 2024-09-15T21:20:33.730 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 2 20 5855 5835 11.3957 10.9336 0.00669658 0.00683884 2024-09-15T21:20:34.730 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 3 20 8588 8568 11.1555 10.6758 0.00573305 0.00699239 2024-09-15T21:20:35.730 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 4 20 11240 11220 10.9563 10.3594 0.00699325 0.00712051 2024-09-15T21:20:36.730 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 5 20 13873 13853 10.8219 10.2852 0.00852152 0.00720981 2024-09-15T21:20:37.730 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 6 20 16369 16349 10.6431 9.75 0.0136694 0.00732904 2024-09-15T21:20:38.731 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 7 20 18729 18709 10.4391 9.21875 0.00670085 0.00747534 2024-09-15T21:20:39.731 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 8 20 21019 20999 10.2523 8.94531 0.0125851 0.0076142 2024-09-15T21:20:40.731 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 9 20 23229 23209 10.0723 8.63281 0.0104877 0.00775003 2024-09-15T21:20:41.682 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:41.731 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 10 19 25354 25335 9.89545 8.30469 0.0113057 0.00788944 2024-09-15T21:20:42.203 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:42.204 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:20:42.228 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-09-15T21:20:40.683053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80815335,"num_objects":19757,"num_object_clones":0,"num_object_copies":39514,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":50,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":19757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19784,"num_write_kb":78956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19785,"ondisk_log_size":19785,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":266668,"kb_used_data":153352,"kb_used_omap":8,"kb_used_meta":113271,"kb_avail":188477012,"statfs":{"total":193273528320,"available":193000460288,"internally_reserved":0,"allocated":157032448,"data_stored":156601163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":115989844},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[12,4,7,13,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":80801792,"num_objects":19727,"num_object_clones":0,"num_object_copies":39454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19727,"num_write_kb":78908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002975"},"pg_stats":[{"pgid":"2.f","version":"36'1333","reported_seq":1385,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:39.871103+0000","last_change":"2024-09-15T21:20:24.031714+0000","last_active":"2024-09-15T21:20:39.871103+0000","last_peered":"2024-09-15T21:20:39.871103+0000","last_clean":"2024-09-15T21:20:39.871103+0000","last_became_active":"2024-09-15T21:20:24.031439+0000","last_became_peered":"2024-09-15T21:20:24.031439+0000","last_unstale":"2024-09-15T21:20:39.871103+0000","last_undegraded":"2024-09-15T21:20:39.871103+0000","last_fullsized":"2024-09-15T21:20:39.871103+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1333,"log_dups_size":0,"ondisk_log_size":1333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:51.235292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5459968,"num_objects":1333,"num_object_clones":0,"num_object_copies":2666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1333,"num_write_kb":5332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'1052","reported_seq":1097,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:20:37.990192+0000","last_change":"2024-09-15T21:20:31.733562+0000","last_active":"2024-09-15T21:20:37.990192+0000","last_peered":"2024-09-15T21:20:37.990192+0000","last_clean":"2024-09-15T21:19:50.263852+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:20:37.990192+0000","last_undegraded":"2024-09-15T21:20:25.038606+0000","last_fullsized":"2024-09-15T21:20:37.990192+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1052,"log_dups_size":0,"ondisk_log_size":1052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4293574,"num_objects":1050,"num_object_clones":0,"num_object_copies":2100,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1052,"num_write_kb":4196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1050}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'1073","reported_seq":1135,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:20:37.987577+0000","last_change":"2024-09-15T21:20:27.983758+0000","last_active":"2024-09-15T21:20:37.987577+0000","last_peered":"2024-09-15T21:20:37.987577+0000","last_clean":"2024-09-15T21:19:50.814098+0000","last_became_active":"2024-09-15T21:20:25.036309+0000","last_became_peered":"2024-09-15T21:20:25.036309+0000","last_unstale":"2024-09-15T21:20:37.987577+0000","last_undegraded":"2024-09-15T21:20:25.040133+0000","last_fullsized":"2024-09-15T21:20:37.987577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1073,"log_dups_size":0,"ondisk_log_size":1073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4383203,"num_objects":1071,"num_object_clones":0,"num_object_copies":2142,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1073,"num_write_kb":4282,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1071}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'1072","reported_seq":1121,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:37.960123+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:20:37.960123+0000","last_peered":"2024-09-15T21:20:37.960123+0000","last_clean":"2024-09-15T21:20:37.960123+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:20:37.960123+0000","last_undegraded":"2024-09-15T21:20:37.960123+0000","last_fullsized":"2024-09-15T21:20:37.960123+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1072,"log_dups_size":0,"ondisk_log_size":1072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":1072,"num_object_clones":0,"num_object_copies":2144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1072,"num_write_kb":4288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'1313","reported_seq":1373,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:39.868506+0000","last_change":"2024-09-15T21:20:24.036146+0000","last_active":"2024-09-15T21:20:39.868506+0000","last_peered":"2024-09-15T21:20:39.868506+0000","last_clean":"2024-09-15T21:19:50.164448+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:20:39.868506+0000","last_undegraded":"2024-09-15T21:20:24.028206+0000","last_fullsized":"2024-09-15T21:20:39.868506+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1313,"log_dups_size":0,"ondisk_log_size":1313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5358534,"num_objects":1310,"num_object_clones":0,"num_object_copies":2620,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1313,"num_write_kb":5236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'1362","reported_seq":1420,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:39.846760+0000","last_change":"2024-09-15T21:20:24.035954+0000","last_active":"2024-09-15T21:20:39.846760+0000","last_peered":"2024-09-15T21:20:39.846760+0000","last_clean":"2024-09-15T21:19:51.013140+0000","last_became_active":"2024-09-15T21:20:24.035700+0000","last_became_peered":"2024-09-15T21:20:24.035700+0000","last_unstale":"2024-09-15T21:20:39.846760+0000","last_undegraded":"2024-09-15T21:20:24.025973+0000","last_fullsized":"2024-09-15T21:20:39.846760+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5567430,"num_objects":1361,"num_object_clones":0,"num_object_copies":2722,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1362,"num_write_kb":5439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1361}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'1353","reported_seq":1410,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:39.871484+0000","last_change":"2024-09-15T21:20:24.036588+0000","last_active":"2024-09-15T21:20:39.871484+0000","last_peered":"2024-09-15T21:20:39.871484+0000","last_clean":"2024-09-15T21:19:50.413489+0000","last_became_active":"2024-09-15T21:20:24.036417+0000","last_became_peered":"2024-09-15T21:20:24.036417+0000","last_unstale":"2024-09-15T21:20:39.871484+0000","last_undegraded":"2024-09-15T21:20:24.028324+0000","last_fullsized":"2024-09-15T21:20:39.871484+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1353,"log_dups_size":0,"ondisk_log_size":1353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5526470,"num_objects":1351,"num_object_clones":0,"num_object_copies":2702,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1353,"num_write_kb":5400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1351}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'1336","reported_seq":1395,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:39.875997+0000","last_change":"2024-09-15T21:20:24.032808+0000","last_active":"2024-09-15T21:20:39.875997+0000","last_peered":"2024-09-15T21:20:39.875997+0000","last_clean":"2024-09-15T21:19:36.740313+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:20:39.875997+0000","last_undegraded":"2024-09-15T21:20:24.026851+0000","last_fullsized":"2024-09-15T21:20:39.875997+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1336,"log_dups_size":0,"ondisk_log_size":1336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5468643,"num_objects":1336,"num_object_clones":0,"num_object_copies":2672,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1336,"num_write_kb":5341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1336}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033321+0000","last_change":"2024-09-15T21:20:24.032881+0000","last_active":"2024-09-15T21:20:25.033321+0000","last_peered":"2024-09-15T21:20:25.033321+0000","last_clean":"2024-09-15T21:20:25.033321+0000","last_became_active":"2024-09-15T21:20:24.032040+0000","last_became_peered":"2024-09-15T21:20:24.032040+0000","last_unstale":"2024-09-15T21:20:25.033321+0000","last_undegraded":"2024-09-15T21:20:25.033321+0000","last_fullsized":"2024-09-15T21:20:25.033321+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:46:45.703820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'1103","reported_seq":1165,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:20:37.987584+0000","last_change":"2024-09-15T21:20:27.983707+0000","last_active":"2024-09-15T21:20:37.987584+0000","last_peered":"2024-09-15T21:20:37.987584+0000","last_clean":"2024-09-15T21:19:50.864243+0000","last_became_active":"2024-09-15T21:20:25.036224+0000","last_became_peered":"2024-09-15T21:20:25.036224+0000","last_unstale":"2024-09-15T21:20:37.987584+0000","last_undegraded":"2024-09-15T21:20:25.040628+0000","last_fullsized":"2024-09-15T21:20:37.987584+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1103,"log_dups_size":0,"ondisk_log_size":1103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4498374,"num_objects":1100,"num_object_clones":0,"num_object_copies":2200,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1103,"num_write_kb":4396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1100}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037255+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:20:25.037255+0000","last_peered":"2024-09-15T21:20:25.037255+0000","last_clean":"2024-09-15T21:20:25.037255+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:20:25.037255+0000","last_undegraded":"2024-09-15T21:20:25.037255+0000","last_fullsized":"2024-09-15T21:20:25.037255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'1276","reported_seq":1327,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:39.871233+0000","last_change":"2024-09-15T21:20:24.035959+0000","last_active":"2024-09-15T21:20:39.871233+0000","last_peered":"2024-09-15T21:20:39.871233+0000","last_clean":"2024-09-15T21:19:51.063279+0000","last_became_active":"2024-09-15T21:20:24.035738+0000","last_became_peered":"2024-09-15T21:20:24.035738+0000","last_unstale":"2024-09-15T21:20:39.871233+0000","last_undegraded":"2024-09-15T21:20:24.026174+0000","last_fullsized":"2024-09-15T21:20:39.871233+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1276,"log_dups_size":0,"ondisk_log_size":1276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5202886,"num_objects":1272,"num_object_clones":0,"num_object_copies":2544,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1276,"num_write_kb":5084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1272}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:25.034061+0000","last_change":"2024-09-15T21:20:24.032266+0000","last_active":"2024-09-15T21:20:25.034061+0000","last_peered":"2024-09-15T21:20:25.034061+0000","last_clean":"2024-09-15T21:20:01.890484+0000","last_became_active":"2024-09-15T21:20:24.032032+0000","last_became_peered":"2024-09-15T21:20:24.032032+0000","last_unstale":"2024-09-15T21:20:25.034061+0000","last_undegraded":"2024-09-15T21:20:24.025362+0000","last_fullsized":"2024-09-15T21:20:25.034061+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'1048","reported_seq":1098,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:20:37.983484+0000","last_change":"2024-09-15T21:20:31.741741+0000","last_active":"2024-09-15T21:20:37.983484+0000","last_peered":"2024-09-15T21:20:37.983484+0000","last_clean":"2024-09-15T21:19:50.113984+0000","last_became_active":"2024-09-15T21:20:25.035785+0000","last_became_peered":"2024-09-15T21:20:25.035785+0000","last_unstale":"2024-09-15T21:20:37.983484+0000","last_undegraded":"2024-09-15T21:20:25.039589+0000","last_fullsized":"2024-09-15T21:20:37.983484+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1048,"log_dups_size":0,"ondisk_log_size":1048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4281286,"num_objects":1047,"num_object_clones":0,"num_object_copies":2094,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1048,"num_write_kb":4183,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1047}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.040332+0000","last_change":"2024-09-15T21:20:25.037062+0000","last_active":"2024-09-15T21:20:25.040332+0000","last_peered":"2024-09-15T21:20:25.040332+0000","last_clean":"2024-09-15T21:20:25.040332+0000","last_became_active":"2024-09-15T21:20:25.036685+0000","last_became_peered":"2024-09-15T21:20:25.036685+0000","last_unstale":"2024-09-15T21:20:25.040332+0000","last_undegraded":"2024-09-15T21:20:25.040332+0000","last_fullsized":"2024-09-15T21:20:25.040332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:47:47.150501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'1097","reported_seq":1148,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-15T21:20:37.979792+0000","last_change":"2024-09-15T21:20:31.733794+0000","last_active":"2024-09-15T21:20:37.979792+0000","last_peered":"2024-09-15T21:20:37.979792+0000","last_clean":"2024-09-15T21:19:50.513634+0000","last_became_active":"2024-09-15T21:20:25.035723+0000","last_became_peered":"2024-09-15T21:20:25.035723+0000","last_unstale":"2024-09-15T21:20:37.979792+0000","last_undegraded":"2024-09-15T21:20:25.039099+0000","last_fullsized":"2024-09-15T21:20:37.979792+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1097,"log_dups_size":0,"ondisk_log_size":1097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4477894,"num_objects":1095,"num_object_clones":0,"num_object_copies":2190,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1097,"num_write_kb":4375,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'1305","reported_seq":1361,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:39.877254+0000","last_change":"2024-09-15T21:20:24.032431+0000","last_active":"2024-09-15T21:20:39.877254+0000","last_peered":"2024-09-15T21:20:39.877254+0000","last_clean":"2024-09-15T21:19:39.745933+0000","last_became_active":"2024-09-15T21:20:24.032201+0000","last_became_peered":"2024-09-15T21:20:24.032201+0000","last_unstale":"2024-09-15T21:20:39.877254+0000","last_undegraded":"2024-09-15T21:20:24.024923+0000","last_fullsized":"2024-09-15T21:20:39.877254+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1305,"log_dups_size":0,"ondisk_log_size":1305,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5341667,"num_objects":1305,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1305,"num_write_kb":5217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1305}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.034081+0000","last_change":"2024-09-15T21:20:24.032468+0000","last_active":"2024-09-15T21:20:25.034081+0000","last_peered":"2024-09-15T21:20:25.034081+0000","last_clean":"2024-09-15T21:20:25.034081+0000","last_became_active":"2024-09-15T21:20:24.032206+0000","last_became_peered":"2024-09-15T21:20:24.032206+0000","last_unstale":"2024-09-15T21:20:25.034081+0000","last_undegraded":"2024-09-15T21:20:25.034081+0000","last_fullsized":"2024-09-15T21:20:25.034081+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:48.044077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'1351","reported_seq":1409,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-15T21:20:39.872243+0000","last_change":"2024-09-15T21:20:25.033511+0000","last_active":"2024-09-15T21:20:39.872243+0000","last_peered":"2024-09-15T21:20:39.872243+0000","last_clean":"2024-09-15T21:19:50.963035+0000","last_became_active":"2024-09-15T21:20:24.031529+0000","last_became_peered":"2024-09-15T21:20:24.031529+0000","last_unstale":"2024-09-15T21:20:39.872243+0000","last_undegraded":"2024-09-15T21:20:39.872243+0000","last_fullsized":"2024-09-15T21:20:39.872243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1351,"log_dups_size":0,"ondisk_log_size":1351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5514665,"num_objects":1349,"num_object_clones":0,"num_object_copies":2698,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1351,"num_write_kb":5388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1349}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.036378+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:20:25.036378+0000","last_peered":"2024-09-15T21:20:25.036378+0000","last_clean":"2024-09-15T21:20:25.036378+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:20:25.036378+0000","last_undegraded":"2024-09-15T21:20:25.036378+0000","last_fullsized":"2024-09-15T21:20:25.036378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'1350","reported_seq":1406,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:39.871250+0000","last_change":"2024-09-15T21:20:24.036145+0000","last_active":"2024-09-15T21:20:39.871250+0000","last_peered":"2024-09-15T21:20:39.871250+0000","last_clean":"2024-09-15T21:19:39.745731+0000","last_became_active":"2024-09-15T21:20:24.035699+0000","last_became_peered":"2024-09-15T21:20:24.035699+0000","last_unstale":"2024-09-15T21:20:39.871250+0000","last_undegraded":"2024-09-15T21:20:24.026554+0000","last_fullsized":"2024-09-15T21:20:39.871250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1350,"log_dups_size":0,"ondisk_log_size":1350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5522374,"num_objects":1350,"num_object_clones":0,"num_object_copies":2700,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1350,"num_write_kb":5394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1350}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033483+0000","last_change":"2024-09-15T21:20:24.031742+0000","last_active":"2024-09-15T21:20:25.033483+0000","last_peered":"2024-09-15T21:20:25.033483+0000","last_clean":"2024-09-15T21:20:25.033483+0000","last_became_active":"2024-09-15T21:20:24.031400+0000","last_became_peered":"2024-09-15T21:20:24.031400+0000","last_unstale":"2024-09-15T21:20:25.033483+0000","last_undegraded":"2024-09-15T21:20:25.033483+0000","last_fullsized":"2024-09-15T21:20:25.033483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:36:52.618460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037955+0000","last_change":"2024-09-15T21:20:25.035926+0000","last_active":"2024-09-15T21:20:25.037955+0000","last_peered":"2024-09-15T21:20:25.037955+0000","last_clean":"2024-09-15T21:20:25.037955+0000","last_became_active":"2024-09-15T21:20:25.035642+0000","last_became_peered":"2024-09-15T21:20:25.035642+0000","last_unstale":"2024-09-15T21:20:25.037955+0000","last_undegraded":"2024-09-15T21:20:25.037955+0000","last_fullsized":"2024-09-15T21:20:25.037955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:40:43.901095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'1358","reported_seq":1414,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:39.878829+0000","last_change":"2024-09-15T21:20:24.035912+0000","last_active":"2024-09-15T21:20:39.878829+0000","last_peered":"2024-09-15T21:20:39.878829+0000","last_clean":"2024-09-15T21:19:50.713871+0000","last_became_active":"2024-09-15T21:20:24.035656+0000","last_became_peered":"2024-09-15T21:20:24.035656+0000","last_unstale":"2024-09-15T21:20:39.878829+0000","last_undegraded":"2024-09-15T21:20:24.026450+0000","last_fullsized":"2024-09-15T21:20:39.878829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":1358,"log_dups_size":0,"ondisk_log_size":1358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5527436,"num_objects":1353,"num_object_clones":0,"num_object_copies":2706,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1358,"num_write_kb":5403,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1352}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":80815316,"num_objects":19755,"num_object_clones":0,"num_object_copies":39510,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":49,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":19755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19782,"num_write_kb":78954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":157216768,"data_stored":157079474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19782,"ondisk_log_size":19782,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986183,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":117436,"kb_used_data":67688,"kb_used_omap":4,"kb_used_meta":49723,"kb_avail":94254404,"statfs":{"total":96636764160,"available":96516509696,"internally_reserved":0,"allocated":69312512,"data_stored":69116973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":50917033},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,6,1,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182412,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":149232,"kb_used_data":85664,"kb_used_omap":4,"kb_used_meta":63547,"kb_avail":94222608,"statfs":{"total":96636764160,"available":96483950592,"internally_reserved":0,"allocated":87719936,"data_stored":87484190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":65072811},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,1,12],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":87601152,"data_stored":87565022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69500928,"data_stored":69500928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:42.230 INFO:tasks.lost_unfound:no unfound in 2.f 2024-09-15T21:20:42.230 INFO:tasks.lost_unfound:listing missing/lost in 2.e state active+recovery_unfound+degraded 2024-09-15T21:20:42.230 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.e list_unfound '{}' 2024-09-15T21:20:42.433 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:42.433 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:42.433 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:42.433 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:42.433 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:42.433 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:42.433 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:42.433 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 2, 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 2, 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existing_4", 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 1224592654, 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:42.434 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'3", 2024-09-15T21:20:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "21'1", 2024-09-15T21:20:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-15T21:20:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existing_6", 2024-09-15T21:20:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:42.435 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 3101443614, 2024-09-15T21:20:42.436 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:42.436 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:42.436 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:42.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:42.436 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'4", 2024-09-15T21:20:42.436 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "21'2", 2024-09-15T21:20:42.436 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:42.436 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-15T21:20:42.436 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:42.436 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:42.436 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:42.436 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "NotRecovering", 2024-09-15T21:20:42.436 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:42.437 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:42.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:42.437 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:42.437 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:42.437 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:42.437 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:42.437 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:42.437 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:42.454 INFO:tasks.lost_unfound:reverting unfound in 2.e on osd.2 2024-09-15T21:20:42.454 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.e mark_unfound_lost revert 2024-09-15T21:20:42.678 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:20:42.693 INFO:tasks.lost_unfound:listing missing/lost in 2.d state active+recovery_unfound+degraded 2024-09-15T21:20:42.694 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.d list_unfound '{}' 2024-09-15T21:20:42.731 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 11 20 27142 27122 9.6304 6.98047 0.0180845 0.00810468 2024-09-15T21:20:42.892 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:42.892 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 1, 2024-09-15T21:20:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 1, 2024-09-15T21:20:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existed_7", 2024-09-15T21:20:42.893 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 2358579165, 2024-09-15T21:20:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'3", 2024-09-15T21:20:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:42.894 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "NotRecovering", 2024-09-15T21:20:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:42.895 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:42.896 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:42.896 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:42.913 INFO:tasks.lost_unfound:reverting unfound in 2.d on osd.2 2024-09-15T21:20:42.913 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.d mark_unfound_lost revert 2024-09-15T21:20:43.187 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-15T21:20:43.207 INFO:tasks.lost_unfound:no unfound in 2.c 2024-09-15T21:20:43.207 INFO:tasks.lost_unfound:listing missing/lost in 2.b state active+recovery_wait+degraded 2024-09-15T21:20:43.208 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.b list_unfound '{}' 2024-09-15T21:20:43.413 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:43.414 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:43.414 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:43.414 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:43.414 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:43.414 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:43.414 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:43.414 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:43.414 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:43.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:43.414 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 2, 2024-09-15T21:20:43.414 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 2, 2024-09-15T21:20:43.414 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:43.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existed_3", 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 943960171, 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'5", 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:43.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existing_1", 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 2885884443, 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'4", 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "21'1", 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:43.416 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-15T21:20:43.417 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:43.417 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:43.417 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:43.417 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "Active", 2024-09-15T21:20:43.417 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:43.417 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:43.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:43.417 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:43.417 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:43.417 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:43.417 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:43.417 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:43.417 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:43.437 INFO:tasks.lost_unfound:reverting unfound in 2.b on osd.0 2024-09-15T21:20:43.437 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.b mark_unfound_lost revert 2024-09-15T21:20:43.663 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:20:43.680 INFO:tasks.lost_unfound:listing missing/lost in 2.a state active+recovery_wait+degraded 2024-09-15T21:20:43.681 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.a list_unfound '{}' 2024-09-15T21:20:43.731 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 12 20 28830 28810 9.37731 6.59375 0.0100626 0.00832223 2024-09-15T21:20:43.881 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:43.882 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 2, 2024-09-15T21:20:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 2, 2024-09-15T21:20:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existing_9", 2024-09-15T21:20:43.883 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 1217469130, 2024-09-15T21:20:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'3", 2024-09-15T21:20:43.884 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "21'1", 2024-09-15T21:20:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-15T21:20:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "new_9", 2024-09-15T21:20:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:43.885 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 170713514, 2024-09-15T21:20:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:43.886 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'2", 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "Active", 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:43.887 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:43.888 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:43.888 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:43.900 INFO:tasks.lost_unfound:reverting unfound in 2.a on osd.0 2024-09-15T21:20:43.900 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.a mark_unfound_lost revert 2024-09-15T21:20:44.120 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:20:44.148 INFO:tasks.lost_unfound:listing missing/lost in 2.9 state active+recovery_wait+degraded 2024-09-15T21:20:44.148 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.9 list_unfound '{}' 2024-09-15T21:20:44.358 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 2, 2024-09-15T21:20:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 2, 2024-09-15T21:20:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:44.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existing_5", 2024-09-15T21:20:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 2998549577, 2024-09-15T21:20:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'4", 2024-09-15T21:20:44.360 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "21'2", 2024-09-15T21:20:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-15T21:20:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existing_2", 2024-09-15T21:20:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 4032093161, 2024-09-15T21:20:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:44.361 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'3", 2024-09-15T21:20:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "21'1", 2024-09-15T21:20:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-15T21:20:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "Active", 2024-09-15T21:20:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:44.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:44.363 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:44.363 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:44.363 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:44.363 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:44.363 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:44.363 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:44.379 INFO:tasks.lost_unfound:reverting unfound in 2.9 on osd.0 2024-09-15T21:20:44.379 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.9 mark_unfound_lost revert 2024-09-15T21:20:44.608 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:20:44.631 INFO:tasks.lost_unfound:listing missing/lost in 2.8 state active+recovery_wait+degraded 2024-09-15T21:20:44.631 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.8 list_unfound '{}' 2024-09-15T21:20:44.732 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 13 20 30446 30426 9.14088 6.3125 0.00814611 0.00853962 2024-09-15T21:20:44.839 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:44.839 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:44.839 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:44.839 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:44.839 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:44.839 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:44.839 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:44.839 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:44.839 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:44.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:44.840 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 1, 2024-09-15T21:20:44.840 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 1, 2024-09-15T21:20:44.840 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:44.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:44.840 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:44.840 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "new_8", 2024-09-15T21:20:44.840 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:44.840 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:44.840 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 2730327960, 2024-09-15T21:20:44.840 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:44.840 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:44.840 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:44.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:44.841 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'1", 2024-09-15T21:20:44.841 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:44.841 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:44.841 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:44.841 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:44.841 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:44.841 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:44.842 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "Active", 2024-09-15T21:20:44.842 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:44.842 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:44.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:44.842 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:44.842 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:44.843 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:44.843 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:44.843 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:44.843 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:44.863 INFO:tasks.lost_unfound:reverting unfound in 2.8 on osd.0 2024-09-15T21:20:44.863 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.8 mark_unfound_lost revert 2024-09-15T21:20:45.083 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-15T21:20:45.103 INFO:tasks.lost_unfound:no unfound in 1.0 2024-09-15T21:20:45.103 INFO:tasks.lost_unfound:listing missing/lost in 2.3 state active+recovery_unfound+degraded 2024-09-15T21:20:45.104 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.3 list_unfound '{}' 2024-09-15T21:20:45.134 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-15T21:20:45.131+0000 7f8e2165a640 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 4 pgs recovery_unfound (PG_DAMAGED) 2024-09-15T21:20:45.301 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:45.301 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:45.301 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:45.301 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:45.301 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:45.301 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:45.301 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:45.301 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:45.301 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:45.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:45.301 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 2, 2024-09-15T21:20:45.302 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 2, 2024-09-15T21:20:45.302 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:45.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:45.302 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:45.302 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existing_8", 2024-09-15T21:20:45.302 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:45.302 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:45.302 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 1901787523, 2024-09-15T21:20:45.303 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:45.303 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:45.303 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:45.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:45.303 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'5", 2024-09-15T21:20:45.303 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "21'3", 2024-09-15T21:20:45.303 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:45.303 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-15T21:20:45.303 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:45.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:45.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:45.304 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:45.304 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existed_1", 2024-09-15T21:20:45.304 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:45.304 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:45.304 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 3122379363, 2024-09-15T21:20:45.304 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:45.304 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:45.304 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:45.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:45.304 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'4", 2024-09-15T21:20:45.304 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:45.305 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:45.305 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:45.305 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:45.305 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:45.305 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:45.305 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "NotRecovering", 2024-09-15T21:20:45.305 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:45.305 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:45.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:45.305 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:45.305 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:45.305 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:45.305 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:45.306 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:45.306 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:45.321 INFO:tasks.lost_unfound:reverting unfound in 2.3 on osd.2 2024-09-15T21:20:45.322 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.3 mark_unfound_lost revert 2024-09-15T21:20:45.576 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:20:45.594 INFO:tasks.lost_unfound:no unfound in 1.1 2024-09-15T21:20:45.594 INFO:tasks.lost_unfound:listing missing/lost in 2.2 state active+recovery_wait+degraded 2024-09-15T21:20:45.595 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.2 list_unfound '{}' 2024-09-15T21:20:45.732 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 14 20 32146 32126 8.96227 6.64062 0.0213229 0.00871004 2024-09-15T21:20:45.800 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:45.801 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:45.801 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:45.801 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:45.801 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:45.801 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:45.801 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:45.801 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:45.801 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:45.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:45.801 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 2, 2024-09-15T21:20:45.801 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 2, 2024-09-15T21:20:45.801 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existed_6", 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 1708673474, 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'5", 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:45.802 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:45.803 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:45.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:45.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:45.803 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:45.803 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existed_9", 2024-09-15T21:20:45.803 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:45.803 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:45.803 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 831402034, 2024-09-15T21:20:45.803 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:45.803 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:45.803 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:45.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:45.804 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'6", 2024-09-15T21:20:45.804 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:45.804 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:45.804 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:45.804 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:45.804 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:45.804 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:45.804 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "Active", 2024-09-15T21:20:45.804 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:45.804 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:45.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:45.804 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:45.805 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:45.805 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:45.805 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:45.805 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:45.805 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:45.822 INFO:tasks.lost_unfound:reverting unfound in 2.2 on osd.0 2024-09-15T21:20:45.823 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.2 mark_unfound_lost revert 2024-09-15T21:20:46.048 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:20:46.068 INFO:tasks.lost_unfound:no unfound in 1.2 2024-09-15T21:20:46.069 INFO:tasks.lost_unfound:listing missing/lost in 2.1 state active+recovery_unfound+degraded 2024-09-15T21:20:46.069 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.1 list_unfound '{}' 2024-09-15T21:20:46.277 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:46.277 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:46.277 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:46.277 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:46.277 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:46.278 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:46.278 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:46.278 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:46.278 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:46.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:46.278 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 2, 2024-09-15T21:20:46.278 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 2, 2024-09-15T21:20:46.278 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:46.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:46.278 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:46.278 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existing_3", 2024-09-15T21:20:46.279 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:46.279 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:46.279 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 459768081, 2024-09-15T21:20:46.279 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:46.279 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:46.279 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:46.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:46.279 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'3", 2024-09-15T21:20:46.279 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "21'1", 2024-09-15T21:20:46.279 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:46.279 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-15T21:20:46.279 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:46.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:46.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:46.280 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:46.280 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "new_3", 2024-09-15T21:20:46.280 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:46.280 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:46.280 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 480221105, 2024-09-15T21:20:46.280 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:46.280 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:46.280 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:46.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:46.280 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'2", 2024-09-15T21:20:46.281 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:46.281 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:46.281 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:46.281 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:46.281 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:46.281 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:46.281 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "NotRecovering", 2024-09-15T21:20:46.281 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:46.281 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:46.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:46.281 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:46.281 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:46.281 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:46.282 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:46.282 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:46.282 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:46.298 INFO:tasks.lost_unfound:reverting unfound in 2.1 on osd.2 2024-09-15T21:20:46.298 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.1 mark_unfound_lost revert 2024-09-15T21:20:46.511 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:20:46.532 INFO:tasks.lost_unfound:no unfound in 1.3 2024-09-15T21:20:46.532 INFO:tasks.lost_unfound:listing missing/lost in 2.0 state active+recovery_unfound+degraded 2024-09-15T21:20:46.532 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.0 list_unfound '{}' 2024-09-15T21:20:46.732 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 15 20 33691 33671 8.76711 6.03516 0.00822571 0.0089049 2024-09-15T21:20:46.747 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:46.747 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 2, 2024-09-15T21:20:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 2, 2024-09-15T21:20:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "new_5", 2024-09-15T21:20:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:46.748 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 4248945696, 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'3", 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existed_5", 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:46.749 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 4212789968, 2024-09-15T21:20:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'4", 2024-09-15T21:20:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:46.750 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "NotRecovering", 2024-09-15T21:20:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:46.751 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:46.751 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:46.770 INFO:tasks.lost_unfound:reverting unfound in 2.0 on osd.2 2024-09-15T21:20:46.770 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.0 mark_unfound_lost revert 2024-09-15T21:20:46.985 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:20:47.005 INFO:tasks.lost_unfound:listing missing/lost in 2.7 state active+recovery_wait+degraded 2024-09-15T21:20:47.006 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.7 list_unfound '{}' 2024-09-15T21:20:47.224 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:47.224 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:47.224 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:47.224 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:47.225 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:47.225 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:47.225 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:47.225 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:47.225 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:47.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:47.225 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 1, 2024-09-15T21:20:47.225 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 1, 2024-09-15T21:20:47.225 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:47.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:47.225 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:47.225 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "new_7", 2024-09-15T21:20:47.225 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 3761017527, 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'1", 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:47.226 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "Active", 2024-09-15T21:20:47.227 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:47.227 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:47.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:47.227 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:47.227 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:47.227 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:47.227 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:47.227 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:47.227 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:47.243 INFO:tasks.lost_unfound:reverting unfound in 2.7 on osd.0 2024-09-15T21:20:47.243 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.7 mark_unfound_lost revert 2024-09-15T21:20:47.474 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-15T21:20:47.496 INFO:tasks.lost_unfound:no unfound in 1.4 2024-09-15T21:20:47.496 INFO:tasks.lost_unfound:listing missing/lost in 2.6 state active+recovering 2024-09-15T21:20:47.497 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.6 list_unfound '{}' 2024-09-15T21:20:47.707 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:47.707 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 3, 2024-09-15T21:20:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 3, 2024-09-15T21:20:47.708 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existed_8", 2024-09-15T21:20:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 1861424966, 2024-09-15T21:20:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'5", 2024-09-15T21:20:47.709 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "new_6", 2024-09-15T21:20:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 3910131350, 2024-09-15T21:20:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:47.710 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'4", 2024-09-15T21:20:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "new_4", 2024-09-15T21:20:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:47.711 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 2787839990, 2024-09-15T21:20:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'3", 2024-09-15T21:20:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:47.712 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "Active", 2024-09-15T21:20:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:47.713 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:47.713 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:47.727 INFO:tasks.lost_unfound:reverting unfound in 2.6 on osd.0 2024-09-15T21:20:47.728 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.6 mark_unfound_lost revert 2024-09-15T21:20:47.732 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 16 20 35156 35136 8.57674 5.72266 0.0160622 0.00910193 2024-09-15T21:20:47.961 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-15T21:20:47.985 INFO:tasks.lost_unfound:no unfound in 1.5 2024-09-15T21:20:47.985 INFO:tasks.lost_unfound:listing missing/lost in 2.5 state active+recovery_wait+degraded 2024-09-15T21:20:47.985 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.5 list_unfound '{}' 2024-09-15T21:20:48.235 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:48.235 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:48.235 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:48.235 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:48.235 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:48.235 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:48.236 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:48.236 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:48.236 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:48.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:48.236 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 2, 2024-09-15T21:20:48.236 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 2, 2024-09-15T21:20:48.236 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:48.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:48.236 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:48.236 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "new_1", 2024-09-15T21:20:48.236 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:48.236 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:48.236 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 26281365, 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'1", 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "new_2", 2024-09-15T21:20:48.237 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:48.238 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:48.238 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 3196340981, 2024-09-15T21:20:48.238 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:48.238 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:48.238 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:48.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:48.238 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'2", 2024-09-15T21:20:48.238 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:48.238 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:48.238 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:48.238 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:48.238 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:48.238 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:48.239 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "Active", 2024-09-15T21:20:48.239 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:48.239 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:48.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:48.239 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:48.239 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:48.239 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:48.239 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:48.239 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:48.239 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:48.255 INFO:tasks.lost_unfound:reverting unfound in 2.5 on osd.0 2024-09-15T21:20:48.255 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.5 mark_unfound_lost revert 2024-09-15T21:20:48.490 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-15T21:20:48.509 INFO:tasks.lost_unfound:no unfound in 1.6 2024-09-15T21:20:48.510 INFO:tasks.lost_unfound:no unfound in 1.7 2024-09-15T21:20:48.510 INFO:tasks.lost_unfound:listing missing/lost in 2.4 state active+recovery_wait+degraded 2024-09-15T21:20:48.510 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.4 list_unfound '{}' 2024-09-15T21:20:48.716 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:20:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: "offset": { 2024-09-15T21:20:48.716 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "", 2024-09-15T21:20:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": 0, 2024-09-15T21:20:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 0, 2024-09-15T21:20:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": -9223372036854775808, 2024-09-15T21:20:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "num_missing": 4, 2024-09-15T21:20:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "num_unfound": 3, 2024-09-15T21:20:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "objects": [ 2024-09-15T21:20:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:48.717 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existed_4", 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 873891716, 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'8", 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existing_7", 2024-09-15T21:20:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 3996155460, 2024-09-15T21:20:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'9", 2024-09-15T21:20:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "21'6", 2024-09-15T21:20:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:48.719 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-09-15T21:20:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": { 2024-09-15T21:20:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "oid": "existed_2", 2024-09-15T21:20:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-09-15T21:20:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "snapid": -2, 2024-09-15T21:20:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "hash": 1767667508, 2024-09-15T21:20:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 0, 2024-09-15T21:20:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "pool": 2, 2024-09-15T21:20:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "namespace": "" 2024-09-15T21:20:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:20:48.720 INFO:teuthology.orchestra.run.smithi073.stdout: "need": "23'7", 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "have": "0'0", 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": "none", 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "locations": [] 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "Active", 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "available_might_have_unfound": true, 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "might_have_unfound": [ 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "1", 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "osd is down" 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:20:48.721 INFO:teuthology.orchestra.run.smithi073.stdout: "more": false 2024-09-15T21:20:48.722 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:20:48.734 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 17 20 36660 36640 8.4178 5.875 0.0140407 0.00927656 2024-09-15T21:20:48.737 INFO:tasks.lost_unfound:reverting unfound in 2.4 on osd.0 2024-09-15T21:20:48.737 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.4 mark_unfound_lost revert 2024-09-15T21:20:48.967 INFO:teuthology.orchestra.run.smithi073.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-15T21:20:48.989 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 5 2024-09-15T21:20:49.205 INFO:teuthology.orchestra.run.smithi073.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-15T21:20:49.227 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 5 2024-09-15T21:20:49.437 INFO:teuthology.orchestra.run.smithi073.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-15T21:20:49.462 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-15T21:20:49.462 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-15T21:20:49.704 INFO:teuthology.orchestra.run.smithi073.stdout:107374182414 2024-09-15T21:20:49.705 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:20:49.705 INFO:teuthology.orchestra.run.smithi073.stdout:133143986186 2024-09-15T21:20:49.706 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:20:49.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 18 20 38172 38152 8.27826 5.90625 0.00964811 0.00943081 2024-09-15T21:20:50.350 INFO:teuthology.orchestra.run.smithi073.stdout:133143986185 2024-09-15T21:20:50.351 INFO:teuthology.orchestra.run.smithi073.stdout:107374182413 2024-09-15T21:20:50.371 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986186 got 133143986185 for osd.2 2024-09-15T21:20:50.373 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182414 got 107374182413 for osd.0 2024-09-15T21:20:50.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 19 20 39653 39633 8.14703 5.78516 0.00635892 0.00958316 2024-09-15T21:20:51.372 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-15T21:20:51.374 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-15T21:20:51.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:2024-09-15T21:20:51.730824+0000 min lat: 0.00235499 max lat: 0.0589451 avg lat: 0.00970795 2024-09-15T21:20:51.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:20:51.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 20 20 41209 41189 8.04356 6.07812 0.0142067 0.00970795 2024-09-15T21:20:51.918 INFO:teuthology.orchestra.run.smithi073.stdout:107374182415 2024-09-15T21:20:51.920 INFO:teuthology.orchestra.run.smithi073.stdout:133143986186 2024-09-15T21:20:51.941 INFO:tasks.lost_unfound.ceph_manager:need seq 107374182414 got 107374182415 for osd.0 2024-09-15T21:20:51.941 DEBUG:teuthology.parallel:result is None 2024-09-15T21:20:51.944 INFO:tasks.lost_unfound.ceph_manager:need seq 133143986186 got 133143986186 for osd.2 2024-09-15T21:20:51.944 DEBUG:teuthology.parallel:result is None 2024-09-15T21:20:51.944 INFO:tasks.lost_unfound.ceph_manager:waiting for recovery to complete 2024-09-15T21:20:51.944 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:52.492 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:52.492 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:20:52.512 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-09-15T21:20:50.685493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":156834680,"num_objects":38312,"num_object_clones":0,"num_object_copies":76624,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":38312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28218,"num_write_kb":112752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38372,"ondisk_log_size":38372,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":513460,"kb_used_data":301588,"kb_used_omap":8,"kb_used_meta":211831,"kb_avail":188230220,"statfs":{"total":193273528320,"available":192747745280,"internally_reserved":0,"allocated":308826112,"data_stored":308380375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":216915284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[11,1,1,16,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":107697809,"num_objects":26289,"num_object_clones":0,"num_object_copies":52578,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":26289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16168,"num_write_kb":64732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002952"},"pg_stats":[{"pgid":"2.f","version":"36'2389","reported_seq":2441,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.923089+0000","last_change":"2024-09-15T21:20:24.031714+0000","last_active":"2024-09-15T21:20:49.923089+0000","last_peered":"2024-09-15T21:20:49.923089+0000","last_clean":"2024-09-15T21:20:49.923089+0000","last_became_active":"2024-09-15T21:20:24.031439+0000","last_became_peered":"2024-09-15T21:20:24.031439+0000","last_unstale":"2024-09-15T21:20:49.923089+0000","last_undegraded":"2024-09-15T21:20:49.923089+0000","last_fullsized":"2024-09-15T21:20:49.923089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2389,"log_dups_size":0,"ondisk_log_size":2389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:51.235292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9785344,"num_objects":2389,"num_object_clones":0,"num_object_copies":4778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2389,"num_write_kb":9556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2322","reported_seq":2406,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.621216+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:20:49.621216+0000","last_peered":"2024-09-15T21:20:49.621216+0000","last_clean":"2024-09-15T21:20:49.621216+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:20:49.621216+0000","last_undegraded":"2024-09-15T21:20:49.621216+0000","last_fullsized":"2024-09-15T21:20:49.621216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":2322,"log_dups_size":0,"ondisk_log_size":2322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9487302,"num_objects":2318,"num_object_clones":0,"num_object_copies":4636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2453","reported_seq":2551,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.658804+0000","last_change":"2024-09-15T21:20:44.532177+0000","last_active":"2024-09-15T21:20:49.658804+0000","last_peered":"2024-09-15T21:20:49.658804+0000","last_clean":"2024-09-15T21:20:49.658804+0000","last_became_active":"2024-09-15T21:20:25.036309+0000","last_became_peered":"2024-09-15T21:20:25.036309+0000","last_unstale":"2024-09-15T21:20:49.658804+0000","last_undegraded":"2024-09-15T21:20:49.658804+0000","last_fullsized":"2024-09-15T21:20:49.658804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":2453,"log_dups_size":0,"ondisk_log_size":2453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:41.470643+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10031104,"num_objects":2449,"num_object_clones":0,"num_object_copies":4898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":553,"num_write_kb":2212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2364","reported_seq":2415,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.667569+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:20:49.667569+0000","last_peered":"2024-09-15T21:20:49.667569+0000","last_clean":"2024-09-15T21:20:49.667569+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:20:49.667569+0000","last_undegraded":"2024-09-15T21:20:49.667569+0000","last_fullsized":"2024-09-15T21:20:49.667569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2364,"log_dups_size":0,"ondisk_log_size":2364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9682944,"num_objects":2364,"num_object_clones":0,"num_object_copies":4728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2364,"num_write_kb":9456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2363","reported_seq":2422,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.868583+0000","last_change":"2024-09-15T21:20:24.036146+0000","last_active":"2024-09-15T21:20:49.868583+0000","last_peered":"2024-09-15T21:20:49.868583+0000","last_clean":"2024-09-15T21:19:50.164448+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:20:49.868583+0000","last_undegraded":"2024-09-15T21:20:24.028206+0000","last_fullsized":"2024-09-15T21:20:49.868583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9651142,"num_objects":2358,"num_object_clones":0,"num_object_copies":4716,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2361,"num_write_kb":9428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2357}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2399","reported_seq":2456,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.915210+0000","last_change":"2024-09-15T21:20:24.035954+0000","last_active":"2024-09-15T21:20:49.915210+0000","last_peered":"2024-09-15T21:20:49.915210+0000","last_clean":"2024-09-15T21:19:51.013140+0000","last_became_active":"2024-09-15T21:20:24.035700+0000","last_became_peered":"2024-09-15T21:20:24.035700+0000","last_unstale":"2024-09-15T21:20:49.915210+0000","last_undegraded":"2024-09-15T21:20:24.025973+0000","last_fullsized":"2024-09-15T21:20:49.915210+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2399,"log_dups_size":0,"ondisk_log_size":2399,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9806790,"num_objects":2396,"num_object_clones":0,"num_object_copies":4792,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2397,"num_write_kb":9579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2395}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2445","reported_seq":2500,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.902816+0000","last_change":"2024-09-15T21:20:24.036588+0000","last_active":"2024-09-15T21:20:49.902816+0000","last_peered":"2024-09-15T21:20:49.902816+0000","last_clean":"2024-09-15T21:19:50.413489+0000","last_became_active":"2024-09-15T21:20:24.036417+0000","last_became_peered":"2024-09-15T21:20:24.036417+0000","last_unstale":"2024-09-15T21:20:49.902816+0000","last_undegraded":"2024-09-15T21:20:24.028324+0000","last_fullsized":"2024-09-15T21:20:49.902816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2445,"log_dups_size":0,"ondisk_log_size":2445,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9991110,"num_objects":2441,"num_object_clones":0,"num_object_copies":4882,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2443,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2439}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2464","reported_seq":2525,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.910565+0000","last_change":"2024-09-15T21:20:24.032808+0000","last_active":"2024-09-15T21:20:49.910565+0000","last_peered":"2024-09-15T21:20:49.910565+0000","last_clean":"2024-09-15T21:19:36.740313+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:20:49.910565+0000","last_undegraded":"2024-09-15T21:20:24.026851+0000","last_fullsized":"2024-09-15T21:20:49.910565+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2464,"log_dups_size":0,"ondisk_log_size":2464,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10084835,"num_objects":2463,"num_object_clones":0,"num_object_copies":4926,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2463}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033321+0000","last_change":"2024-09-15T21:20:24.032881+0000","last_active":"2024-09-15T21:20:25.033321+0000","last_peered":"2024-09-15T21:20:25.033321+0000","last_clean":"2024-09-15T21:20:25.033321+0000","last_became_active":"2024-09-15T21:20:24.032040+0000","last_became_peered":"2024-09-15T21:20:24.032040+0000","last_unstale":"2024-09-15T21:20:25.033321+0000","last_undegraded":"2024-09-15T21:20:25.033321+0000","last_fullsized":"2024-09-15T21:20:25.033321+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:46:45.703820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2387","reported_seq":2489,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.670841+0000","last_change":"2024-09-15T21:20:46.455158+0000","last_active":"2024-09-15T21:20:49.670841+0000","last_peered":"2024-09-15T21:20:49.670841+0000","last_clean":"2024-09-15T21:20:49.670841+0000","last_became_active":"2024-09-15T21:20:25.036224+0000","last_became_peered":"2024-09-15T21:20:25.036224+0000","last_unstale":"2024-09-15T21:20:49.670841+0000","last_undegraded":"2024-09-15T21:20:49.670841+0000","last_fullsized":"2024-09-15T21:20:49.670841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":2387,"log_dups_size":0,"ondisk_log_size":2387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:34.020580+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9748963,"num_objects":2381,"num_object_clones":0,"num_object_copies":4762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":319,"num_write_kb":1276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037255+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:20:25.037255+0000","last_peered":"2024-09-15T21:20:25.037255+0000","last_clean":"2024-09-15T21:20:25.037255+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:20:25.037255+0000","last_undegraded":"2024-09-15T21:20:25.037255+0000","last_fullsized":"2024-09-15T21:20:25.037255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2318","reported_seq":2368,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.912735+0000","last_change":"2024-09-15T21:20:24.035959+0000","last_active":"2024-09-15T21:20:49.912735+0000","last_peered":"2024-09-15T21:20:49.912735+0000","last_clean":"2024-09-15T21:19:51.063279+0000","last_became_active":"2024-09-15T21:20:24.035738+0000","last_became_peered":"2024-09-15T21:20:24.035738+0000","last_unstale":"2024-09-15T21:20:49.912735+0000","last_undegraded":"2024-09-15T21:20:24.026174+0000","last_fullsized":"2024-09-15T21:20:49.912735+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2318,"log_dups_size":0,"ondisk_log_size":2318,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9462726,"num_objects":2312,"num_object_clones":0,"num_object_copies":4624,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2316,"num_write_kb":9244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:25.034061+0000","last_change":"2024-09-15T21:20:24.032266+0000","last_active":"2024-09-15T21:20:25.034061+0000","last_peered":"2024-09-15T21:20:25.034061+0000","last_clean":"2024-09-15T21:20:01.890484+0000","last_became_active":"2024-09-15T21:20:24.032032+0000","last_became_peered":"2024-09-15T21:20:24.032032+0000","last_unstale":"2024-09-15T21:20:25.034061+0000","last_undegraded":"2024-09-15T21:20:24.025362+0000","last_fullsized":"2024-09-15T21:20:25.034061+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2363","reported_seq":2454,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.621135+0000","last_change":"2024-09-15T21:20:47.548905+0000","last_active":"2024-09-15T21:20:49.621135+0000","last_peered":"2024-09-15T21:20:49.621135+0000","last_clean":"2024-09-15T21:20:49.621135+0000","last_became_active":"2024-09-15T21:20:25.035785+0000","last_became_peered":"2024-09-15T21:20:25.035785+0000","last_unstale":"2024-09-15T21:20:49.621135+0000","last_undegraded":"2024-09-15T21:20:49.621135+0000","last_fullsized":"2024-09-15T21:20:49.621135+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.975722+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9658851,"num_objects":2359,"num_object_clones":0,"num_object_copies":4718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":222,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.040332+0000","last_change":"2024-09-15T21:20:25.037062+0000","last_active":"2024-09-15T21:20:25.040332+0000","last_peered":"2024-09-15T21:20:25.040332+0000","last_clean":"2024-09-15T21:20:25.040332+0000","last_became_active":"2024-09-15T21:20:25.036685+0000","last_became_peered":"2024-09-15T21:20:25.036685+0000","last_unstale":"2024-09-15T21:20:25.040332+0000","last_undegraded":"2024-09-15T21:20:25.040332+0000","last_fullsized":"2024-09-15T21:20:25.040332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:47:47.150501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2428","reported_seq":2523,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.658782+0000","last_change":"2024-09-15T21:20:48.647971+0000","last_active":"2024-09-15T21:20:49.658782+0000","last_peered":"2024-09-15T21:20:49.658782+0000","last_clean":"2024-09-15T21:20:49.658782+0000","last_became_active":"2024-09-15T21:20:25.035723+0000","last_became_peered":"2024-09-15T21:20:25.035723+0000","last_unstale":"2024-09-15T21:20:49.658782+0000","last_undegraded":"2024-09-15T21:20:49.658782+0000","last_fullsized":"2024-09-15T21:20:49.658782+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":2428,"log_dups_size":0,"ondisk_log_size":2428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:03.225931+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9920512,"num_objects":2422,"num_object_clones":0,"num_object_copies":4844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2387","reported_seq":2442,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.925722+0000","last_change":"2024-09-15T21:20:24.032431+0000","last_active":"2024-09-15T21:20:49.925722+0000","last_peered":"2024-09-15T21:20:49.925722+0000","last_clean":"2024-09-15T21:19:39.745933+0000","last_became_active":"2024-09-15T21:20:24.032201+0000","last_became_peered":"2024-09-15T21:20:24.032201+0000","last_unstale":"2024-09-15T21:20:49.925722+0000","last_undegraded":"2024-09-15T21:20:24.024923+0000","last_fullsized":"2024-09-15T21:20:49.925722+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2387,"log_dups_size":0,"ondisk_log_size":2387,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9769443,"num_objects":2386,"num_object_clones":0,"num_object_copies":4772,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2386,"num_write_kb":9541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2386}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.034081+0000","last_change":"2024-09-15T21:20:24.032468+0000","last_active":"2024-09-15T21:20:25.034081+0000","last_peered":"2024-09-15T21:20:25.034081+0000","last_clean":"2024-09-15T21:20:25.034081+0000","last_became_active":"2024-09-15T21:20:24.032206+0000","last_became_peered":"2024-09-15T21:20:24.032206+0000","last_unstale":"2024-09-15T21:20:25.034081+0000","last_undegraded":"2024-09-15T21:20:25.034081+0000","last_fullsized":"2024-09-15T21:20:25.034081+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:48.044077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2469","reported_seq":2525,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-15T21:20:49.923043+0000","last_change":"2024-09-15T21:20:25.033511+0000","last_active":"2024-09-15T21:20:49.923043+0000","last_peered":"2024-09-15T21:20:49.923043+0000","last_clean":"2024-09-15T21:19:50.963035+0000","last_became_active":"2024-09-15T21:20:24.031529+0000","last_became_peered":"2024-09-15T21:20:24.031529+0000","last_unstale":"2024-09-15T21:20:49.923043+0000","last_undegraded":"2024-09-15T21:20:49.923043+0000","last_fullsized":"2024-09-15T21:20:49.923043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2469,"log_dups_size":0,"ondisk_log_size":2469,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10081705,"num_objects":2464,"num_object_clones":0,"num_object_copies":4928,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2466,"num_write_kb":9848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2464}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.036378+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:20:25.036378+0000","last_peered":"2024-09-15T21:20:25.036378+0000","last_clean":"2024-09-15T21:20:25.036378+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:20:25.036378+0000","last_undegraded":"2024-09-15T21:20:25.036378+0000","last_fullsized":"2024-09-15T21:20:25.036378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2382","reported_seq":2437,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.899285+0000","last_change":"2024-09-15T21:20:24.036145+0000","last_active":"2024-09-15T21:20:49.899285+0000","last_peered":"2024-09-15T21:20:49.899285+0000","last_clean":"2024-09-15T21:19:39.745731+0000","last_became_active":"2024-09-15T21:20:24.035699+0000","last_became_peered":"2024-09-15T21:20:24.035699+0000","last_unstale":"2024-09-15T21:20:49.899285+0000","last_undegraded":"2024-09-15T21:20:24.026554+0000","last_fullsized":"2024-09-15T21:20:49.899285+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2382,"log_dups_size":0,"ondisk_log_size":2382,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9741254,"num_objects":2380,"num_object_clones":0,"num_object_copies":4760,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2380,"num_write_kb":9514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033483+0000","last_change":"2024-09-15T21:20:24.031742+0000","last_active":"2024-09-15T21:20:25.033483+0000","last_peered":"2024-09-15T21:20:25.033483+0000","last_clean":"2024-09-15T21:20:25.033483+0000","last_became_active":"2024-09-15T21:20:24.031400+0000","last_became_peered":"2024-09-15T21:20:24.031400+0000","last_unstale":"2024-09-15T21:20:25.033483+0000","last_undegraded":"2024-09-15T21:20:25.033483+0000","last_fullsized":"2024-09-15T21:20:25.033483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:36:52.618460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037955+0000","last_change":"2024-09-15T21:20:25.035926+0000","last_active":"2024-09-15T21:20:25.037955+0000","last_peered":"2024-09-15T21:20:25.037955+0000","last_clean":"2024-09-15T21:20:25.037955+0000","last_became_active":"2024-09-15T21:20:25.035642+0000","last_became_peered":"2024-09-15T21:20:25.035642+0000","last_unstale":"2024-09-15T21:20:25.037955+0000","last_undegraded":"2024-09-15T21:20:25.037955+0000","last_fullsized":"2024-09-15T21:20:25.037955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:40:43.901095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2436","reported_seq":2491,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.870830+0000","last_change":"2024-09-15T21:20:24.035912+0000","last_active":"2024-09-15T21:20:49.870830+0000","last_peered":"2024-09-15T21:20:49.870830+0000","last_clean":"2024-09-15T21:19:50.713871+0000","last_became_active":"2024-09-15T21:20:24.035656+0000","last_became_peered":"2024-09-15T21:20:24.035656+0000","last_unstale":"2024-09-15T21:20:49.870830+0000","last_undegraded":"2024-09-15T21:20:24.026450+0000","last_fullsized":"2024-09-15T21:20:49.870830+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2436,"log_dups_size":0,"ondisk_log_size":2436,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9930636,"num_objects":2428,"num_object_clones":0,"num_object_copies":4856,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2433,"num_write_kb":9703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2426}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":156834661,"num_objects":38310,"num_object_clones":0,"num_object_copies":76620,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":38310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28216,"num_write_kb":112750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":313597952,"data_stored":313446206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38369,"ondisk_log_size":38369,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986186,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":255828,"kb_used_data":147780,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94116012,"statfs":{"total":96636764160,"available":96374796288,"internally_reserved":0,"allocated":151326720,"data_stored":151116729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":110620329},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,7,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":257632,"kb_used_data":153808,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94114208,"statfs":{"total":96636764160,"available":96372948992,"internally_reserved":0,"allocated":157499392,"data_stored":157263646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":106294955},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,0,0,9,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157491200,"data_stored":157455070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155992064,"data_stored":155977612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:52.514 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:52.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 21 20 42643 42623 7.92726 5.60156 0.0188811 0.00984864 2024-09-15T21:20:53.028 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:53.029 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:20:53.052 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-15T21:20:52.685937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":156834680,"num_objects":38312,"num_object_clones":0,"num_object_copies":76624,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":38312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28218,"num_write_kb":112752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38372,"ondisk_log_size":38372,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":513460,"kb_used_data":301588,"kb_used_omap":8,"kb_used_meta":211831,"kb_avail":188230220,"statfs":{"total":193273528320,"available":192747745280,"internally_reserved":0,"allocated":308826112,"data_stored":308380375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":216915284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[11,1,1,16,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":76019345,"num_objects":18555,"num_object_clones":0,"num_object_copies":37110,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":18555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8434,"num_write_kb":33796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002883"},"pg_stats":[{"pgid":"2.f","version":"36'2389","reported_seq":2441,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.923089+0000","last_change":"2024-09-15T21:20:24.031714+0000","last_active":"2024-09-15T21:20:49.923089+0000","last_peered":"2024-09-15T21:20:49.923089+0000","last_clean":"2024-09-15T21:20:49.923089+0000","last_became_active":"2024-09-15T21:20:24.031439+0000","last_became_peered":"2024-09-15T21:20:24.031439+0000","last_unstale":"2024-09-15T21:20:49.923089+0000","last_undegraded":"2024-09-15T21:20:49.923089+0000","last_fullsized":"2024-09-15T21:20:49.923089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2389,"log_dups_size":0,"ondisk_log_size":2389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:51.235292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9785344,"num_objects":2389,"num_object_clones":0,"num_object_copies":4778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2389,"num_write_kb":9556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2322","reported_seq":2406,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.621216+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:20:49.621216+0000","last_peered":"2024-09-15T21:20:49.621216+0000","last_clean":"2024-09-15T21:20:49.621216+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:20:49.621216+0000","last_undegraded":"2024-09-15T21:20:49.621216+0000","last_fullsized":"2024-09-15T21:20:49.621216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":2322,"log_dups_size":0,"ondisk_log_size":2322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9487302,"num_objects":2318,"num_object_clones":0,"num_object_copies":4636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2453","reported_seq":2551,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.658804+0000","last_change":"2024-09-15T21:20:44.532177+0000","last_active":"2024-09-15T21:20:49.658804+0000","last_peered":"2024-09-15T21:20:49.658804+0000","last_clean":"2024-09-15T21:20:49.658804+0000","last_became_active":"2024-09-15T21:20:25.036309+0000","last_became_peered":"2024-09-15T21:20:25.036309+0000","last_unstale":"2024-09-15T21:20:49.658804+0000","last_undegraded":"2024-09-15T21:20:49.658804+0000","last_fullsized":"2024-09-15T21:20:49.658804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":2453,"log_dups_size":0,"ondisk_log_size":2453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:41.470643+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10031104,"num_objects":2449,"num_object_clones":0,"num_object_copies":4898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":553,"num_write_kb":2212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2364","reported_seq":2415,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.667569+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:20:49.667569+0000","last_peered":"2024-09-15T21:20:49.667569+0000","last_clean":"2024-09-15T21:20:49.667569+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:20:49.667569+0000","last_undegraded":"2024-09-15T21:20:49.667569+0000","last_fullsized":"2024-09-15T21:20:49.667569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2364,"log_dups_size":0,"ondisk_log_size":2364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9682944,"num_objects":2364,"num_object_clones":0,"num_object_copies":4728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2364,"num_write_kb":9456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2363","reported_seq":2422,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.868583+0000","last_change":"2024-09-15T21:20:24.036146+0000","last_active":"2024-09-15T21:20:49.868583+0000","last_peered":"2024-09-15T21:20:49.868583+0000","last_clean":"2024-09-15T21:19:50.164448+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:20:49.868583+0000","last_undegraded":"2024-09-15T21:20:24.028206+0000","last_fullsized":"2024-09-15T21:20:49.868583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9651142,"num_objects":2358,"num_object_clones":0,"num_object_copies":4716,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2361,"num_write_kb":9428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2357}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2399","reported_seq":2456,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.915210+0000","last_change":"2024-09-15T21:20:24.035954+0000","last_active":"2024-09-15T21:20:49.915210+0000","last_peered":"2024-09-15T21:20:49.915210+0000","last_clean":"2024-09-15T21:19:51.013140+0000","last_became_active":"2024-09-15T21:20:24.035700+0000","last_became_peered":"2024-09-15T21:20:24.035700+0000","last_unstale":"2024-09-15T21:20:49.915210+0000","last_undegraded":"2024-09-15T21:20:24.025973+0000","last_fullsized":"2024-09-15T21:20:49.915210+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2399,"log_dups_size":0,"ondisk_log_size":2399,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9806790,"num_objects":2396,"num_object_clones":0,"num_object_copies":4792,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2397,"num_write_kb":9579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2395}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2445","reported_seq":2500,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.902816+0000","last_change":"2024-09-15T21:20:24.036588+0000","last_active":"2024-09-15T21:20:49.902816+0000","last_peered":"2024-09-15T21:20:49.902816+0000","last_clean":"2024-09-15T21:19:50.413489+0000","last_became_active":"2024-09-15T21:20:24.036417+0000","last_became_peered":"2024-09-15T21:20:24.036417+0000","last_unstale":"2024-09-15T21:20:49.902816+0000","last_undegraded":"2024-09-15T21:20:24.028324+0000","last_fullsized":"2024-09-15T21:20:49.902816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2445,"log_dups_size":0,"ondisk_log_size":2445,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9991110,"num_objects":2441,"num_object_clones":0,"num_object_copies":4882,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2443,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2439}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2464","reported_seq":2525,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.910565+0000","last_change":"2024-09-15T21:20:24.032808+0000","last_active":"2024-09-15T21:20:49.910565+0000","last_peered":"2024-09-15T21:20:49.910565+0000","last_clean":"2024-09-15T21:19:36.740313+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:20:49.910565+0000","last_undegraded":"2024-09-15T21:20:24.026851+0000","last_fullsized":"2024-09-15T21:20:49.910565+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2464,"log_dups_size":0,"ondisk_log_size":2464,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10084835,"num_objects":2463,"num_object_clones":0,"num_object_copies":4926,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2463}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033321+0000","last_change":"2024-09-15T21:20:24.032881+0000","last_active":"2024-09-15T21:20:25.033321+0000","last_peered":"2024-09-15T21:20:25.033321+0000","last_clean":"2024-09-15T21:20:25.033321+0000","last_became_active":"2024-09-15T21:20:24.032040+0000","last_became_peered":"2024-09-15T21:20:24.032040+0000","last_unstale":"2024-09-15T21:20:25.033321+0000","last_undegraded":"2024-09-15T21:20:25.033321+0000","last_fullsized":"2024-09-15T21:20:25.033321+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:46:45.703820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2387","reported_seq":2489,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.670841+0000","last_change":"2024-09-15T21:20:46.455158+0000","last_active":"2024-09-15T21:20:49.670841+0000","last_peered":"2024-09-15T21:20:49.670841+0000","last_clean":"2024-09-15T21:20:49.670841+0000","last_became_active":"2024-09-15T21:20:25.036224+0000","last_became_peered":"2024-09-15T21:20:25.036224+0000","last_unstale":"2024-09-15T21:20:49.670841+0000","last_undegraded":"2024-09-15T21:20:49.670841+0000","last_fullsized":"2024-09-15T21:20:49.670841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":2387,"log_dups_size":0,"ondisk_log_size":2387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:34.020580+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9748963,"num_objects":2381,"num_object_clones":0,"num_object_copies":4762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":319,"num_write_kb":1276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037255+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:20:25.037255+0000","last_peered":"2024-09-15T21:20:25.037255+0000","last_clean":"2024-09-15T21:20:25.037255+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:20:25.037255+0000","last_undegraded":"2024-09-15T21:20:25.037255+0000","last_fullsized":"2024-09-15T21:20:25.037255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2318","reported_seq":2368,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.912735+0000","last_change":"2024-09-15T21:20:24.035959+0000","last_active":"2024-09-15T21:20:49.912735+0000","last_peered":"2024-09-15T21:20:49.912735+0000","last_clean":"2024-09-15T21:19:51.063279+0000","last_became_active":"2024-09-15T21:20:24.035738+0000","last_became_peered":"2024-09-15T21:20:24.035738+0000","last_unstale":"2024-09-15T21:20:49.912735+0000","last_undegraded":"2024-09-15T21:20:24.026174+0000","last_fullsized":"2024-09-15T21:20:49.912735+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2318,"log_dups_size":0,"ondisk_log_size":2318,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9462726,"num_objects":2312,"num_object_clones":0,"num_object_copies":4624,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2316,"num_write_kb":9244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:25.034061+0000","last_change":"2024-09-15T21:20:24.032266+0000","last_active":"2024-09-15T21:20:25.034061+0000","last_peered":"2024-09-15T21:20:25.034061+0000","last_clean":"2024-09-15T21:20:01.890484+0000","last_became_active":"2024-09-15T21:20:24.032032+0000","last_became_peered":"2024-09-15T21:20:24.032032+0000","last_unstale":"2024-09-15T21:20:25.034061+0000","last_undegraded":"2024-09-15T21:20:24.025362+0000","last_fullsized":"2024-09-15T21:20:25.034061+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2363","reported_seq":2454,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.621135+0000","last_change":"2024-09-15T21:20:47.548905+0000","last_active":"2024-09-15T21:20:49.621135+0000","last_peered":"2024-09-15T21:20:49.621135+0000","last_clean":"2024-09-15T21:20:49.621135+0000","last_became_active":"2024-09-15T21:20:25.035785+0000","last_became_peered":"2024-09-15T21:20:25.035785+0000","last_unstale":"2024-09-15T21:20:49.621135+0000","last_undegraded":"2024-09-15T21:20:49.621135+0000","last_fullsized":"2024-09-15T21:20:49.621135+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.975722+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9658851,"num_objects":2359,"num_object_clones":0,"num_object_copies":4718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":222,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.040332+0000","last_change":"2024-09-15T21:20:25.037062+0000","last_active":"2024-09-15T21:20:25.040332+0000","last_peered":"2024-09-15T21:20:25.040332+0000","last_clean":"2024-09-15T21:20:25.040332+0000","last_became_active":"2024-09-15T21:20:25.036685+0000","last_became_peered":"2024-09-15T21:20:25.036685+0000","last_unstale":"2024-09-15T21:20:25.040332+0000","last_undegraded":"2024-09-15T21:20:25.040332+0000","last_fullsized":"2024-09-15T21:20:25.040332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:47:47.150501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2428","reported_seq":2523,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.658782+0000","last_change":"2024-09-15T21:20:48.647971+0000","last_active":"2024-09-15T21:20:49.658782+0000","last_peered":"2024-09-15T21:20:49.658782+0000","last_clean":"2024-09-15T21:20:49.658782+0000","last_became_active":"2024-09-15T21:20:25.035723+0000","last_became_peered":"2024-09-15T21:20:25.035723+0000","last_unstale":"2024-09-15T21:20:49.658782+0000","last_undegraded":"2024-09-15T21:20:49.658782+0000","last_fullsized":"2024-09-15T21:20:49.658782+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":2428,"log_dups_size":0,"ondisk_log_size":2428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:03.225931+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9920512,"num_objects":2422,"num_object_clones":0,"num_object_copies":4844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2387","reported_seq":2442,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.925722+0000","last_change":"2024-09-15T21:20:24.032431+0000","last_active":"2024-09-15T21:20:49.925722+0000","last_peered":"2024-09-15T21:20:49.925722+0000","last_clean":"2024-09-15T21:19:39.745933+0000","last_became_active":"2024-09-15T21:20:24.032201+0000","last_became_peered":"2024-09-15T21:20:24.032201+0000","last_unstale":"2024-09-15T21:20:49.925722+0000","last_undegraded":"2024-09-15T21:20:24.024923+0000","last_fullsized":"2024-09-15T21:20:49.925722+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2387,"log_dups_size":0,"ondisk_log_size":2387,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9769443,"num_objects":2386,"num_object_clones":0,"num_object_copies":4772,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2386,"num_write_kb":9541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2386}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.034081+0000","last_change":"2024-09-15T21:20:24.032468+0000","last_active":"2024-09-15T21:20:25.034081+0000","last_peered":"2024-09-15T21:20:25.034081+0000","last_clean":"2024-09-15T21:20:25.034081+0000","last_became_active":"2024-09-15T21:20:24.032206+0000","last_became_peered":"2024-09-15T21:20:24.032206+0000","last_unstale":"2024-09-15T21:20:25.034081+0000","last_undegraded":"2024-09-15T21:20:25.034081+0000","last_fullsized":"2024-09-15T21:20:25.034081+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:48.044077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2469","reported_seq":2525,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-15T21:20:49.923043+0000","last_change":"2024-09-15T21:20:25.033511+0000","last_active":"2024-09-15T21:20:49.923043+0000","last_peered":"2024-09-15T21:20:49.923043+0000","last_clean":"2024-09-15T21:19:50.963035+0000","last_became_active":"2024-09-15T21:20:24.031529+0000","last_became_peered":"2024-09-15T21:20:24.031529+0000","last_unstale":"2024-09-15T21:20:49.923043+0000","last_undegraded":"2024-09-15T21:20:49.923043+0000","last_fullsized":"2024-09-15T21:20:49.923043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2469,"log_dups_size":0,"ondisk_log_size":2469,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10081705,"num_objects":2464,"num_object_clones":0,"num_object_copies":4928,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2466,"num_write_kb":9848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2464}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.036378+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:20:25.036378+0000","last_peered":"2024-09-15T21:20:25.036378+0000","last_clean":"2024-09-15T21:20:25.036378+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:20:25.036378+0000","last_undegraded":"2024-09-15T21:20:25.036378+0000","last_fullsized":"2024-09-15T21:20:25.036378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2382","reported_seq":2437,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.899285+0000","last_change":"2024-09-15T21:20:24.036145+0000","last_active":"2024-09-15T21:20:49.899285+0000","last_peered":"2024-09-15T21:20:49.899285+0000","last_clean":"2024-09-15T21:19:39.745731+0000","last_became_active":"2024-09-15T21:20:24.035699+0000","last_became_peered":"2024-09-15T21:20:24.035699+0000","last_unstale":"2024-09-15T21:20:49.899285+0000","last_undegraded":"2024-09-15T21:20:24.026554+0000","last_fullsized":"2024-09-15T21:20:49.899285+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2382,"log_dups_size":0,"ondisk_log_size":2382,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9741254,"num_objects":2380,"num_object_clones":0,"num_object_copies":4760,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2380,"num_write_kb":9514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033483+0000","last_change":"2024-09-15T21:20:24.031742+0000","last_active":"2024-09-15T21:20:25.033483+0000","last_peered":"2024-09-15T21:20:25.033483+0000","last_clean":"2024-09-15T21:20:25.033483+0000","last_became_active":"2024-09-15T21:20:24.031400+0000","last_became_peered":"2024-09-15T21:20:24.031400+0000","last_unstale":"2024-09-15T21:20:25.033483+0000","last_undegraded":"2024-09-15T21:20:25.033483+0000","last_fullsized":"2024-09-15T21:20:25.033483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:36:52.618460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037955+0000","last_change":"2024-09-15T21:20:25.035926+0000","last_active":"2024-09-15T21:20:25.037955+0000","last_peered":"2024-09-15T21:20:25.037955+0000","last_clean":"2024-09-15T21:20:25.037955+0000","last_became_active":"2024-09-15T21:20:25.035642+0000","last_became_peered":"2024-09-15T21:20:25.035642+0000","last_unstale":"2024-09-15T21:20:25.037955+0000","last_undegraded":"2024-09-15T21:20:25.037955+0000","last_fullsized":"2024-09-15T21:20:25.037955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:40:43.901095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2436","reported_seq":2491,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.870830+0000","last_change":"2024-09-15T21:20:24.035912+0000","last_active":"2024-09-15T21:20:49.870830+0000","last_peered":"2024-09-15T21:20:49.870830+0000","last_clean":"2024-09-15T21:19:50.713871+0000","last_became_active":"2024-09-15T21:20:24.035656+0000","last_became_peered":"2024-09-15T21:20:24.035656+0000","last_unstale":"2024-09-15T21:20:49.870830+0000","last_undegraded":"2024-09-15T21:20:24.026450+0000","last_fullsized":"2024-09-15T21:20:49.870830+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2436,"log_dups_size":0,"ondisk_log_size":2436,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9930636,"num_objects":2428,"num_object_clones":0,"num_object_copies":4856,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2433,"num_write_kb":9703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2426}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":156834661,"num_objects":38310,"num_object_clones":0,"num_object_copies":76620,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":38310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28216,"num_write_kb":112750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":313597952,"data_stored":313446206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38369,"ondisk_log_size":38369,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986186,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":255828,"kb_used_data":147780,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94116012,"statfs":{"total":96636764160,"available":96374796288,"internally_reserved":0,"allocated":151326720,"data_stored":151116729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":110620329},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,7,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":257632,"kb_used_data":153808,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94114208,"statfs":{"total":96636764160,"available":96372948992,"internally_reserved":0,"allocated":157499392,"data_stored":157263646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":106294955},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,0,0,9,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157491200,"data_stored":157455070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155992064,"data_stored":155977612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:53.053 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:53.583 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:53.583 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:20:53.605 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-15T21:20:52.685937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":156834680,"num_objects":38312,"num_object_clones":0,"num_object_copies":76624,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":38312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28218,"num_write_kb":112752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38372,"ondisk_log_size":38372,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":513460,"kb_used_data":301588,"kb_used_omap":8,"kb_used_meta":211831,"kb_avail":188230220,"statfs":{"total":193273528320,"available":192747745280,"internally_reserved":0,"allocated":308826112,"data_stored":308380375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":216915284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[11,1,1,16,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":76019345,"num_objects":18555,"num_object_clones":0,"num_object_copies":37110,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-22,"num_objects_dirty":18555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8434,"num_write_kb":33796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002883"},"pg_stats":[{"pgid":"2.f","version":"36'2389","reported_seq":2441,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.923089+0000","last_change":"2024-09-15T21:20:24.031714+0000","last_active":"2024-09-15T21:20:49.923089+0000","last_peered":"2024-09-15T21:20:49.923089+0000","last_clean":"2024-09-15T21:20:49.923089+0000","last_became_active":"2024-09-15T21:20:24.031439+0000","last_became_peered":"2024-09-15T21:20:24.031439+0000","last_unstale":"2024-09-15T21:20:49.923089+0000","last_undegraded":"2024-09-15T21:20:49.923089+0000","last_fullsized":"2024-09-15T21:20:49.923089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2389,"log_dups_size":0,"ondisk_log_size":2389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:51.235292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9785344,"num_objects":2389,"num_object_clones":0,"num_object_copies":4778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2389,"num_write_kb":9556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2322","reported_seq":2406,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.621216+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:20:49.621216+0000","last_peered":"2024-09-15T21:20:49.621216+0000","last_clean":"2024-09-15T21:20:49.621216+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:20:49.621216+0000","last_undegraded":"2024-09-15T21:20:49.621216+0000","last_fullsized":"2024-09-15T21:20:49.621216+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":2322,"log_dups_size":0,"ondisk_log_size":2322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9487302,"num_objects":2318,"num_object_clones":0,"num_object_copies":4636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":603,"num_write_kb":2412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2453","reported_seq":2551,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.658804+0000","last_change":"2024-09-15T21:20:44.532177+0000","last_active":"2024-09-15T21:20:49.658804+0000","last_peered":"2024-09-15T21:20:49.658804+0000","last_clean":"2024-09-15T21:20:49.658804+0000","last_became_active":"2024-09-15T21:20:25.036309+0000","last_became_peered":"2024-09-15T21:20:25.036309+0000","last_unstale":"2024-09-15T21:20:49.658804+0000","last_undegraded":"2024-09-15T21:20:49.658804+0000","last_fullsized":"2024-09-15T21:20:49.658804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":2453,"log_dups_size":0,"ondisk_log_size":2453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:41.470643+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10031104,"num_objects":2449,"num_object_clones":0,"num_object_copies":4898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":553,"num_write_kb":2212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2364","reported_seq":2415,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.667569+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:20:49.667569+0000","last_peered":"2024-09-15T21:20:49.667569+0000","last_clean":"2024-09-15T21:20:49.667569+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:20:49.667569+0000","last_undegraded":"2024-09-15T21:20:49.667569+0000","last_fullsized":"2024-09-15T21:20:49.667569+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2364,"log_dups_size":0,"ondisk_log_size":2364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9682944,"num_objects":2364,"num_object_clones":0,"num_object_copies":4728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2364,"num_write_kb":9456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2363","reported_seq":2422,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.868583+0000","last_change":"2024-09-15T21:20:24.036146+0000","last_active":"2024-09-15T21:20:49.868583+0000","last_peered":"2024-09-15T21:20:49.868583+0000","last_clean":"2024-09-15T21:19:50.164448+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:20:49.868583+0000","last_undegraded":"2024-09-15T21:20:24.028206+0000","last_fullsized":"2024-09-15T21:20:49.868583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9651142,"num_objects":2358,"num_object_clones":0,"num_object_copies":4716,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2361,"num_write_kb":9428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2357}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2399","reported_seq":2456,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.915210+0000","last_change":"2024-09-15T21:20:24.035954+0000","last_active":"2024-09-15T21:20:49.915210+0000","last_peered":"2024-09-15T21:20:49.915210+0000","last_clean":"2024-09-15T21:19:51.013140+0000","last_became_active":"2024-09-15T21:20:24.035700+0000","last_became_peered":"2024-09-15T21:20:24.035700+0000","last_unstale":"2024-09-15T21:20:49.915210+0000","last_undegraded":"2024-09-15T21:20:24.025973+0000","last_fullsized":"2024-09-15T21:20:49.915210+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2399,"log_dups_size":0,"ondisk_log_size":2399,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9806790,"num_objects":2396,"num_object_clones":0,"num_object_copies":4792,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2397,"num_write_kb":9579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2395}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2445","reported_seq":2500,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.902816+0000","last_change":"2024-09-15T21:20:24.036588+0000","last_active":"2024-09-15T21:20:49.902816+0000","last_peered":"2024-09-15T21:20:49.902816+0000","last_clean":"2024-09-15T21:19:50.413489+0000","last_became_active":"2024-09-15T21:20:24.036417+0000","last_became_peered":"2024-09-15T21:20:24.036417+0000","last_unstale":"2024-09-15T21:20:49.902816+0000","last_undegraded":"2024-09-15T21:20:24.028324+0000","last_fullsized":"2024-09-15T21:20:49.902816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2445,"log_dups_size":0,"ondisk_log_size":2445,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9991110,"num_objects":2441,"num_object_clones":0,"num_object_copies":4882,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2443,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2439}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2464","reported_seq":2525,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.910565+0000","last_change":"2024-09-15T21:20:24.032808+0000","last_active":"2024-09-15T21:20:49.910565+0000","last_peered":"2024-09-15T21:20:49.910565+0000","last_clean":"2024-09-15T21:19:36.740313+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:20:49.910565+0000","last_undegraded":"2024-09-15T21:20:24.026851+0000","last_fullsized":"2024-09-15T21:20:49.910565+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2464,"log_dups_size":0,"ondisk_log_size":2464,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10084835,"num_objects":2463,"num_object_clones":0,"num_object_copies":4926,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2463,"num_write_kb":9849,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2463}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033321+0000","last_change":"2024-09-15T21:20:24.032881+0000","last_active":"2024-09-15T21:20:25.033321+0000","last_peered":"2024-09-15T21:20:25.033321+0000","last_clean":"2024-09-15T21:20:25.033321+0000","last_became_active":"2024-09-15T21:20:24.032040+0000","last_became_peered":"2024-09-15T21:20:24.032040+0000","last_unstale":"2024-09-15T21:20:25.033321+0000","last_undegraded":"2024-09-15T21:20:25.033321+0000","last_fullsized":"2024-09-15T21:20:25.033321+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:46:45.703820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2387","reported_seq":2489,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.670841+0000","last_change":"2024-09-15T21:20:46.455158+0000","last_active":"2024-09-15T21:20:49.670841+0000","last_peered":"2024-09-15T21:20:49.670841+0000","last_clean":"2024-09-15T21:20:49.670841+0000","last_became_active":"2024-09-15T21:20:25.036224+0000","last_became_peered":"2024-09-15T21:20:25.036224+0000","last_unstale":"2024-09-15T21:20:49.670841+0000","last_undegraded":"2024-09-15T21:20:49.670841+0000","last_fullsized":"2024-09-15T21:20:49.670841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":2387,"log_dups_size":0,"ondisk_log_size":2387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:34.020580+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9748963,"num_objects":2381,"num_object_clones":0,"num_object_copies":4762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":319,"num_write_kb":1276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037255+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:20:25.037255+0000","last_peered":"2024-09-15T21:20:25.037255+0000","last_clean":"2024-09-15T21:20:25.037255+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:20:25.037255+0000","last_undegraded":"2024-09-15T21:20:25.037255+0000","last_fullsized":"2024-09-15T21:20:25.037255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2318","reported_seq":2368,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.912735+0000","last_change":"2024-09-15T21:20:24.035959+0000","last_active":"2024-09-15T21:20:49.912735+0000","last_peered":"2024-09-15T21:20:49.912735+0000","last_clean":"2024-09-15T21:19:51.063279+0000","last_became_active":"2024-09-15T21:20:24.035738+0000","last_became_peered":"2024-09-15T21:20:24.035738+0000","last_unstale":"2024-09-15T21:20:49.912735+0000","last_undegraded":"2024-09-15T21:20:24.026174+0000","last_fullsized":"2024-09-15T21:20:49.912735+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2318,"log_dups_size":0,"ondisk_log_size":2318,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9462726,"num_objects":2312,"num_object_clones":0,"num_object_copies":4624,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2316,"num_write_kb":9244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:25.034061+0000","last_change":"2024-09-15T21:20:24.032266+0000","last_active":"2024-09-15T21:20:25.034061+0000","last_peered":"2024-09-15T21:20:25.034061+0000","last_clean":"2024-09-15T21:20:01.890484+0000","last_became_active":"2024-09-15T21:20:24.032032+0000","last_became_peered":"2024-09-15T21:20:24.032032+0000","last_unstale":"2024-09-15T21:20:25.034061+0000","last_undegraded":"2024-09-15T21:20:24.025362+0000","last_fullsized":"2024-09-15T21:20:25.034061+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2363","reported_seq":2454,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.621135+0000","last_change":"2024-09-15T21:20:47.548905+0000","last_active":"2024-09-15T21:20:49.621135+0000","last_peered":"2024-09-15T21:20:49.621135+0000","last_clean":"2024-09-15T21:20:49.621135+0000","last_became_active":"2024-09-15T21:20:25.035785+0000","last_became_peered":"2024-09-15T21:20:25.035785+0000","last_unstale":"2024-09-15T21:20:49.621135+0000","last_undegraded":"2024-09-15T21:20:49.621135+0000","last_fullsized":"2024-09-15T21:20:49.621135+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":2363,"log_dups_size":0,"ondisk_log_size":2363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.975722+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9658851,"num_objects":2359,"num_object_clones":0,"num_object_copies":4718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":222,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.040332+0000","last_change":"2024-09-15T21:20:25.037062+0000","last_active":"2024-09-15T21:20:25.040332+0000","last_peered":"2024-09-15T21:20:25.040332+0000","last_clean":"2024-09-15T21:20:25.040332+0000","last_became_active":"2024-09-15T21:20:25.036685+0000","last_became_peered":"2024-09-15T21:20:25.036685+0000","last_unstale":"2024-09-15T21:20:25.040332+0000","last_undegraded":"2024-09-15T21:20:25.040332+0000","last_fullsized":"2024-09-15T21:20:25.040332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:47:47.150501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2428","reported_seq":2523,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:49.658782+0000","last_change":"2024-09-15T21:20:48.647971+0000","last_active":"2024-09-15T21:20:49.658782+0000","last_peered":"2024-09-15T21:20:49.658782+0000","last_clean":"2024-09-15T21:20:49.658782+0000","last_became_active":"2024-09-15T21:20:25.035723+0000","last_became_peered":"2024-09-15T21:20:25.035723+0000","last_unstale":"2024-09-15T21:20:49.658782+0000","last_undegraded":"2024-09-15T21:20:49.658782+0000","last_fullsized":"2024-09-15T21:20:49.658782+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":2428,"log_dups_size":0,"ondisk_log_size":2428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:03.225931+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9920512,"num_objects":2422,"num_object_clones":0,"num_object_copies":4844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2387","reported_seq":2442,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.925722+0000","last_change":"2024-09-15T21:20:24.032431+0000","last_active":"2024-09-15T21:20:49.925722+0000","last_peered":"2024-09-15T21:20:49.925722+0000","last_clean":"2024-09-15T21:19:39.745933+0000","last_became_active":"2024-09-15T21:20:24.032201+0000","last_became_peered":"2024-09-15T21:20:24.032201+0000","last_unstale":"2024-09-15T21:20:49.925722+0000","last_undegraded":"2024-09-15T21:20:24.024923+0000","last_fullsized":"2024-09-15T21:20:49.925722+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2387,"log_dups_size":0,"ondisk_log_size":2387,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9769443,"num_objects":2386,"num_object_clones":0,"num_object_copies":4772,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2386,"num_write_kb":9541,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2386}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.034081+0000","last_change":"2024-09-15T21:20:24.032468+0000","last_active":"2024-09-15T21:20:25.034081+0000","last_peered":"2024-09-15T21:20:25.034081+0000","last_clean":"2024-09-15T21:20:25.034081+0000","last_became_active":"2024-09-15T21:20:24.032206+0000","last_became_peered":"2024-09-15T21:20:24.032206+0000","last_unstale":"2024-09-15T21:20:25.034081+0000","last_undegraded":"2024-09-15T21:20:25.034081+0000","last_fullsized":"2024-09-15T21:20:25.034081+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:48.044077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2469","reported_seq":2525,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-15T21:20:49.923043+0000","last_change":"2024-09-15T21:20:25.033511+0000","last_active":"2024-09-15T21:20:49.923043+0000","last_peered":"2024-09-15T21:20:49.923043+0000","last_clean":"2024-09-15T21:19:50.963035+0000","last_became_active":"2024-09-15T21:20:24.031529+0000","last_became_peered":"2024-09-15T21:20:24.031529+0000","last_unstale":"2024-09-15T21:20:49.923043+0000","last_undegraded":"2024-09-15T21:20:49.923043+0000","last_fullsized":"2024-09-15T21:20:49.923043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2469,"log_dups_size":0,"ondisk_log_size":2469,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10081705,"num_objects":2464,"num_object_clones":0,"num_object_copies":4928,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2466,"num_write_kb":9848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2464}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.036378+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:20:25.036378+0000","last_peered":"2024-09-15T21:20:25.036378+0000","last_clean":"2024-09-15T21:20:25.036378+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:20:25.036378+0000","last_undegraded":"2024-09-15T21:20:25.036378+0000","last_fullsized":"2024-09-15T21:20:25.036378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2382","reported_seq":2437,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.899285+0000","last_change":"2024-09-15T21:20:24.036145+0000","last_active":"2024-09-15T21:20:49.899285+0000","last_peered":"2024-09-15T21:20:49.899285+0000","last_clean":"2024-09-15T21:19:39.745731+0000","last_became_active":"2024-09-15T21:20:24.035699+0000","last_became_peered":"2024-09-15T21:20:24.035699+0000","last_unstale":"2024-09-15T21:20:49.899285+0000","last_undegraded":"2024-09-15T21:20:24.026554+0000","last_fullsized":"2024-09-15T21:20:49.899285+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2382,"log_dups_size":0,"ondisk_log_size":2382,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9741254,"num_objects":2380,"num_object_clones":0,"num_object_copies":4760,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2380,"num_write_kb":9514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033483+0000","last_change":"2024-09-15T21:20:24.031742+0000","last_active":"2024-09-15T21:20:25.033483+0000","last_peered":"2024-09-15T21:20:25.033483+0000","last_clean":"2024-09-15T21:20:25.033483+0000","last_became_active":"2024-09-15T21:20:24.031400+0000","last_became_peered":"2024-09-15T21:20:24.031400+0000","last_unstale":"2024-09-15T21:20:25.033483+0000","last_undegraded":"2024-09-15T21:20:25.033483+0000","last_fullsized":"2024-09-15T21:20:25.033483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:36:52.618460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037955+0000","last_change":"2024-09-15T21:20:25.035926+0000","last_active":"2024-09-15T21:20:25.037955+0000","last_peered":"2024-09-15T21:20:25.037955+0000","last_clean":"2024-09-15T21:20:25.037955+0000","last_became_active":"2024-09-15T21:20:25.035642+0000","last_became_peered":"2024-09-15T21:20:25.035642+0000","last_unstale":"2024-09-15T21:20:25.037955+0000","last_undegraded":"2024-09-15T21:20:25.037955+0000","last_fullsized":"2024-09-15T21:20:25.037955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:40:43.901095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2436","reported_seq":2491,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:49.870830+0000","last_change":"2024-09-15T21:20:24.035912+0000","last_active":"2024-09-15T21:20:49.870830+0000","last_peered":"2024-09-15T21:20:49.870830+0000","last_clean":"2024-09-15T21:19:50.713871+0000","last_became_active":"2024-09-15T21:20:24.035656+0000","last_became_peered":"2024-09-15T21:20:24.035656+0000","last_unstale":"2024-09-15T21:20:49.870830+0000","last_undegraded":"2024-09-15T21:20:24.026450+0000","last_fullsized":"2024-09-15T21:20:49.870830+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2436,"log_dups_size":0,"ondisk_log_size":2436,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9930636,"num_objects":2428,"num_object_clones":0,"num_object_copies":4856,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2433,"num_write_kb":9703,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2426}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":156834661,"num_objects":38310,"num_object_clones":0,"num_object_copies":76620,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":38310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28216,"num_write_kb":112750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":313597952,"data_stored":313446206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38369,"ondisk_log_size":38369,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986186,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":255828,"kb_used_data":147780,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94116012,"statfs":{"total":96636764160,"available":96374796288,"internally_reserved":0,"allocated":151326720,"data_stored":151116729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":110620329},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,7,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":257632,"kb_used_data":153808,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94114208,"statfs":{"total":96636764160,"available":96372948992,"internally_reserved":0,"allocated":157499392,"data_stored":157263646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":106294955},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,0,0,9,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157491200,"data_stored":157455070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":155992064,"data_stored":155977612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:53.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 22 20 44141 44121 7.8329 5.85156 0.0198324 0.00996741 2024-09-15T21:20:54.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 23 20 45788 45768 7.77204 6.43359 0.0166832 0.0100466 2024-09-15T21:20:55.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 24 20 47376 47356 7.70665 6.20312 0.0151303 0.0101315 2024-09-15T21:20:56.607 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:56.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 25 20 48829 48809 7.62541 5.67578 0.0185778 0.0102389 2024-09-15T21:20:57.124 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:57.126 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:20:57.147 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-15T21:20:56.686687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":184105848,"num_objects":44970,"num_object_clones":0,"num_object_copies":89940,"num_objects_missing_on_primary":14,"num_objects_missing":14,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":44970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34876,"num_write_kb":139384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45030,"ondisk_log_size":45030,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":584344,"kb_used_data":355832,"kb_used_omap":8,"kb_used_meta":228471,"kb_avail":188159336,"statfs":{"total":193273528320,"available":192675160064,"internally_reserved":0,"allocated":364371968,"data_stored":363926231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":233954644},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[11,3,4,7,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":86967953,"num_objects":21228,"num_object_clones":0,"num_object_copies":42456,"num_objects_missing_on_primary":-11,"num_objects_missing":-11,"num_objects_degraded":-16,"num_objects_misplaced":0,"num_objects_unfound":-20,"num_objects_dirty":21228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11107,"num_write_kb":44488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":-966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002582"},"pg_stats":[{"pgid":"2.f","version":"36'2883","reported_seq":2935,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:54.931734+0000","last_change":"2024-09-15T21:20:24.031714+0000","last_active":"2024-09-15T21:20:54.931734+0000","last_peered":"2024-09-15T21:20:54.931734+0000","last_clean":"2024-09-15T21:20:54.931734+0000","last_became_active":"2024-09-15T21:20:24.031439+0000","last_became_peered":"2024-09-15T21:20:24.031439+0000","last_unstale":"2024-09-15T21:20:54.931734+0000","last_undegraded":"2024-09-15T21:20:54.931734+0000","last_fullsized":"2024-09-15T21:20:54.931734+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2883,"log_dups_size":0,"ondisk_log_size":2883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:51.235292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11808768,"num_objects":2883,"num_object_clones":0,"num_object_copies":5766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2883,"num_write_kb":11532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2649","reported_seq":2733,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.994035+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:20:52.994035+0000","last_peered":"2024-09-15T21:20:52.994035+0000","last_clean":"2024-09-15T21:20:52.994035+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:20:52.994035+0000","last_undegraded":"2024-09-15T21:20:52.994035+0000","last_fullsized":"2024-09-15T21:20:52.994035+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":2649,"log_dups_size":0,"ondisk_log_size":2649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10826694,"num_objects":2645,"num_object_clones":0,"num_object_copies":5290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":930,"num_write_kb":3720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2807","reported_seq":2905,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.990305+0000","last_change":"2024-09-15T21:20:44.532177+0000","last_active":"2024-09-15T21:20:52.990305+0000","last_peered":"2024-09-15T21:20:52.990305+0000","last_clean":"2024-09-15T21:20:52.990305+0000","last_became_active":"2024-09-15T21:20:25.036309+0000","last_became_peered":"2024-09-15T21:20:25.036309+0000","last_unstale":"2024-09-15T21:20:52.990305+0000","last_undegraded":"2024-09-15T21:20:52.990305+0000","last_fullsized":"2024-09-15T21:20:52.990305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":2807,"log_dups_size":0,"ondisk_log_size":2807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:41.470643+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11481088,"num_objects":2803,"num_object_clones":0,"num_object_copies":5606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":907,"num_write_kb":3628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2650","reported_seq":2701,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.985504+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:20:52.985504+0000","last_peered":"2024-09-15T21:20:52.985504+0000","last_clean":"2024-09-15T21:20:52.985504+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:20:52.985504+0000","last_undegraded":"2024-09-15T21:20:52.985504+0000","last_fullsized":"2024-09-15T21:20:52.985504+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2650,"log_dups_size":0,"ondisk_log_size":2650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10854400,"num_objects":2650,"num_object_clones":0,"num_object_copies":5300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2650,"num_write_kb":10600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2832","reported_seq":2890,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.940659+0000","last_change":"2024-09-15T21:20:24.036146+0000","last_active":"2024-09-15T21:20:54.940659+0000","last_peered":"2024-09-15T21:20:54.940659+0000","last_clean":"2024-09-15T21:19:50.164448+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:20:54.940659+0000","last_undegraded":"2024-09-15T21:20:24.028206+0000","last_fullsized":"2024-09-15T21:20:54.940659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2832,"log_dups_size":0,"ondisk_log_size":2832,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11572166,"num_objects":2827,"num_object_clones":0,"num_object_copies":5654,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2830,"num_write_kb":11304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2826}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2870","reported_seq":2926,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.942764+0000","last_change":"2024-09-15T21:20:24.035954+0000","last_active":"2024-09-15T21:20:54.942764+0000","last_peered":"2024-09-15T21:20:54.942764+0000","last_clean":"2024-09-15T21:19:51.013140+0000","last_became_active":"2024-09-15T21:20:24.035700+0000","last_became_peered":"2024-09-15T21:20:24.035700+0000","last_unstale":"2024-09-15T21:20:54.942764+0000","last_undegraded":"2024-09-15T21:20:24.025973+0000","last_fullsized":"2024-09-15T21:20:54.942764+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2870,"log_dups_size":0,"ondisk_log_size":2870,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11736006,"num_objects":2867,"num_object_clones":0,"num_object_copies":5734,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2868,"num_write_kb":11463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2866}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2950","reported_seq":3006,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.929058+0000","last_change":"2024-09-15T21:20:24.036588+0000","last_active":"2024-09-15T21:20:54.929058+0000","last_peered":"2024-09-15T21:20:54.929058+0000","last_clean":"2024-09-15T21:19:50.413489+0000","last_became_active":"2024-09-15T21:20:24.036417+0000","last_became_peered":"2024-09-15T21:20:24.036417+0000","last_unstale":"2024-09-15T21:20:54.929058+0000","last_undegraded":"2024-09-15T21:20:24.028324+0000","last_fullsized":"2024-09-15T21:20:54.929058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2950,"log_dups_size":0,"ondisk_log_size":2950,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12059590,"num_objects":2946,"num_object_clones":0,"num_object_copies":5892,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2948,"num_write_kb":11780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2944}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2936","reported_seq":2995,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.937586+0000","last_change":"2024-09-15T21:20:24.032808+0000","last_active":"2024-09-15T21:20:54.937586+0000","last_peered":"2024-09-15T21:20:54.937586+0000","last_clean":"2024-09-15T21:19:36.740313+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:20:54.937586+0000","last_undegraded":"2024-09-15T21:20:24.026851+0000","last_fullsized":"2024-09-15T21:20:54.937586+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2936,"log_dups_size":0,"ondisk_log_size":2936,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12018147,"num_objects":2935,"num_object_clones":0,"num_object_copies":5870,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2935,"num_write_kb":11737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2935}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033321+0000","last_change":"2024-09-15T21:20:24.032881+0000","last_active":"2024-09-15T21:20:25.033321+0000","last_peered":"2024-09-15T21:20:25.033321+0000","last_clean":"2024-09-15T21:20:25.033321+0000","last_became_active":"2024-09-15T21:20:24.032040+0000","last_became_peered":"2024-09-15T21:20:24.032040+0000","last_unstale":"2024-09-15T21:20:25.033321+0000","last_undegraded":"2024-09-15T21:20:25.033321+0000","last_fullsized":"2024-09-15T21:20:25.033321+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:46:45.703820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2673","reported_seq":2775,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.958052+0000","last_change":"2024-09-15T21:20:46.455158+0000","last_active":"2024-09-15T21:20:52.958052+0000","last_peered":"2024-09-15T21:20:52.958052+0000","last_clean":"2024-09-15T21:20:52.958052+0000","last_became_active":"2024-09-15T21:20:25.036224+0000","last_became_peered":"2024-09-15T21:20:25.036224+0000","last_unstale":"2024-09-15T21:20:52.958052+0000","last_undegraded":"2024-09-15T21:20:52.958052+0000","last_fullsized":"2024-09-15T21:20:52.958052+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":2673,"log_dups_size":0,"ondisk_log_size":2673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:34.020580+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10920419,"num_objects":2667,"num_object_clones":0,"num_object_copies":5334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":605,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037255+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:20:25.037255+0000","last_peered":"2024-09-15T21:20:25.037255+0000","last_clean":"2024-09-15T21:20:25.037255+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:20:25.037255+0000","last_undegraded":"2024-09-15T21:20:25.037255+0000","last_fullsized":"2024-09-15T21:20:25.037255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2775","reported_seq":2825,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.928593+0000","last_change":"2024-09-15T21:20:24.035959+0000","last_active":"2024-09-15T21:20:54.928593+0000","last_peered":"2024-09-15T21:20:54.928593+0000","last_clean":"2024-09-15T21:19:51.063279+0000","last_became_active":"2024-09-15T21:20:24.035738+0000","last_became_peered":"2024-09-15T21:20:24.035738+0000","last_unstale":"2024-09-15T21:20:54.928593+0000","last_undegraded":"2024-09-15T21:20:24.026174+0000","last_fullsized":"2024-09-15T21:20:54.928593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2775,"log_dups_size":0,"ondisk_log_size":2775,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11334598,"num_objects":2769,"num_object_clones":0,"num_object_copies":5538,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2773,"num_write_kb":11072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2769}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:25.034061+0000","last_change":"2024-09-15T21:20:24.032266+0000","last_active":"2024-09-15T21:20:25.034061+0000","last_peered":"2024-09-15T21:20:25.034061+0000","last_clean":"2024-09-15T21:20:01.890484+0000","last_became_active":"2024-09-15T21:20:24.032032+0000","last_became_peered":"2024-09-15T21:20:24.032032+0000","last_unstale":"2024-09-15T21:20:25.034061+0000","last_undegraded":"2024-09-15T21:20:24.025362+0000","last_fullsized":"2024-09-15T21:20:25.034061+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2674","reported_seq":2765,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.980263+0000","last_change":"2024-09-15T21:20:47.548905+0000","last_active":"2024-09-15T21:20:52.980263+0000","last_peered":"2024-09-15T21:20:52.980263+0000","last_clean":"2024-09-15T21:20:52.980263+0000","last_became_active":"2024-09-15T21:20:25.035785+0000","last_became_peered":"2024-09-15T21:20:25.035785+0000","last_unstale":"2024-09-15T21:20:52.980263+0000","last_undegraded":"2024-09-15T21:20:52.980263+0000","last_fullsized":"2024-09-15T21:20:52.980263+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":2674,"log_dups_size":0,"ondisk_log_size":2674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.975722+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10932707,"num_objects":2670,"num_object_clones":0,"num_object_copies":5340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":2132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.040332+0000","last_change":"2024-09-15T21:20:25.037062+0000","last_active":"2024-09-15T21:20:25.040332+0000","last_peered":"2024-09-15T21:20:25.040332+0000","last_clean":"2024-09-15T21:20:25.040332+0000","last_became_active":"2024-09-15T21:20:25.036685+0000","last_became_peered":"2024-09-15T21:20:25.036685+0000","last_unstale":"2024-09-15T21:20:25.040332+0000","last_undegraded":"2024-09-15T21:20:25.040332+0000","last_fullsized":"2024-09-15T21:20:25.040332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:47:47.150501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2759","reported_seq":2855,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.986168+0000","last_change":"2024-09-15T21:20:48.647971+0000","last_active":"2024-09-15T21:20:52.986168+0000","last_peered":"2024-09-15T21:20:52.986168+0000","last_clean":"2024-09-15T21:20:52.986168+0000","last_became_active":"2024-09-15T21:20:25.035723+0000","last_became_peered":"2024-09-15T21:20:25.035723+0000","last_unstale":"2024-09-15T21:20:52.986168+0000","last_undegraded":"2024-09-15T21:20:52.986168+0000","last_fullsized":"2024-09-15T21:20:52.986168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":2759,"log_dups_size":0,"ondisk_log_size":2759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:03.225931+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11276288,"num_objects":2753,"num_object_clones":0,"num_object_copies":5506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":452,"num_write_kb":1808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2839","reported_seq":2896,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-15T21:20:54.941930+0000","last_change":"2024-09-15T21:20:54.941930+0000","last_active":"2024-09-15T21:20:54.941930+0000","last_peered":"2024-09-15T21:20:54.941930+0000","last_clean":"2024-09-15T21:19:39.745933+0000","last_became_active":"2024-09-15T21:20:24.032201+0000","last_became_peered":"2024-09-15T21:20:24.032201+0000","last_unstale":"2024-09-15T21:20:54.941930+0000","last_undegraded":"2024-09-15T21:20:54.941930+0000","last_fullsized":"2024-09-15T21:20:54.941930+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2839,"log_dups_size":0,"ondisk_log_size":2839,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11620835,"num_objects":2838,"num_object_clones":0,"num_object_copies":5676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2838,"num_write_kb":11349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0,2","objects":2838}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.034081+0000","last_change":"2024-09-15T21:20:24.032468+0000","last_active":"2024-09-15T21:20:25.034081+0000","last_peered":"2024-09-15T21:20:25.034081+0000","last_clean":"2024-09-15T21:20:25.034081+0000","last_became_active":"2024-09-15T21:20:24.032206+0000","last_became_peered":"2024-09-15T21:20:24.032206+0000","last_unstale":"2024-09-15T21:20:25.034081+0000","last_undegraded":"2024-09-15T21:20:25.034081+0000","last_fullsized":"2024-09-15T21:20:25.034081+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:48.044077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2958","reported_seq":3025,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:54.941321+0000","last_change":"2024-09-15T21:20:54.932313+0000","last_active":"2024-09-15T21:20:54.941321+0000","last_peered":"2024-09-15T21:20:54.941321+0000","last_clean":"2024-09-15T21:20:54.941321+0000","last_became_active":"2024-09-15T21:20:24.031529+0000","last_became_peered":"2024-09-15T21:20:24.031529+0000","last_unstale":"2024-09-15T21:20:54.941321+0000","last_undegraded":"2024-09-15T21:20:54.941321+0000","last_fullsized":"2024-09-15T21:20:54.941321+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2958,"log_dups_size":0,"ondisk_log_size":2958,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12084649,"num_objects":2953,"num_object_clones":0,"num_object_copies":5906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2955,"num_write_kb":11804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.036378+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:20:25.036378+0000","last_peered":"2024-09-15T21:20:25.036378+0000","last_clean":"2024-09-15T21:20:25.036378+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:20:25.036378+0000","last_undegraded":"2024-09-15T21:20:25.036378+0000","last_fullsized":"2024-09-15T21:20:25.036378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2865","reported_seq":2920,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.922357+0000","last_change":"2024-09-15T21:20:24.036145+0000","last_active":"2024-09-15T21:20:54.922357+0000","last_peered":"2024-09-15T21:20:54.922357+0000","last_clean":"2024-09-15T21:19:39.745731+0000","last_became_active":"2024-09-15T21:20:24.035699+0000","last_became_peered":"2024-09-15T21:20:24.035699+0000","last_unstale":"2024-09-15T21:20:54.922357+0000","last_undegraded":"2024-09-15T21:20:24.026554+0000","last_fullsized":"2024-09-15T21:20:54.922357+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2865,"log_dups_size":0,"ondisk_log_size":2865,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11719622,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2863,"num_write_kb":11446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033483+0000","last_change":"2024-09-15T21:20:24.031742+0000","last_active":"2024-09-15T21:20:25.033483+0000","last_peered":"2024-09-15T21:20:25.033483+0000","last_clean":"2024-09-15T21:20:25.033483+0000","last_became_active":"2024-09-15T21:20:24.031400+0000","last_became_peered":"2024-09-15T21:20:24.031400+0000","last_unstale":"2024-09-15T21:20:25.033483+0000","last_undegraded":"2024-09-15T21:20:25.033483+0000","last_fullsized":"2024-09-15T21:20:25.033483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:36:52.618460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037955+0000","last_change":"2024-09-15T21:20:25.035926+0000","last_active":"2024-09-15T21:20:25.037955+0000","last_peered":"2024-09-15T21:20:25.037955+0000","last_clean":"2024-09-15T21:20:25.037955+0000","last_became_active":"2024-09-15T21:20:25.035642+0000","last_became_peered":"2024-09-15T21:20:25.035642+0000","last_unstale":"2024-09-15T21:20:25.037955+0000","last_undegraded":"2024-09-15T21:20:25.037955+0000","last_fullsized":"2024-09-15T21:20:25.037955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:40:43.901095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2907","reported_seq":2962,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.913427+0000","last_change":"2024-09-15T21:20:24.035912+0000","last_active":"2024-09-15T21:20:54.913427+0000","last_peered":"2024-09-15T21:20:54.913427+0000","last_clean":"2024-09-15T21:19:50.713871+0000","last_became_active":"2024-09-15T21:20:24.035656+0000","last_became_peered":"2024-09-15T21:20:24.035656+0000","last_unstale":"2024-09-15T21:20:54.913427+0000","last_undegraded":"2024-09-15T21:20:24.026450+0000","last_fullsized":"2024-09-15T21:20:54.913427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2907,"log_dups_size":0,"ondisk_log_size":2907,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11859852,"num_objects":2899,"num_object_clones":0,"num_object_copies":5798,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2904,"num_write_kb":11587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2897}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":184105829,"num_objects":44968,"num_object_clones":0,"num_object_copies":89936,"num_objects_missing_on_primary":14,"num_objects_missing":14,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":44968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34874,"num_write_kb":139382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":365416448,"data_stored":365264702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45027,"ondisk_log_size":45027,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287916,"kb_used_data":171548,"kb_used_omap":4,"kb_used_meta":116347,"kb_avail":94083924,"statfs":{"total":96636764160,"available":96341938176,"internally_reserved":0,"allocated":175665152,"data_stored":175455161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":119140009},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,5,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182416,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":296428,"kb_used_data":184284,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94075412,"statfs":{"total":96636764160,"available":96333221888,"internally_reserved":0,"allocated":188706816,"data_stored":188471070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":114814635},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,3,4,2,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":188956672,"data_stored":188920542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":176345088,"data_stored":176330636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:57.148 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:20:57.690 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:20:57.690 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:20:57.709 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-15T21:20:56.686687+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":184105848,"num_objects":44970,"num_object_clones":0,"num_object_copies":89940,"num_objects_missing_on_primary":14,"num_objects_missing":14,"num_objects_degraded":30,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":44970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34876,"num_write_kb":139384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45030,"ondisk_log_size":45030,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":584344,"kb_used_data":355832,"kb_used_omap":8,"kb_used_meta":228471,"kb_avail":188159336,"statfs":{"total":193273528320,"available":192675160064,"internally_reserved":0,"allocated":364371968,"data_stored":363926231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":233954644},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[11,3,4,7,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":86967953,"num_objects":21228,"num_object_clones":0,"num_object_copies":42456,"num_objects_missing_on_primary":-11,"num_objects_missing":-11,"num_objects_degraded":-16,"num_objects_misplaced":0,"num_objects_unfound":-20,"num_objects_dirty":21228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11107,"num_write_kb":44488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":-966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002582"},"pg_stats":[{"pgid":"2.f","version":"36'2883","reported_seq":2935,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:54.931734+0000","last_change":"2024-09-15T21:20:24.031714+0000","last_active":"2024-09-15T21:20:54.931734+0000","last_peered":"2024-09-15T21:20:54.931734+0000","last_clean":"2024-09-15T21:20:54.931734+0000","last_became_active":"2024-09-15T21:20:24.031439+0000","last_became_peered":"2024-09-15T21:20:24.031439+0000","last_unstale":"2024-09-15T21:20:54.931734+0000","last_undegraded":"2024-09-15T21:20:54.931734+0000","last_fullsized":"2024-09-15T21:20:54.931734+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2883,"log_dups_size":0,"ondisk_log_size":2883,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:51.235292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11808768,"num_objects":2883,"num_object_clones":0,"num_object_copies":5766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2883,"num_write_kb":11532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2649","reported_seq":2733,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.994035+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:20:52.994035+0000","last_peered":"2024-09-15T21:20:52.994035+0000","last_clean":"2024-09-15T21:20:52.994035+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:20:52.994035+0000","last_undegraded":"2024-09-15T21:20:52.994035+0000","last_fullsized":"2024-09-15T21:20:52.994035+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":2649,"log_dups_size":0,"ondisk_log_size":2649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10826694,"num_objects":2645,"num_object_clones":0,"num_object_copies":5290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":930,"num_write_kb":3720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2807","reported_seq":2905,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.990305+0000","last_change":"2024-09-15T21:20:44.532177+0000","last_active":"2024-09-15T21:20:52.990305+0000","last_peered":"2024-09-15T21:20:52.990305+0000","last_clean":"2024-09-15T21:20:52.990305+0000","last_became_active":"2024-09-15T21:20:25.036309+0000","last_became_peered":"2024-09-15T21:20:25.036309+0000","last_unstale":"2024-09-15T21:20:52.990305+0000","last_undegraded":"2024-09-15T21:20:52.990305+0000","last_fullsized":"2024-09-15T21:20:52.990305+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":2807,"log_dups_size":0,"ondisk_log_size":2807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:41.470643+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11481088,"num_objects":2803,"num_object_clones":0,"num_object_copies":5606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":907,"num_write_kb":3628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2650","reported_seq":2701,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.985504+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:20:52.985504+0000","last_peered":"2024-09-15T21:20:52.985504+0000","last_clean":"2024-09-15T21:20:52.985504+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:20:52.985504+0000","last_undegraded":"2024-09-15T21:20:52.985504+0000","last_fullsized":"2024-09-15T21:20:52.985504+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2650,"log_dups_size":0,"ondisk_log_size":2650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10854400,"num_objects":2650,"num_object_clones":0,"num_object_copies":5300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2650,"num_write_kb":10600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2832","reported_seq":2890,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.940659+0000","last_change":"2024-09-15T21:20:24.036146+0000","last_active":"2024-09-15T21:20:54.940659+0000","last_peered":"2024-09-15T21:20:54.940659+0000","last_clean":"2024-09-15T21:19:50.164448+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:20:54.940659+0000","last_undegraded":"2024-09-15T21:20:24.028206+0000","last_fullsized":"2024-09-15T21:20:54.940659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2832,"log_dups_size":0,"ondisk_log_size":2832,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11572166,"num_objects":2827,"num_object_clones":0,"num_object_copies":5654,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2830,"num_write_kb":11304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2826}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2870","reported_seq":2926,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.942764+0000","last_change":"2024-09-15T21:20:24.035954+0000","last_active":"2024-09-15T21:20:54.942764+0000","last_peered":"2024-09-15T21:20:54.942764+0000","last_clean":"2024-09-15T21:19:51.013140+0000","last_became_active":"2024-09-15T21:20:24.035700+0000","last_became_peered":"2024-09-15T21:20:24.035700+0000","last_unstale":"2024-09-15T21:20:54.942764+0000","last_undegraded":"2024-09-15T21:20:24.025973+0000","last_fullsized":"2024-09-15T21:20:54.942764+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2870,"log_dups_size":0,"ondisk_log_size":2870,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11736006,"num_objects":2867,"num_object_clones":0,"num_object_copies":5734,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2868,"num_write_kb":11463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0,2","objects":2866}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2950","reported_seq":3006,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.929058+0000","last_change":"2024-09-15T21:20:24.036588+0000","last_active":"2024-09-15T21:20:54.929058+0000","last_peered":"2024-09-15T21:20:54.929058+0000","last_clean":"2024-09-15T21:19:50.413489+0000","last_became_active":"2024-09-15T21:20:24.036417+0000","last_became_peered":"2024-09-15T21:20:24.036417+0000","last_unstale":"2024-09-15T21:20:54.929058+0000","last_undegraded":"2024-09-15T21:20:24.028324+0000","last_fullsized":"2024-09-15T21:20:54.929058+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2950,"log_dups_size":0,"ondisk_log_size":2950,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12059590,"num_objects":2946,"num_object_clones":0,"num_object_copies":5892,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2948,"num_write_kb":11780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":2},{"shards":"0,2","objects":2944}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2936","reported_seq":2995,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.937586+0000","last_change":"2024-09-15T21:20:24.032808+0000","last_active":"2024-09-15T21:20:54.937586+0000","last_peered":"2024-09-15T21:20:54.937586+0000","last_clean":"2024-09-15T21:19:36.740313+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:20:54.937586+0000","last_undegraded":"2024-09-15T21:20:24.026851+0000","last_fullsized":"2024-09-15T21:20:54.937586+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2936,"log_dups_size":0,"ondisk_log_size":2936,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12018147,"num_objects":2935,"num_object_clones":0,"num_object_copies":5870,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2935,"num_write_kb":11737,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2935}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033321+0000","last_change":"2024-09-15T21:20:24.032881+0000","last_active":"2024-09-15T21:20:25.033321+0000","last_peered":"2024-09-15T21:20:25.033321+0000","last_clean":"2024-09-15T21:20:25.033321+0000","last_became_active":"2024-09-15T21:20:24.032040+0000","last_became_peered":"2024-09-15T21:20:24.032040+0000","last_unstale":"2024-09-15T21:20:25.033321+0000","last_undegraded":"2024-09-15T21:20:25.033321+0000","last_fullsized":"2024-09-15T21:20:25.033321+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:46:45.703820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2673","reported_seq":2775,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.958052+0000","last_change":"2024-09-15T21:20:46.455158+0000","last_active":"2024-09-15T21:20:52.958052+0000","last_peered":"2024-09-15T21:20:52.958052+0000","last_clean":"2024-09-15T21:20:52.958052+0000","last_became_active":"2024-09-15T21:20:25.036224+0000","last_became_peered":"2024-09-15T21:20:25.036224+0000","last_unstale":"2024-09-15T21:20:52.958052+0000","last_undegraded":"2024-09-15T21:20:52.958052+0000","last_fullsized":"2024-09-15T21:20:52.958052+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":2673,"log_dups_size":0,"ondisk_log_size":2673,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:34.020580+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10920419,"num_objects":2667,"num_object_clones":0,"num_object_copies":5334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":605,"num_write_kb":2420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037255+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:20:25.037255+0000","last_peered":"2024-09-15T21:20:25.037255+0000","last_clean":"2024-09-15T21:20:25.037255+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:20:25.037255+0000","last_undegraded":"2024-09-15T21:20:25.037255+0000","last_fullsized":"2024-09-15T21:20:25.037255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2775","reported_seq":2825,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.928593+0000","last_change":"2024-09-15T21:20:24.035959+0000","last_active":"2024-09-15T21:20:54.928593+0000","last_peered":"2024-09-15T21:20:54.928593+0000","last_clean":"2024-09-15T21:19:51.063279+0000","last_became_active":"2024-09-15T21:20:24.035738+0000","last_became_peered":"2024-09-15T21:20:24.035738+0000","last_unstale":"2024-09-15T21:20:54.928593+0000","last_undegraded":"2024-09-15T21:20:24.026174+0000","last_fullsized":"2024-09-15T21:20:54.928593+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2775,"log_dups_size":0,"ondisk_log_size":2775,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11334598,"num_objects":2769,"num_object_clones":0,"num_object_copies":5538,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2773,"num_write_kb":11072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2769}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:25.034061+0000","last_change":"2024-09-15T21:20:24.032266+0000","last_active":"2024-09-15T21:20:25.034061+0000","last_peered":"2024-09-15T21:20:25.034061+0000","last_clean":"2024-09-15T21:20:01.890484+0000","last_became_active":"2024-09-15T21:20:24.032032+0000","last_became_peered":"2024-09-15T21:20:24.032032+0000","last_unstale":"2024-09-15T21:20:25.034061+0000","last_undegraded":"2024-09-15T21:20:24.025362+0000","last_fullsized":"2024-09-15T21:20:25.034061+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2674","reported_seq":2765,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.980263+0000","last_change":"2024-09-15T21:20:47.548905+0000","last_active":"2024-09-15T21:20:52.980263+0000","last_peered":"2024-09-15T21:20:52.980263+0000","last_clean":"2024-09-15T21:20:52.980263+0000","last_became_active":"2024-09-15T21:20:25.035785+0000","last_became_peered":"2024-09-15T21:20:25.035785+0000","last_unstale":"2024-09-15T21:20:52.980263+0000","last_undegraded":"2024-09-15T21:20:52.980263+0000","last_fullsized":"2024-09-15T21:20:52.980263+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":2674,"log_dups_size":0,"ondisk_log_size":2674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.975722+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10932707,"num_objects":2670,"num_object_clones":0,"num_object_copies":5340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":533,"num_write_kb":2132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.040332+0000","last_change":"2024-09-15T21:20:25.037062+0000","last_active":"2024-09-15T21:20:25.040332+0000","last_peered":"2024-09-15T21:20:25.040332+0000","last_clean":"2024-09-15T21:20:25.040332+0000","last_became_active":"2024-09-15T21:20:25.036685+0000","last_became_peered":"2024-09-15T21:20:25.036685+0000","last_unstale":"2024-09-15T21:20:25.040332+0000","last_undegraded":"2024-09-15T21:20:25.040332+0000","last_fullsized":"2024-09-15T21:20:25.040332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:47:47.150501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2759","reported_seq":2855,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:52.986168+0000","last_change":"2024-09-15T21:20:48.647971+0000","last_active":"2024-09-15T21:20:52.986168+0000","last_peered":"2024-09-15T21:20:52.986168+0000","last_clean":"2024-09-15T21:20:52.986168+0000","last_became_active":"2024-09-15T21:20:25.035723+0000","last_became_peered":"2024-09-15T21:20:25.035723+0000","last_unstale":"2024-09-15T21:20:52.986168+0000","last_undegraded":"2024-09-15T21:20:52.986168+0000","last_fullsized":"2024-09-15T21:20:52.986168+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":2759,"log_dups_size":0,"ondisk_log_size":2759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:03.225931+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11276288,"num_objects":2753,"num_object_clones":0,"num_object_copies":5506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":452,"num_write_kb":1808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2839","reported_seq":2896,"reported_epoch":36,"state":"active+recovering","last_fresh":"2024-09-15T21:20:54.941930+0000","last_change":"2024-09-15T21:20:54.941930+0000","last_active":"2024-09-15T21:20:54.941930+0000","last_peered":"2024-09-15T21:20:54.941930+0000","last_clean":"2024-09-15T21:19:39.745933+0000","last_became_active":"2024-09-15T21:20:24.032201+0000","last_became_peered":"2024-09-15T21:20:24.032201+0000","last_unstale":"2024-09-15T21:20:54.941930+0000","last_undegraded":"2024-09-15T21:20:54.941930+0000","last_fullsized":"2024-09-15T21:20:54.941930+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2839,"log_dups_size":0,"ondisk_log_size":2839,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11620835,"num_objects":2838,"num_object_clones":0,"num_object_copies":5676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2838,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2838,"num_write_kb":11349,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0,2","objects":2838}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.034081+0000","last_change":"2024-09-15T21:20:24.032468+0000","last_active":"2024-09-15T21:20:25.034081+0000","last_peered":"2024-09-15T21:20:25.034081+0000","last_clean":"2024-09-15T21:20:25.034081+0000","last_became_active":"2024-09-15T21:20:24.032206+0000","last_became_peered":"2024-09-15T21:20:24.032206+0000","last_unstale":"2024-09-15T21:20:25.034081+0000","last_undegraded":"2024-09-15T21:20:25.034081+0000","last_fullsized":"2024-09-15T21:20:25.034081+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:48.044077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2958","reported_seq":3025,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:54.941321+0000","last_change":"2024-09-15T21:20:54.932313+0000","last_active":"2024-09-15T21:20:54.941321+0000","last_peered":"2024-09-15T21:20:54.941321+0000","last_clean":"2024-09-15T21:20:54.941321+0000","last_became_active":"2024-09-15T21:20:24.031529+0000","last_became_peered":"2024-09-15T21:20:24.031529+0000","last_unstale":"2024-09-15T21:20:54.941321+0000","last_undegraded":"2024-09-15T21:20:54.941321+0000","last_fullsized":"2024-09-15T21:20:54.941321+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2958,"log_dups_size":0,"ondisk_log_size":2958,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12084649,"num_objects":2953,"num_object_clones":0,"num_object_copies":5906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2955,"num_write_kb":11804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.036378+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:20:25.036378+0000","last_peered":"2024-09-15T21:20:25.036378+0000","last_clean":"2024-09-15T21:20:25.036378+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:20:25.036378+0000","last_undegraded":"2024-09-15T21:20:25.036378+0000","last_fullsized":"2024-09-15T21:20:25.036378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2865","reported_seq":2920,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.922357+0000","last_change":"2024-09-15T21:20:24.036145+0000","last_active":"2024-09-15T21:20:54.922357+0000","last_peered":"2024-09-15T21:20:54.922357+0000","last_clean":"2024-09-15T21:19:39.745731+0000","last_became_active":"2024-09-15T21:20:24.035699+0000","last_became_peered":"2024-09-15T21:20:24.035699+0000","last_unstale":"2024-09-15T21:20:54.922357+0000","last_undegraded":"2024-09-15T21:20:24.026554+0000","last_fullsized":"2024-09-15T21:20:54.922357+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2865,"log_dups_size":0,"ondisk_log_size":2865,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11719622,"num_objects":2863,"num_object_clones":0,"num_object_copies":5726,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2863,"num_write_kb":11446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2863}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033483+0000","last_change":"2024-09-15T21:20:24.031742+0000","last_active":"2024-09-15T21:20:25.033483+0000","last_peered":"2024-09-15T21:20:25.033483+0000","last_clean":"2024-09-15T21:20:25.033483+0000","last_became_active":"2024-09-15T21:20:24.031400+0000","last_became_peered":"2024-09-15T21:20:24.031400+0000","last_unstale":"2024-09-15T21:20:25.033483+0000","last_undegraded":"2024-09-15T21:20:25.033483+0000","last_fullsized":"2024-09-15T21:20:25.033483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:36:52.618460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037955+0000","last_change":"2024-09-15T21:20:25.035926+0000","last_active":"2024-09-15T21:20:25.037955+0000","last_peered":"2024-09-15T21:20:25.037955+0000","last_clean":"2024-09-15T21:20:25.037955+0000","last_became_active":"2024-09-15T21:20:25.035642+0000","last_became_peered":"2024-09-15T21:20:25.035642+0000","last_unstale":"2024-09-15T21:20:25.037955+0000","last_undegraded":"2024-09-15T21:20:25.037955+0000","last_fullsized":"2024-09-15T21:20:25.037955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:40:43.901095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2907","reported_seq":2962,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-15T21:20:54.913427+0000","last_change":"2024-09-15T21:20:24.035912+0000","last_active":"2024-09-15T21:20:54.913427+0000","last_peered":"2024-09-15T21:20:54.913427+0000","last_clean":"2024-09-15T21:19:50.713871+0000","last_became_active":"2024-09-15T21:20:24.035656+0000","last_became_peered":"2024-09-15T21:20:24.035656+0000","last_unstale":"2024-09-15T21:20:54.913427+0000","last_undegraded":"2024-09-15T21:20:24.026450+0000","last_fullsized":"2024-09-15T21:20:54.913427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":2907,"log_dups_size":0,"ondisk_log_size":2907,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11859852,"num_objects":2899,"num_object_clones":0,"num_object_copies":5798,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":2899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2904,"num_write_kb":11587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"","objects":1},{"shards":"0","objects":1},{"shards":"0,2","objects":2897}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":184105829,"num_objects":44968,"num_object_clones":0,"num_object_copies":89936,"num_objects_missing_on_primary":14,"num_objects_missing":14,"num_objects_degraded":29,"num_objects_misplaced":0,"num_objects_unfound":5,"num_objects_dirty":44968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34874,"num_write_kb":139382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":365416448,"data_stored":365264702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45027,"ondisk_log_size":45027,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":287916,"kb_used_data":171548,"kb_used_omap":4,"kb_used_meta":116347,"kb_avail":94083924,"statfs":{"total":96636764160,"available":96341938176,"internally_reserved":0,"allocated":175665152,"data_stored":175455161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":119140009},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,5,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182416,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":296428,"kb_used_data":184284,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94075412,"statfs":{"total":96636764160,"available":96333221888,"internally_reserved":0,"allocated":188706816,"data_stored":188471070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":114814635},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,3,4,2,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":188956672,"data_stored":188920542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":176345088,"data_stored":176330636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:20:57.735 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 26 20 50082 50062 7.52036 4.89453 0.0126695 0.0103821 2024-09-15T21:20:58.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 27 20 51463 51443 7.44161 5.39453 0.0109572 0.010493 2024-09-15T21:20:59.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 28 20 52848 52828 7.36904 5.41016 0.0114282 0.0105965 2024-09-15T21:21:00.711 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:00.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 29 20 54200 54180 7.29704 5.28125 0.0195398 0.0107003 2024-09-15T21:21:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:01.240 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:01.263 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-09-15T21:21:00.687578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":213790275,"num_objects":52211,"num_object_clones":0,"num_object_copies":104422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29461,"num_write_kb":117775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1951,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52278,"ondisk_log_size":52278,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":647088,"kb_used_data":413004,"kb_used_omap":8,"kb_used_meta":234039,"kb_avail":188096592,"statfs":{"total":193273528320,"available":192610910208,"internally_reserved":0,"allocated":422916096,"data_stored":422444604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8876,"internal_metadata":239656276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,4,6,8,9,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":80633605,"num_objects":19678,"num_object_clones":0,"num_object_copies":39356,"num_objects_missing_on_primary":-18,"num_objects_missing":-18,"num_objects_degraded":-32,"num_objects_misplaced":0,"num_objects_unfound":-16,"num_objects_dirty":19678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4719,"num_write_kb":18940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":1951,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002498"},"pg_stats":[{"pgid":"2.f","version":"36'3339","reported_seq":3391,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:59.968084+0000","last_change":"2024-09-15T21:20:24.031714+0000","last_active":"2024-09-15T21:20:59.968084+0000","last_peered":"2024-09-15T21:20:59.968084+0000","last_clean":"2024-09-15T21:20:59.968084+0000","last_became_active":"2024-09-15T21:20:24.031439+0000","last_became_peered":"2024-09-15T21:20:24.031439+0000","last_unstale":"2024-09-15T21:20:59.968084+0000","last_undegraded":"2024-09-15T21:20:59.968084+0000","last_fullsized":"2024-09-15T21:20:59.968084+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3339,"log_dups_size":0,"ondisk_log_size":3339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:06:51.235292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13676544,"num_objects":3339,"num_object_clones":0,"num_object_copies":6678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3339,"num_write_kb":13356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'3126","reported_seq":3210,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:58.000719+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:20:58.000719+0000","last_peered":"2024-09-15T21:20:58.000719+0000","last_clean":"2024-09-15T21:20:58.000719+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:20:58.000719+0000","last_undegraded":"2024-09-15T21:20:58.000719+0000","last_fullsized":"2024-09-15T21:20:58.000719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":3126,"log_dups_size":0,"ondisk_log_size":3126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12780486,"num_objects":3122,"num_object_clones":0,"num_object_copies":6244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1407,"num_write_kb":5628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'3278","reported_seq":3376,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:57.989655+0000","last_change":"2024-09-15T21:20:44.532177+0000","last_active":"2024-09-15T21:20:57.989655+0000","last_peered":"2024-09-15T21:20:57.989655+0000","last_clean":"2024-09-15T21:20:57.989655+0000","last_became_active":"2024-09-15T21:20:25.036309+0000","last_became_peered":"2024-09-15T21:20:25.036309+0000","last_unstale":"2024-09-15T21:20:57.989655+0000","last_undegraded":"2024-09-15T21:20:57.989655+0000","last_fullsized":"2024-09-15T21:20:57.989655+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":3278,"log_dups_size":0,"ondisk_log_size":3278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:39:41.470643+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13410304,"num_objects":3274,"num_object_clones":0,"num_object_copies":6548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1378,"num_write_kb":5512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'3081","reported_seq":3131,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:58.011751+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:20:58.011751+0000","last_peered":"2024-09-15T21:20:58.011751+0000","last_clean":"2024-09-15T21:20:58.011751+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:20:58.011751+0000","last_undegraded":"2024-09-15T21:20:58.011751+0000","last_fullsized":"2024-09-15T21:20:58.011751+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3081,"log_dups_size":0,"ondisk_log_size":3081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12619776,"num_objects":3081,"num_object_clones":0,"num_object_copies":6162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3081,"num_write_kb":12324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'3293","reported_seq":3360,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:59.982730+0000","last_change":"2024-09-15T21:20:55.276763+0000","last_active":"2024-09-15T21:20:59.982730+0000","last_peered":"2024-09-15T21:20:59.982730+0000","last_clean":"2024-09-15T21:20:59.982730+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:20:59.982730+0000","last_undegraded":"2024-09-15T21:20:59.982730+0000","last_fullsized":"2024-09-15T21:20:59.982730+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3293,"log_dups_size":0,"ondisk_log_size":3293,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13460422,"num_objects":3288,"num_object_clones":0,"num_object_copies":6576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3291,"num_write_kb":13148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'3308","reported_seq":3384,"reported_epoch":36,"state":"active+clean+scrubbing","last_fresh":"2024-09-15T21:20:59.971527+0000","last_change":"2024-09-15T21:20:59.760380+0000","last_active":"2024-09-15T21:20:59.971527+0000","last_peered":"2024-09-15T21:20:59.971527+0000","last_clean":"2024-09-15T21:20:59.971527+0000","last_became_active":"2024-09-15T21:20:24.035700+0000","last_became_peered":"2024-09-15T21:20:24.035700+0000","last_unstale":"2024-09-15T21:20:59.971527+0000","last_undegraded":"2024-09-15T21:20:59.971527+0000","last_fullsized":"2024-09-15T21:20:59.971527+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":699,"log_size":3308,"log_dups_size":0,"ondisk_log_size":3308,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13530054,"num_objects":3305,"num_object_clones":0,"num_object_copies":6610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3306,"num_write_kb":13215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'3367","reported_seq":3432,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:59.953389+0000","last_change":"2024-09-15T21:20:55.317152+0000","last_active":"2024-09-15T21:20:59.953389+0000","last_peered":"2024-09-15T21:20:59.953389+0000","last_clean":"2024-09-15T21:20:59.953389+0000","last_became_active":"2024-09-15T21:20:24.036417+0000","last_became_peered":"2024-09-15T21:20:24.036417+0000","last_unstale":"2024-09-15T21:20:59.953389+0000","last_undegraded":"2024-09-15T21:20:59.953389+0000","last_fullsized":"2024-09-15T21:20:59.953389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3367,"log_dups_size":0,"ondisk_log_size":3367,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13767622,"num_objects":3363,"num_object_clones":0,"num_object_copies":6726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3365,"num_write_kb":13448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'3375","reported_seq":3484,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:59.940275+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:20:59.940275+0000","last_peered":"2024-09-15T21:20:59.940275+0000","last_clean":"2024-09-15T21:20:59.940275+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:20:59.940275+0000","last_undegraded":"2024-09-15T21:20:59.940275+0000","last_fullsized":"2024-09-15T21:20:59.940275+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":3375,"log_dups_size":0,"ondisk_log_size":3375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13815808,"num_objects":3373,"num_object_clones":0,"num_object_copies":6746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":147,"num_write_kb":588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033321+0000","last_change":"2024-09-15T21:20:24.032881+0000","last_active":"2024-09-15T21:20:25.033321+0000","last_peered":"2024-09-15T21:20:25.033321+0000","last_clean":"2024-09-15T21:20:25.033321+0000","last_became_active":"2024-09-15T21:20:24.032040+0000","last_became_peered":"2024-09-15T21:20:24.032040+0000","last_unstale":"2024-09-15T21:20:25.033321+0000","last_undegraded":"2024-09-15T21:20:25.033321+0000","last_fullsized":"2024-09-15T21:20:25.033321+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:46:45.703820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'3137","reported_seq":3239,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:58.005146+0000","last_change":"2024-09-15T21:20:46.455158+0000","last_active":"2024-09-15T21:20:58.005146+0000","last_peered":"2024-09-15T21:20:58.005146+0000","last_clean":"2024-09-15T21:20:58.005146+0000","last_became_active":"2024-09-15T21:20:25.036224+0000","last_became_peered":"2024-09-15T21:20:25.036224+0000","last_unstale":"2024-09-15T21:20:58.005146+0000","last_undegraded":"2024-09-15T21:20:58.005146+0000","last_fullsized":"2024-09-15T21:20:58.005146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":3137,"log_dups_size":0,"ondisk_log_size":3137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:22:34.020580+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12820963,"num_objects":3131,"num_object_clones":0,"num_object_copies":6262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1069,"num_write_kb":4276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037255+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:20:25.037255+0000","last_peered":"2024-09-15T21:20:25.037255+0000","last_clean":"2024-09-15T21:20:25.037255+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:20:25.037255+0000","last_undegraded":"2024-09-15T21:20:25.037255+0000","last_fullsized":"2024-09-15T21:20:25.037255+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'3231","reported_seq":3292,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:59.971434+0000","last_change":"2024-09-15T21:20:55.154428+0000","last_active":"2024-09-15T21:20:59.971434+0000","last_peered":"2024-09-15T21:20:59.971434+0000","last_clean":"2024-09-15T21:20:59.971434+0000","last_became_active":"2024-09-15T21:20:24.035738+0000","last_became_peered":"2024-09-15T21:20:24.035738+0000","last_unstale":"2024-09-15T21:20:59.971434+0000","last_undegraded":"2024-09-15T21:20:59.971434+0000","last_fullsized":"2024-09-15T21:20:59.971434+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3231,"log_dups_size":0,"ondisk_log_size":3231,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13202374,"num_objects":3225,"num_object_clones":0,"num_object_copies":6450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3229,"num_write_kb":12896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:55.326846+0000","last_change":"2024-09-15T21:20:55.326763+0000","last_active":"2024-09-15T21:20:55.326846+0000","last_peered":"2024-09-15T21:20:55.326846+0000","last_clean":"2024-09-15T21:20:55.326846+0000","last_became_active":"2024-09-15T21:20:24.032032+0000","last_became_peered":"2024-09-15T21:20:24.032032+0000","last_unstale":"2024-09-15T21:20:55.326846+0000","last_undegraded":"2024-09-15T21:20:55.326846+0000","last_fullsized":"2024-09-15T21:20:55.326846+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:58:20.566654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'3149","reported_seq":3238,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:58.007545+0000","last_change":"2024-09-15T21:20:47.548905+0000","last_active":"2024-09-15T21:20:58.007545+0000","last_peered":"2024-09-15T21:20:58.007545+0000","last_clean":"2024-09-15T21:20:58.007545+0000","last_became_active":"2024-09-15T21:20:25.035785+0000","last_became_peered":"2024-09-15T21:20:25.035785+0000","last_unstale":"2024-09-15T21:20:58.007545+0000","last_undegraded":"2024-09-15T21:20:58.007545+0000","last_fullsized":"2024-09-15T21:20:58.007545+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":3149,"log_dups_size":0,"ondisk_log_size":3149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:01:31.975722+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12878307,"num_objects":3145,"num_object_clones":0,"num_object_copies":6290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1008,"num_write_kb":4032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":19,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.040332+0000","last_change":"2024-09-15T21:20:25.037062+0000","last_active":"2024-09-15T21:20:25.040332+0000","last_peered":"2024-09-15T21:20:25.040332+0000","last_clean":"2024-09-15T21:20:25.040332+0000","last_became_active":"2024-09-15T21:20:25.036685+0000","last_became_peered":"2024-09-15T21:20:25.036685+0000","last_unstale":"2024-09-15T21:20:25.040332+0000","last_undegraded":"2024-09-15T21:20:25.040332+0000","last_fullsized":"2024-09-15T21:20:25.040332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:47:47.150501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'3213","reported_seq":3309,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:58.005535+0000","last_change":"2024-09-15T21:20:48.647971+0000","last_active":"2024-09-15T21:20:58.005535+0000","last_peered":"2024-09-15T21:20:58.005535+0000","last_clean":"2024-09-15T21:20:58.005535+0000","last_became_active":"2024-09-15T21:20:25.035723+0000","last_became_peered":"2024-09-15T21:20:25.035723+0000","last_unstale":"2024-09-15T21:20:58.005535+0000","last_undegraded":"2024-09-15T21:20:58.005535+0000","last_fullsized":"2024-09-15T21:20:58.005535+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":3213,"log_dups_size":0,"ondisk_log_size":3213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:42:03.225931+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13135872,"num_objects":3207,"num_object_clones":0,"num_object_copies":6414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":906,"num_write_kb":3624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'3283","reported_seq":3386,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:59.963701+0000","last_change":"2024-09-15T21:20:57.978233+0000","last_active":"2024-09-15T21:20:59.963701+0000","last_peered":"2024-09-15T21:20:59.963701+0000","last_clean":"2024-09-15T21:20:59.963701+0000","last_became_active":"2024-09-15T21:20:24.032201+0000","last_became_peered":"2024-09-15T21:20:24.032201+0000","last_unstale":"2024-09-15T21:20:59.963701+0000","last_undegraded":"2024-09-15T21:20:59.963701+0000","last_fullsized":"2024-09-15T21:20:59.963701+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":3283,"log_dups_size":0,"ondisk_log_size":3283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:25:31.576151+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13438976,"num_objects":3281,"num_object_clones":0,"num_object_copies":6562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3281,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":227,"num_write_kb":908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.034081+0000","last_change":"2024-09-15T21:20:24.032468+0000","last_active":"2024-09-15T21:20:25.034081+0000","last_peered":"2024-09-15T21:20:25.034081+0000","last_clean":"2024-09-15T21:20:25.034081+0000","last_became_active":"2024-09-15T21:20:24.032206+0000","last_became_peered":"2024-09-15T21:20:24.032206+0000","last_unstale":"2024-09-15T21:20:25.034081+0000","last_undegraded":"2024-09-15T21:20:25.034081+0000","last_fullsized":"2024-09-15T21:20:25.034081+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:00:48.044077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'3407","reported_seq":3514,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:59.963921+0000","last_change":"2024-09-15T21:20:56.700342+0000","last_active":"2024-09-15T21:20:59.963921+0000","last_peered":"2024-09-15T21:20:59.963921+0000","last_clean":"2024-09-15T21:20:59.963921+0000","last_became_active":"2024-09-15T21:20:24.031529+0000","last_became_peered":"2024-09-15T21:20:24.031529+0000","last_unstale":"2024-09-15T21:20:59.963921+0000","last_undegraded":"2024-09-15T21:20:59.963921+0000","last_fullsized":"2024-09-15T21:20:59.963921+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":3407,"log_dups_size":0,"ondisk_log_size":3407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:13:02.799500+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13922304,"num_objects":3399,"num_object_clones":0,"num_object_copies":6798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":316,"num_write_kb":1264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.036378+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:20:25.036378+0000","last_peered":"2024-09-15T21:20:25.036378+0000","last_clean":"2024-09-15T21:20:25.036378+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:20:25.036378+0000","last_undegraded":"2024-09-15T21:20:25.036378+0000","last_fullsized":"2024-09-15T21:20:25.036378+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'3340","reported_seq":3406,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:59.965269+0000","last_change":"2024-09-15T21:20:55.224574+0000","last_active":"2024-09-15T21:20:59.965269+0000","last_peered":"2024-09-15T21:20:59.965269+0000","last_clean":"2024-09-15T21:20:59.965269+0000","last_became_active":"2024-09-15T21:20:24.035699+0000","last_became_peered":"2024-09-15T21:20:24.035699+0000","last_unstale":"2024-09-15T21:20:59.965269+0000","last_undegraded":"2024-09-15T21:20:59.965269+0000","last_fullsized":"2024-09-15T21:20:59.965269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":3340,"log_dups_size":0,"ondisk_log_size":3340,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13665222,"num_objects":3338,"num_object_clones":0,"num_object_copies":6676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3338,"num_write_kb":13346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.033483+0000","last_change":"2024-09-15T21:20:24.031742+0000","last_active":"2024-09-15T21:20:25.033483+0000","last_peered":"2024-09-15T21:20:25.033483+0000","last_clean":"2024-09-15T21:20:25.033483+0000","last_became_active":"2024-09-15T21:20:24.031400+0000","last_became_peered":"2024-09-15T21:20:24.031400+0000","last_unstale":"2024-09-15T21:20:25.033483+0000","last_undegraded":"2024-09-15T21:20:25.033483+0000","last_fullsized":"2024-09-15T21:20:25.033483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:36:52.618460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:25.037955+0000","last_change":"2024-09-15T21:20:25.035926+0000","last_active":"2024-09-15T21:20:25.037955+0000","last_peered":"2024-09-15T21:20:25.037955+0000","last_clean":"2024-09-15T21:20:25.037955+0000","last_became_active":"2024-09-15T21:20:25.035642+0000","last_became_peered":"2024-09-15T21:20:25.035642+0000","last_unstale":"2024-09-15T21:20:25.037955+0000","last_undegraded":"2024-09-15T21:20:25.037955+0000","last_fullsized":"2024-09-15T21:20:25.037955+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:40:43.901095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'3348","reported_seq":3459,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-15T21:20:59.963957+0000","last_change":"2024-09-15T21:20:59.798098+0000","last_active":"2024-09-15T21:20:59.963957+0000","last_peered":"2024-09-15T21:20:59.963957+0000","last_clean":"2024-09-15T21:20:59.963957+0000","last_became_active":"2024-09-15T21:20:24.035656+0000","last_became_peered":"2024-09-15T21:20:24.035656+0000","last_unstale":"2024-09-15T21:20:59.963957+0000","last_undegraded":"2024-09-15T21:20:59.963957+0000","last_fullsized":"2024-09-15T21:20:59.963957+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":3348,"log_dups_size":0,"ondisk_log_size":3348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:25:47.029702+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13665222,"num_objects":3338,"num_object_clones":0,"num_object_copies":6676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52,"num_write_kb":208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":213790256,"num_objects":52209,"num_object_clones":0,"num_object_copies":104418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29459,"num_write_kb":117773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1932,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":424779776,"data_stored":424606352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52275,"ondisk_log_size":52275,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986188,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309200,"kb_used_data":201152,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94062640,"statfs":{"total":96636764160,"available":96320143360,"internally_reserved":0,"allocated":205979648,"data_stored":205743902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":110620329},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,5,4,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182417,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":337888,"kb_used_data":211852,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94033952,"statfs":{"total":96636764160,"available":96290766848,"internally_reserved":0,"allocated":216936448,"data_stored":216700702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":129035947},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,1,4,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":217890816,"data_stored":217854686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":206774272,"data_stored":206738142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:01.321 INFO:tasks.lost_unfound.ceph_manager:recovered! 2024-09-15T21:21:01.321 INFO:tasks.util.rados:rados -p unfound_pool get new_1 - 2024-09-15T21:21:01.322 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_1 - 2024-09-15T21:21:01.420 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/new_1: (2) No such file or directory 2024-09-15T21:21:01.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:01.436 INFO:tasks.util.rados:rados -p unfound_pool get existed_1 - 2024-09-15T21:21:01.436 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_1 - 2024-09-15T21:21:01.533 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/existed_1: (2) No such file or directory 2024-09-15T21:21:01.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:01.542 INFO:tasks.util.rados:rados -p unfound_pool get existing_1 - 2024-09-15T21:21:01.542 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_1 - 2024-09-15T21:21:01.638 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:01.638 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:01.638 INFO:teuthology.orchestra.run.smithi073.stdout:# /etc/fstab 2024-09-15T21:21:01.638 INFO:teuthology.orchestra.run.smithi073.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-15T21:21:01.638 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:01.638 INFO:teuthology.orchestra.run.smithi073.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-15T21:21:01.638 INFO:teuthology.orchestra.run.smithi073.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-15T21:21:01.639 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:01.639 INFO:teuthology.orchestra.run.smithi073.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-15T21:21:01.639 INFO:teuthology.orchestra.run.smithi073.stdout:# units generated from this file. 2024-09-15T21:21:01.639 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:01.639 INFO:teuthology.orchestra.run.smithi073.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-15T21:21:01.639 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-15T21:21:01.648 INFO:tasks.util.rados:rados -p unfound_pool get new_2 - 2024-09-15T21:21:01.649 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_2 - 2024-09-15T21:21:01.733 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 30 20 55355 55335 7.20419 4.51172 0.0138298 0.0108363 2024-09-15T21:21:01.738 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/new_2: (2) No such file or directory 2024-09-15T21:21:01.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:01.744 INFO:tasks.util.rados:rados -p unfound_pool get existed_2 - 2024-09-15T21:21:01.744 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_2 - 2024-09-15T21:21:01.863 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/existed_2: (2) No such file or directory 2024-09-15T21:21:01.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:01.882 INFO:tasks.util.rados:rados -p unfound_pool get existing_2 - 2024-09-15T21:21:01.882 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_2 - 2024-09-15T21:21:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:01.974 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:01.974 INFO:teuthology.orchestra.run.smithi073.stdout:# /etc/fstab 2024-09-15T21:21:01.974 INFO:teuthology.orchestra.run.smithi073.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-15T21:21:01.974 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:01.974 INFO:teuthology.orchestra.run.smithi073.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-15T21:21:01.974 INFO:teuthology.orchestra.run.smithi073.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-15T21:21:01.974 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:01.974 INFO:teuthology.orchestra.run.smithi073.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-15T21:21:01.975 INFO:teuthology.orchestra.run.smithi073.stdout:# units generated from this file. 2024-09-15T21:21:01.975 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:01.975 INFO:teuthology.orchestra.run.smithi073.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-15T21:21:01.975 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-15T21:21:01.986 INFO:tasks.util.rados:rados -p unfound_pool get new_3 - 2024-09-15T21:21:01.986 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_3 - 2024-09-15T21:21:02.076 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/new_3: (2) No such file or directory 2024-09-15T21:21:02.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:02.085 INFO:tasks.util.rados:rados -p unfound_pool get existed_3 - 2024-09-15T21:21:02.085 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_3 - 2024-09-15T21:21:02.178 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/existed_3: (2) No such file or directory 2024-09-15T21:21:02.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:02.184 INFO:tasks.util.rados:rados -p unfound_pool get existing_3 - 2024-09-15T21:21:02.184 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_3 - 2024-09-15T21:21:02.279 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:02.279 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:02.279 INFO:teuthology.orchestra.run.smithi073.stdout:# /etc/fstab 2024-09-15T21:21:02.279 INFO:teuthology.orchestra.run.smithi073.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-15T21:21:02.279 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:02.280 INFO:teuthology.orchestra.run.smithi073.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-15T21:21:02.280 INFO:teuthology.orchestra.run.smithi073.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-15T21:21:02.280 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:02.280 INFO:teuthology.orchestra.run.smithi073.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-15T21:21:02.280 INFO:teuthology.orchestra.run.smithi073.stdout:# units generated from this file. 2024-09-15T21:21:02.280 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:02.280 INFO:teuthology.orchestra.run.smithi073.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-15T21:21:02.280 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-15T21:21:02.290 INFO:tasks.util.rados:rados -p unfound_pool get new_4 - 2024-09-15T21:21:02.290 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_4 - 2024-09-15T21:21:02.384 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/new_4: (2) No such file or directory 2024-09-15T21:21:02.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:02.392 INFO:tasks.util.rados:rados -p unfound_pool get existed_4 - 2024-09-15T21:21:02.392 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_4 - 2024-09-15T21:21:02.484 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/existed_4: (2) No such file or directory 2024-09-15T21:21:02.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:02.495 INFO:tasks.util.rados:rados -p unfound_pool get existing_4 - 2024-09-15T21:21:02.495 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_4 - 2024-09-15T21:21:02.594 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:02.594 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:02.595 INFO:teuthology.orchestra.run.smithi073.stdout:# /etc/fstab 2024-09-15T21:21:02.595 INFO:teuthology.orchestra.run.smithi073.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-15T21:21:02.595 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:02.595 INFO:teuthology.orchestra.run.smithi073.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-15T21:21:02.595 INFO:teuthology.orchestra.run.smithi073.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-15T21:21:02.595 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:02.595 INFO:teuthology.orchestra.run.smithi073.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-15T21:21:02.595 INFO:teuthology.orchestra.run.smithi073.stdout:# units generated from this file. 2024-09-15T21:21:02.595 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:02.595 INFO:teuthology.orchestra.run.smithi073.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-15T21:21:02.595 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-15T21:21:02.606 INFO:tasks.util.rados:rados -p unfound_pool get new_5 - 2024-09-15T21:21:02.606 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_5 - 2024-09-15T21:21:02.695 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/new_5: (2) No such file or directory 2024-09-15T21:21:02.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:02.701 INFO:tasks.util.rados:rados -p unfound_pool get existed_5 - 2024-09-15T21:21:02.702 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_5 - 2024-09-15T21:21:02.734 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 31 20 56502 56482 7.11632 4.48047 0.0266184 0.0109722 2024-09-15T21:21:02.802 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/existed_5: (2) No such file or directory 2024-09-15T21:21:02.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:02.810 INFO:tasks.util.rados:rados -p unfound_pool get existing_5 - 2024-09-15T21:21:02.810 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_5 - 2024-09-15T21:21:02.945 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:02.945 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:02.945 INFO:teuthology.orchestra.run.smithi073.stdout:# /etc/fstab 2024-09-15T21:21:02.945 INFO:teuthology.orchestra.run.smithi073.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-15T21:21:02.945 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:02.945 INFO:teuthology.orchestra.run.smithi073.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-15T21:21:02.945 INFO:teuthology.orchestra.run.smithi073.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-15T21:21:02.945 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:02.946 INFO:teuthology.orchestra.run.smithi073.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-15T21:21:02.946 INFO:teuthology.orchestra.run.smithi073.stdout:# units generated from this file. 2024-09-15T21:21:02.946 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:02.946 INFO:teuthology.orchestra.run.smithi073.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-15T21:21:02.946 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-15T21:21:02.957 INFO:tasks.util.rados:rados -p unfound_pool get new_6 - 2024-09-15T21:21:02.957 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_6 - 2024-09-15T21:21:03.042 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/new_6: (2) No such file or directory 2024-09-15T21:21:03.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:03.052 INFO:tasks.util.rados:rados -p unfound_pool get existed_6 - 2024-09-15T21:21:03.052 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_6 - 2024-09-15T21:21:03.142 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/existed_6: (2) No such file or directory 2024-09-15T21:21:03.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:03.149 INFO:tasks.util.rados:rados -p unfound_pool get existing_6 - 2024-09-15T21:21:03.149 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_6 - 2024-09-15T21:21:03.242 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:03.242 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:03.242 INFO:teuthology.orchestra.run.smithi073.stdout:# /etc/fstab 2024-09-15T21:21:03.242 INFO:teuthology.orchestra.run.smithi073.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-15T21:21:03.242 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:03.242 INFO:teuthology.orchestra.run.smithi073.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-15T21:21:03.242 INFO:teuthology.orchestra.run.smithi073.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-15T21:21:03.242 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:03.243 INFO:teuthology.orchestra.run.smithi073.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-15T21:21:03.243 INFO:teuthology.orchestra.run.smithi073.stdout:# units generated from this file. 2024-09-15T21:21:03.243 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:03.243 INFO:teuthology.orchestra.run.smithi073.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-15T21:21:03.243 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-15T21:21:03.255 INFO:tasks.util.rados:rados -p unfound_pool get new_7 - 2024-09-15T21:21:03.255 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_7 - 2024-09-15T21:21:03.358 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/new_7: (2) No such file or directory 2024-09-15T21:21:03.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:03.374 INFO:tasks.util.rados:rados -p unfound_pool get existed_7 - 2024-09-15T21:21:03.374 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_7 - 2024-09-15T21:21:03.477 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/existed_7: (2) No such file or directory 2024-09-15T21:21:03.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:03.484 INFO:tasks.util.rados:rados -p unfound_pool get existing_7 - 2024-09-15T21:21:03.484 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_7 - 2024-09-15T21:21:03.565 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:03.565 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:03.565 INFO:teuthology.orchestra.run.smithi073.stdout:# /etc/fstab 2024-09-15T21:21:03.565 INFO:teuthology.orchestra.run.smithi073.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-15T21:21:03.565 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:03.565 INFO:teuthology.orchestra.run.smithi073.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-15T21:21:03.566 INFO:teuthology.orchestra.run.smithi073.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-15T21:21:03.566 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:03.566 INFO:teuthology.orchestra.run.smithi073.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-15T21:21:03.566 INFO:teuthology.orchestra.run.smithi073.stdout:# units generated from this file. 2024-09-15T21:21:03.566 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:03.566 INFO:teuthology.orchestra.run.smithi073.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-15T21:21:03.566 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-15T21:21:03.578 INFO:tasks.util.rados:rados -p unfound_pool get new_8 - 2024-09-15T21:21:03.578 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_8 - 2024-09-15T21:21:03.677 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/new_8: (2) No such file or directory 2024-09-15T21:21:03.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:03.685 INFO:tasks.util.rados:rados -p unfound_pool get existed_8 - 2024-09-15T21:21:03.685 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_8 - 2024-09-15T21:21:03.734 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 32 20 57631 57611 7.03174 4.41016 0.0175392 0.0111036 2024-09-15T21:21:03.782 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/existed_8: (2) No such file or directory 2024-09-15T21:21:03.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:03.790 INFO:tasks.util.rados:rados -p unfound_pool get existing_8 - 2024-09-15T21:21:03.790 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_8 - 2024-09-15T21:21:03.878 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:03.878 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:03.878 INFO:teuthology.orchestra.run.smithi073.stdout:# /etc/fstab 2024-09-15T21:21:03.878 INFO:teuthology.orchestra.run.smithi073.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-15T21:21:03.878 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:03.878 INFO:teuthology.orchestra.run.smithi073.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-15T21:21:03.878 INFO:teuthology.orchestra.run.smithi073.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-15T21:21:03.878 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:03.878 INFO:teuthology.orchestra.run.smithi073.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-15T21:21:03.878 INFO:teuthology.orchestra.run.smithi073.stdout:# units generated from this file. 2024-09-15T21:21:03.879 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:03.879 INFO:teuthology.orchestra.run.smithi073.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-15T21:21:03.879 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-15T21:21:03.883 INFO:tasks.util.rados:rados -p unfound_pool get new_9 - 2024-09-15T21:21:03.883 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get new_9 - 2024-09-15T21:21:03.992 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/new_9: (2) No such file or directory 2024-09-15T21:21:04.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:04.006 INFO:tasks.util.rados:rados -p unfound_pool get existed_9 - 2024-09-15T21:21:04.006 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existed_9 - 2024-09-15T21:21:04.107 INFO:teuthology.orchestra.run.smithi073.stderr:error getting unfound_pool/existed_9: (2) No such file or directory 2024-09-15T21:21:04.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:21:04.118 INFO:tasks.util.rados:rados -p unfound_pool get existing_9 - 2024-09-15T21:21:04.118 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfound_pool get existing_9 - 2024-09-15T21:21:04.233 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:04.233 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:04.233 INFO:teuthology.orchestra.run.smithi073.stdout:# /etc/fstab 2024-09-15T21:21:04.233 INFO:teuthology.orchestra.run.smithi073.stdout:# Created by anaconda on Wed Aug 28 20:55:26 2024 2024-09-15T21:21:04.233 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:04.233 INFO:teuthology.orchestra.run.smithi073.stdout:# Accessible filesystems, by reference, are maintained under '/dev/disk/'. 2024-09-15T21:21:04.233 INFO:teuthology.orchestra.run.smithi073.stdout:# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info. 2024-09-15T21:21:04.233 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:04.234 INFO:teuthology.orchestra.run.smithi073.stdout:# After editing this file, run 'systemctl daemon-reload' to update systemd 2024-09-15T21:21:04.234 INFO:teuthology.orchestra.run.smithi073.stdout:# units generated from this file. 2024-09-15T21:21:04.234 INFO:teuthology.orchestra.run.smithi073.stdout:# 2024-09-15T21:21:04.234 INFO:teuthology.orchestra.run.smithi073.stdout:UUID=fa8d2923-554d-41e2-ac4a-c752ff079ceb / ext4 defaults 1 1 2024-09-15T21:21:04.234 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/vg_nvme/lv_5 /var/lib/ceph xfs defaults,nofail 0 0 2024-09-15T21:21:04.243 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-15T21:21:04.734 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 33 20 58774 58754 6.95395 4.46484 0.0165484 0.0112291 2024-09-15T21:21:04.796 INFO:tasks.ceph.mon.a.smithi073.stderr:2024-09-15T21:21:04.792+0000 7f8e1ee55640 -1 mon.a@0(leader).osd e36 definitely_dead 0 2024-09-15T21:21:05.306 INFO:teuthology.orchestra.run.smithi073.stderr:marked in osd.1. 2024-09-15T21:21:05.330 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-15T21:21:05.330 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-15T21:21:05.332 INFO:tasks.ceph.osd.1:Started 2024-09-15T21:21:05.332 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:21:05.429 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-15T21:21:05.429 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-15T21:21:05.430 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-15T21:21:05.430 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-15T21:21:05.430 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-15T21:21:05.430 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-15T21:21:05.430 INFO:teuthology.orchestra.run.smithi073.stderr:config diff 2024-09-15T21:21:05.430 INFO:teuthology.orchestra.run.smithi073.stderr:config diff get 2024-09-15T21:21:05.430 INFO:teuthology.orchestra.run.smithi073.stderr:config get 2024-09-15T21:21:05.430 INFO:teuthology.orchestra.run.smithi073.stderr:config help [] 2024-09-15T21:21:05.430 INFO:teuthology.orchestra.run.smithi073.stderr:config set ... 2024-09-15T21:21:05.430 INFO:teuthology.orchestra.run.smithi073.stderr:admin_socket: invalid command 2024-09-15T21:21:05.431 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:21:05.432 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T21:21:05.734 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 34 20 59529 59509 6.83616 2.94922 0.00266808 0.0112827 2024-09-15T21:21:06.253 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-15T21:21:06.249+0000 7fca43618640 -1 Falling back to public interface 2024-09-15T21:21:06.734 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 35 20 59529 59509 6.64084 0 - 0.0112827 2024-09-15T21:21:07.734 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 36 20 60070 60050 6.51508 1.05664 0.01578 0.011986 2024-09-15T21:21:08.734 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 37 20 61439 61419 6.48351 5.34766 0.0112023 0.0120444 2024-09-15T21:21:09.734 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 38 20 62737 62717 6.44632 5.07031 0.011346 0.0121128 2024-09-15T21:21:10.306 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-15T21:21:10.302+0000 7fca43618640 -1 osd.1 26 log_to_monitors true 2024-09-15T21:21:10.433 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:21:10.599 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-09-15T21:21:10.599 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-09-15T21:21:10.599 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-09-15T21:21:10.599 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-09-15T21:21:10.599 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-09-15T21:21:10.599 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-09-15T21:21:10.600 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-09-15T21:21:10.600 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-09-15T21:21:10.600 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-09-15T21:21:10.600 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-09-15T21:21:10.600 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator fragmentation block 2024-09-15T21:21:10.600 INFO:teuthology.orchestra.run.smithi073.stderr:admin_socket: invalid command 2024-09-15T21:21:10.603 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-15T21:21:10.603 INFO:tasks.lost_unfound.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-15T21:21:10.734 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 39 20 63921 63901 6.39961 4.625 0.0244933 0.0122023 2024-09-15T21:21:11.334 INFO:tasks.ceph.osd.1.smithi073.stderr:2024-09-15T21:21:11.330+0000 7fca36d99640 -1 osd.1 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-15T21:21:11.734 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:2024-09-15T21:21:11.732477+0000 min lat: 0.00235499 max lat: 2.06602 avg lat: 0.0122959 2024-09-15T21:21:11.734 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:21:11.735 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 40 20 65059 65039 6.35074 4.44531 0.0109842 0.0122959 2024-09-15T21:21:12.735 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 41 20 65864 65844 6.27254 3.14453 0.00842959 0.0123323 2024-09-15T21:21:13.735 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 42 20 65864 65844 6.12319 0 - 0.0123323 2024-09-15T21:21:14.735 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 43 20 65864 65844 5.98079 0 - 0.0123323 2024-09-15T21:21:15.604 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-15T21:21:15.736 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 44 20 66371 66351 5.88968 0.660156 0.0206326 0.0132526 2024-09-15T21:21:15.790 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:21:15.790 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-09-15T21:21:15.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.790 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.5385.0:66394 2.6 e43/42)", 2024-09-15T21:21:15.790 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-15T21:21:15.780133+0000", 2024-09-15T21:21:15.790 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0069626719999999996, 2024-09-15T21:21:15.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069846120000000003, 2024-09-15T21:21:15.791 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-15T21:21:15.791 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-15T21:21:15.791 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-09-15T21:21:15.791 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-15T21:21:15.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-15T21:21:15.791 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780133+0000", 2024-09-15T21:21:15.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-15T21:21:15.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780133+0000", 2024-09-15T21:21:15.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-15T21:21:15.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780133+0000", 2024-09-15T21:21:15.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5199999999999998e-07 2024-09-15T21:21:15.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-15T21:21:15.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780140+0000", 2024-09-15T21:21:15.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.849e-06 2024-09-15T21:21:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-15T21:21:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780145+0000", 2024-09-15T21:21:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8170000000000004e-06 2024-09-15T21:21:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-15T21:21:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780149+0000", 2024-09-15T21:21:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9859999999999996e-06 2024-09-15T21:21:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-15T21:21:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.5385.0:66386 2.7 e43/42 2:e6738840:::benchmark_data_smithi073_42248_object66385:head v 43'4137, mlcod=23'1)", 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-15T21:21:15.780156+0000", 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0069390290000000002, 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0070060560000000001, 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780156+0000", 2024-09-15T21:21:15.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780156+0000", 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780156+0000", 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.86e-07 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-15T21:21:15.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780162+0000", 2024-09-15T21:21:15.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6080000000000003e-06 2024-09-15T21:21:15.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-15T21:21:15.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780167+0000", 2024-09-15T21:21:15.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.045e-06 2024-09-15T21:21:15.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-15T21:21:15.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780171+0000", 2024-09-15T21:21:15.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.568e-06 2024-09-15T21:21:15.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780211+0000", 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0187e-05 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780231+0000", 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0548e-05 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.5385.0:66405 2.5 e43/42 2:a70bde81:::benchmark_data_smithi073_42248_object66404:head v 43'4120, mlcod=23'2)", 2024-09-15T21:21:15.797 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-15T21:21:15.780879+0000", 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0062161559999999996, 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0063298679999999998, 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780879+0000", 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-15T21:21:15.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780879+0000", 2024-09-15T21:21:15.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-15T21:21:15.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780880+0000", 2024-09-15T21:21:15.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6000000000000003e-07 2024-09-15T21:21:15.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-15T21:21:15.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780886+0000", 2024-09-15T21:21:15.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.178e-06 2024-09-15T21:21:15.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-15T21:21:15.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780899+0000", 2024-09-15T21:21:15.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.33e-05 2024-09-15T21:21:15.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-15T21:21:15.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780905+0000", 2024-09-15T21:21:15.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7270000000000004e-06 2024-09-15T21:21:15.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-15T21:21:15.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780972+0000", 2024-09-15T21:21:15.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7540999999999996e-05 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780993+0000", 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0333e-05 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.5385.0:66391 2.6 e43/42 2:6644abf4:::benchmark_data_smithi073_42248_object66390:head v 43'4176, mlcod=23'5)", 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-15T21:21:15.769973+0000", 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.017122017, 2024-09-15T21:21:15.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.01728472, 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.769973+0000", 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.769973+0000", 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-15T21:21:15.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.769974+0000", 2024-09-15T21:21:15.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3500000000000002e-07 2024-09-15T21:21:15.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-15T21:21:15.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.769982+0000", 2024-09-15T21:21:15.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2050000000000002e-06 2024-09-15T21:21:15.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-15T21:21:15.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.769989+0000", 2024-09-15T21:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9600000000000003e-06 2024-09-15T21:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-15T21:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.769994+0000", 2024-09-15T21:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8579999999999999e-06 2024-09-15T21:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-15T21:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.773508+0000", 2024-09-15T21:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0035144320000000001 2024-09-15T21:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-15T21:21:15.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.773536+0000", 2024-09-15T21:21:15.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7599000000000001e-05 2024-09-15T21:21:15.805 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.805 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-15T21:21:15.805 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.805 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.5385.0:66399 2.4 e43/42 2:248a0a6f:::benchmark_data_smithi073_42248_object66398:head v 43'4200, mlcod=23'9)", 2024-09-15T21:21:15.805 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-15T21:21:15.773373+0000", 2024-09-15T21:21:15.805 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.013721875, 2024-09-15T21:21:15.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.013932757, 2024-09-15T21:21:15.805 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.773373+0000", 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.773373+0000", 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-15T21:21:15.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.773375+0000", 2024-09-15T21:21:15.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.344e-06 2024-09-15T21:21:15.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-15T21:21:15.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.773387+0000", 2024-09-15T21:21:15.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.223e-05 2024-09-15T21:21:15.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-15T21:21:15.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.773399+0000", 2024-09-15T21:21:15.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1922999999999999e-05 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.773408+0000", 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7830000000000004e-06 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.776109+0000", 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0027017450000000002 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-15T21:21:15.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.776136+0000", 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7024000000000001e-05 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.5385.0:66393 2.6 e43/42 2:6756e023:::benchmark_data_smithi073_42248_object66392:head v 43'4177, mlcod=23'5)", 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-15T21:21:15.775548+0000", 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.01154755, 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.011803973000000001, 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-15T21:21:15.809 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.775548+0000", 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.775548+0000", 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.775549+0000", 2024-09-15T21:21:15.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0359999999999999e-06 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.775570+0000", 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0863999999999999e-05 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.775582+0000", 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1997999999999999e-05 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-15T21:21:15.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.775589+0000", 2024-09-15T21:21:15.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7020000000000001e-06 2024-09-15T21:21:15.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.812 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-15T21:21:15.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.783746+0000", 2024-09-15T21:21:15.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0081564280000000003 2024-09-15T21:21:15.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.812 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-15T21:21:15.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.783775+0000", 2024-09-15T21:21:15.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8915e-05 2024-09-15T21:21:15.812 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.812 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_repop(client.5385.0:66401 2.a e43/42 2:529c772d:::benchmark_data_smithi073_42248_object66400:head v 43'4122, mlcod=23'3)", 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-09-15T21:21:15.780085+0000", 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0070101740000000001, 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0073131639999999996, 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-09-15T21:21:15.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780085+0000", 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780085+0000", 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780086+0000", 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6599999999999994e-07 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780095+0000", 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8179999999999993e-06 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780105+0000", 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0057999999999999e-05 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780111+0000", 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4729999999999997e-06 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780168+0000", 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6888000000000001e-05 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-09-15T21:21:15.780189+0000", 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0675000000000001e-05 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-09-15T21:21:15.816 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 7 2024-09-15T21:21:15.817 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:21:15.818 INFO:tasks.lost_unfound.ceph_manager:waiting for osd.1 to be up 2024-09-15T21:21:15.818 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:21:16.373 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:16.373 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":43,"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","created":"2024-09-15T21:19:07.254022+0000","modified":"2024-09-15T21:21:14.337434+0000","last_up_change":"2024-09-15T21:21:12.333829+0000","last_in_change":"2024-09-15T21:21:05.287877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:19:25.465189+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-15T21:19:34.551635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c55f097a-59ed-4ce1-9803-4c6019429ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":42,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6801","nonce":173704734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6803","nonce":173704734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6807","nonce":173704734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6805","nonce":173704734}]},"public_addr":"172.21.15.73:6801/173704734","cluster_addr":"172.21.15.73:6803/173704734","heartbeat_back_addr":"172.21.15.73:6807/173704734","heartbeat_front_addr":"172.21.15.73:6805/173704734","state":["exists","up"]},{"osd":1,"uuid":"7daff6cd-a09c-4c15-a282-0859e8372dd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":26,"up_from":41,"up_thru":42,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3018414478},{"type":"v1","addr":"172.21.15.73:6817","nonce":3018414478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":3018414478},{"type":"v1","addr":"172.21.15.73:6819","nonce":3018414478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3018414478},{"type":"v1","addr":"172.21.15.73:6823","nonce":3018414478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":3018414478},{"type":"v1","addr":"172.21.15.73:6821","nonce":3018414478}]},"public_addr":"172.21.15.73:6817/3018414478","cluster_addr":"172.21.15.73:6819/3018414478","heartbeat_back_addr":"172.21.15.73:6823/3018414478","heartbeat_front_addr":"172.21.15.73:6821/3018414478","state":["exists","up"]},{"osd":2,"uuid":"9abad4dd-4a88-4080-9ab4-5c4203d7fd51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":17,"up_from":31,"up_thru":42,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":508991462},{"type":"v1","addr":"172.21.15.73:6809","nonce":508991462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":508991462},{"type":"v1","addr":"172.21.15.73:6811","nonce":508991462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":508991462},{"type":"v1","addr":"172.21.15.73:6815","nonce":508991462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":508991462},{"type":"v1","addr":"172.21.15.73:6813","nonce":508991462}]},"public_addr":"172.21.15.73:6809/508991462","cluster_addr":"172.21.15.73:6811/508991462","heartbeat_back_addr":"172.21.15.73:6815/508991462","heartbeat_front_addr":"172.21.15.73:6813/508991462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.867529+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.608114+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.891920+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"2.0","osds":[2]},{"pgid":"2.1","osds":[2]},{"pgid":"2.2","osds":[0]},{"pgid":"2.3","osds":[2]},{"pgid":"2.4","osds":[0]},{"pgid":"2.5","osds":[0]},{"pgid":"2.6","osds":[0]},{"pgid":"2.7","osds":[0]},{"pgid":"2.9","osds":[0]},{"pgid":"2.a","osds":[0]},{"pgid":"2.d","osds":[2]},{"pgid":"2.f","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:21:16.402 INFO:tasks.lost_unfound.ceph_manager:osd.1 is up 2024-09-15T21:21:16.402 INFO:tasks.lost_unfound.ceph_manager:waiting for clean 2024-09-15T21:21:16.402 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:16.736 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 45 20 66967 66947 5.81054 2.32812 0.0261735 0.0134376 2024-09-15T21:21:16.973 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:16.974 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:16.996 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-15T21:21:16.690237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":270013169,"num_objects":65932,"num_object_clones":0,"num_object_copies":131864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65932,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25797,"num_write_kb":103182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66005,"ondisk_log_size":66005,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":876840,"kb_used_data":529132,"kb_used_omap":12,"kb_used_meta":347635,"kb_avail":282238680,"statfs":{"total":289910292480,"available":289012408320,"internally_reserved":0,"allocated":541831168,"data_stored":540995008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":355978243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,4,0,0,0,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15441920,"num_objects":3770,"num_object_clones":0,"num_object_copies":7540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32862,"num_objects_misplaced":-46778,"num_objects_unfound":0,"num_objects_dirty":3770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3770,"num_write_kb":15080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000986"},"pg_stats":[{"pgid":"2.f","version":"43'4138","reported_seq":4208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.973249+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:14.973249+0000","last_peered":"2024-09-15T21:21:14.973249+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:14.973249+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16949248,"num_objects":4138,"num_object_clones":0,"num_object_copies":8276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4138,"num_write_kb":16552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"40'4049","reported_seq":4145,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.355550+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:12.355550+0000","last_peered":"2024-09-15T21:21:12.355550+0000","last_clean":"2024-09-15T21:21:12.355550+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:12.355550+0000","last_undegraded":"2024-09-15T21:21:12.355550+0000","last_fullsized":"2024-09-15T21:21:12.355550+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4049,"log_dups_size":0,"ondisk_log_size":4049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16561094,"num_objects":4045,"num_object_clones":0,"num_object_copies":8090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4045,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2330,"num_write_kb":9320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'4271","reported_seq":4383,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-15T21:21:12.349985+0000","last_change":"2024-09-15T21:21:12.352701+0000","last_active":"2024-09-15T21:21:12.352701+0000","last_peered":"2024-09-15T21:21:12.349985+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:07.341580+0000","last_became_peered":"2024-09-15T21:21:07.341580+0000","last_unstale":"2024-09-15T21:21:12.349985+0000","last_undegraded":"2024-09-15T21:21:12.349985+0000","last_fullsized":"2024-09-15T21:21:12.349985+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":4271,"log_dups_size":0,"ondisk_log_size":4271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17477632,"num_objects":4267,"num_object_clones":0,"num_object_copies":8534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2371,"num_write_kb":9484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"40'4048","reported_seq":4109,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.352096+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:12.352096+0000","last_peered":"2024-09-15T21:21:12.352096+0000","last_clean":"2024-09-15T21:21:12.352096+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:12.352096+0000","last_undegraded":"2024-09-15T21:21:12.352096+0000","last_fullsized":"2024-09-15T21:21:12.352096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4048,"log_dups_size":0,"ondisk_log_size":4048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16580608,"num_objects":4048,"num_object_clones":0,"num_object_copies":8096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4048,"num_write_kb":16192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4095","reported_seq":4222,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:15.017995+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:15.017995+0000","last_peered":"2024-09-15T21:21:15.017995+0000","last_clean":"2024-09-15T21:21:15.017995+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:15.017995+0000","last_undegraded":"2024-09-15T21:21:15.017995+0000","last_fullsized":"2024-09-15T21:21:15.017995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4095,"log_dups_size":0,"ondisk_log_size":4095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16744931,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":550,"num_write_kb":2200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4085","reported_seq":4216,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.005433+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:15.005433+0000","last_peered":"2024-09-15T21:21:15.005433+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:15.005433+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4085,"log_dups_size":0,"ondisk_log_size":4085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16712163,"num_objects":4081,"num_object_clones":0,"num_object_copies":8162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":761,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4171","reported_seq":4302,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.978720+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:14.978720+0000","last_peered":"2024-09-15T21:21:14.978720+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:14.978720+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4171,"log_dups_size":0,"ondisk_log_size":4171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17060806,"num_objects":4167,"num_object_clones":0,"num_object_copies":8334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":482,"num_write_kb":1928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4241","reported_seq":4364,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:15.005584+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:15.005584+0000","last_peered":"2024-09-15T21:21:15.005584+0000","last_clean":"2024-09-15T21:21:15.005584+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:15.005584+0000","last_undegraded":"2024-09-15T21:21:15.005584+0000","last_fullsized":"2024-09-15T21:21:15.005584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":4241,"log_dups_size":0,"ondisk_log_size":4241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17362944,"num_objects":4239,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1013,"num_write_kb":4052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"40'4093","reported_seq":4206,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-15T21:21:12.300354+0000","last_change":"2024-09-15T21:21:12.353444+0000","last_active":"2024-09-15T21:21:12.353444+0000","last_peered":"2024-09-15T21:21:12.300354+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:07.317452+0000","last_became_peered":"2024-09-15T21:21:07.317452+0000","last_unstale":"2024-09-15T21:21:12.300354+0000","last_undegraded":"2024-09-15T21:21:12.300354+0000","last_fullsized":"2024-09-15T21:21:12.300354+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16736739,"num_objects":4087,"num_object_clones":0,"num_object_copies":8174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2025,"num_write_kb":8100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.352390+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:12.352390+0000","last_peered":"2024-09-15T21:21:12.352390+0000","last_clean":"2024-09-15T21:21:12.352390+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:12.352390+0000","last_undegraded":"2024-09-15T21:21:12.352390+0000","last_fullsized":"2024-09-15T21:21:12.352390+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4034","reported_seq":4159,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.022851+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:15.022851+0000","last_peered":"2024-09-15T21:21:15.022851+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:15.022851+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4034,"log_dups_size":0,"ondisk_log_size":4034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16490496,"num_objects":4026,"num_object_clones":0,"num_object_copies":8052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":707,"num_write_kb":2828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'4094","reported_seq":4199,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-15T21:21:12.325580+0000","last_change":"2024-09-15T21:21:12.352115+0000","last_active":"2024-09-15T21:21:12.352115+0000","last_peered":"2024-09-15T21:21:12.325580+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:07.316154+0000","last_became_peered":"2024-09-15T21:21:07.316154+0000","last_unstale":"2024-09-15T21:21:12.325580+0000","last_undegraded":"2024-09-15T21:21:12.325580+0000","last_fullsized":"2024-09-15T21:21:12.325580+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4094,"log_dups_size":0,"ondisk_log_size":4094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16749027,"num_objects":4090,"num_object_clones":0,"num_object_copies":8180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4090,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1953,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"40'4168","reported_seq":4277,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-15T21:21:12.335143+0000","last_change":"2024-09-15T21:21:12.352131+0000","last_active":"2024-09-15T21:21:12.352131+0000","last_peered":"2024-09-15T21:21:12.335143+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:07.321423+0000","last_became_peered":"2024-09-15T21:21:07.321423+0000","last_unstale":"2024-09-15T21:21:12.335143+0000","last_undegraded":"2024-09-15T21:21:12.335143+0000","last_fullsized":"2024-09-15T21:21:12.335143+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":4168,"log_dups_size":0,"ondisk_log_size":4168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17047552,"num_objects":4162,"num_object_clones":0,"num_object_copies":8324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1861,"num_write_kb":7444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4100","reported_seq":4228,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.970102+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:14.970102+0000","last_peered":"2024-09-15T21:21:14.970102+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:14.970102+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4100,"log_dups_size":0,"ondisk_log_size":4100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16785408,"num_objects":4098,"num_object_clones":0,"num_object_copies":8196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1044,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4154","reported_seq":4281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.015247+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:15.015247+0000","last_peered":"2024-09-15T21:21:15.015247+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:15.015247+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4154,"log_dups_size":0,"ondisk_log_size":4154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16982016,"num_objects":4146,"num_object_clones":0,"num_object_copies":8292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1063,"num_write_kb":4252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.351173+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:12.351173+0000","last_peered":"2024-09-15T21:21:12.351173+0000","last_clean":"2024-09-15T21:21:12.351173+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:12.351173+0000","last_undegraded":"2024-09-15T21:21:12.351173+0000","last_fullsized":"2024-09-15T21:21:12.351173+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4093","reported_seq":4222,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.930199+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:14.930199+0000","last_peered":"2024-09-15T21:21:14.930199+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:14.930199+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16748544,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":577,"num_write_kb":2308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4168","reported_seq":4297,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.039892+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:15.039892+0000","last_peered":"2024-09-15T21:21:15.039892+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:15.039892+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4168,"log_dups_size":0,"ondisk_log_size":4168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17023942,"num_objects":4158,"num_object_clones":0,"num_object_copies":8316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4158,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":872,"num_write_kb":3488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":11},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":270013150,"num_objects":65930,"num_object_clones":0,"num_object_copies":131860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65930,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25795,"num_write_kb":103180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":541175808,"data_stored":541020449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66002,"ondisk_log_size":66002,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399632,"kb_used_data":263804,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93972208,"statfs":{"total":96636764160,"available":96227540992,"internally_reserved":0,"allocated":270135296,"data_stored":269878287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":139062953},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412052,"kb_used_data":263812,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93959788,"statfs":{"total":96636764160,"available":96214822912,"internally_reserved":0,"allocated":270143488,"data_stored":269877397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151776939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65156,"kb_used_data":1516,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94306684,"statfs":{"total":96636764160,"available":96570044416,"internally_reserved":0,"allocated":1552384,"data_stored":1239324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":65138351},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":270139392,"data_stored":270103262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1249280,"data_stored":1166181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":269787136,"data_stored":269751006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:16.998 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:17.555 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:17.555 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:17.589 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-15T21:21:16.690237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":270013169,"num_objects":65932,"num_object_clones":0,"num_object_copies":131864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65932,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25797,"num_write_kb":103182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66005,"ondisk_log_size":66005,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":876840,"kb_used_data":529132,"kb_used_omap":12,"kb_used_meta":347635,"kb_avail":282238680,"statfs":{"total":289910292480,"available":289012408320,"internally_reserved":0,"allocated":541831168,"data_stored":540995008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":355978243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,4,0,0,0,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15441920,"num_objects":3770,"num_object_clones":0,"num_object_copies":7540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32862,"num_objects_misplaced":-46778,"num_objects_unfound":0,"num_objects_dirty":3770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3770,"num_write_kb":15080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000986"},"pg_stats":[{"pgid":"2.f","version":"43'4138","reported_seq":4208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.973249+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:14.973249+0000","last_peered":"2024-09-15T21:21:14.973249+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:14.973249+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16949248,"num_objects":4138,"num_object_clones":0,"num_object_copies":8276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4138,"num_write_kb":16552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"40'4049","reported_seq":4145,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.355550+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:12.355550+0000","last_peered":"2024-09-15T21:21:12.355550+0000","last_clean":"2024-09-15T21:21:12.355550+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:12.355550+0000","last_undegraded":"2024-09-15T21:21:12.355550+0000","last_fullsized":"2024-09-15T21:21:12.355550+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4049,"log_dups_size":0,"ondisk_log_size":4049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16561094,"num_objects":4045,"num_object_clones":0,"num_object_copies":8090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4045,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2330,"num_write_kb":9320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'4271","reported_seq":4383,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-15T21:21:12.349985+0000","last_change":"2024-09-15T21:21:12.352701+0000","last_active":"2024-09-15T21:21:12.352701+0000","last_peered":"2024-09-15T21:21:12.349985+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:07.341580+0000","last_became_peered":"2024-09-15T21:21:07.341580+0000","last_unstale":"2024-09-15T21:21:12.349985+0000","last_undegraded":"2024-09-15T21:21:12.349985+0000","last_fullsized":"2024-09-15T21:21:12.349985+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":4271,"log_dups_size":0,"ondisk_log_size":4271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17477632,"num_objects":4267,"num_object_clones":0,"num_object_copies":8534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2371,"num_write_kb":9484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"40'4048","reported_seq":4109,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.352096+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:12.352096+0000","last_peered":"2024-09-15T21:21:12.352096+0000","last_clean":"2024-09-15T21:21:12.352096+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:12.352096+0000","last_undegraded":"2024-09-15T21:21:12.352096+0000","last_fullsized":"2024-09-15T21:21:12.352096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4048,"log_dups_size":0,"ondisk_log_size":4048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16580608,"num_objects":4048,"num_object_clones":0,"num_object_copies":8096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4048,"num_write_kb":16192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4095","reported_seq":4222,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:15.017995+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:15.017995+0000","last_peered":"2024-09-15T21:21:15.017995+0000","last_clean":"2024-09-15T21:21:15.017995+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:15.017995+0000","last_undegraded":"2024-09-15T21:21:15.017995+0000","last_fullsized":"2024-09-15T21:21:15.017995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4095,"log_dups_size":0,"ondisk_log_size":4095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16744931,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":550,"num_write_kb":2200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4085","reported_seq":4216,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.005433+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:15.005433+0000","last_peered":"2024-09-15T21:21:15.005433+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:15.005433+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4085,"log_dups_size":0,"ondisk_log_size":4085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16712163,"num_objects":4081,"num_object_clones":0,"num_object_copies":8162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":761,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4171","reported_seq":4302,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.978720+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:14.978720+0000","last_peered":"2024-09-15T21:21:14.978720+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:14.978720+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4171,"log_dups_size":0,"ondisk_log_size":4171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17060806,"num_objects":4167,"num_object_clones":0,"num_object_copies":8334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":482,"num_write_kb":1928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4241","reported_seq":4364,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:15.005584+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:15.005584+0000","last_peered":"2024-09-15T21:21:15.005584+0000","last_clean":"2024-09-15T21:21:15.005584+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:15.005584+0000","last_undegraded":"2024-09-15T21:21:15.005584+0000","last_fullsized":"2024-09-15T21:21:15.005584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":4241,"log_dups_size":0,"ondisk_log_size":4241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17362944,"num_objects":4239,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1013,"num_write_kb":4052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"40'4093","reported_seq":4206,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-15T21:21:12.300354+0000","last_change":"2024-09-15T21:21:12.353444+0000","last_active":"2024-09-15T21:21:12.353444+0000","last_peered":"2024-09-15T21:21:12.300354+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:07.317452+0000","last_became_peered":"2024-09-15T21:21:07.317452+0000","last_unstale":"2024-09-15T21:21:12.300354+0000","last_undegraded":"2024-09-15T21:21:12.300354+0000","last_fullsized":"2024-09-15T21:21:12.300354+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16736739,"num_objects":4087,"num_object_clones":0,"num_object_copies":8174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2025,"num_write_kb":8100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.352390+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:12.352390+0000","last_peered":"2024-09-15T21:21:12.352390+0000","last_clean":"2024-09-15T21:21:12.352390+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:12.352390+0000","last_undegraded":"2024-09-15T21:21:12.352390+0000","last_fullsized":"2024-09-15T21:21:12.352390+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4034","reported_seq":4159,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.022851+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:15.022851+0000","last_peered":"2024-09-15T21:21:15.022851+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:15.022851+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4034,"log_dups_size":0,"ondisk_log_size":4034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16490496,"num_objects":4026,"num_object_clones":0,"num_object_copies":8052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":707,"num_write_kb":2828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'4094","reported_seq":4199,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-15T21:21:12.325580+0000","last_change":"2024-09-15T21:21:12.352115+0000","last_active":"2024-09-15T21:21:12.352115+0000","last_peered":"2024-09-15T21:21:12.325580+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:07.316154+0000","last_became_peered":"2024-09-15T21:21:07.316154+0000","last_unstale":"2024-09-15T21:21:12.325580+0000","last_undegraded":"2024-09-15T21:21:12.325580+0000","last_fullsized":"2024-09-15T21:21:12.325580+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4094,"log_dups_size":0,"ondisk_log_size":4094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16749027,"num_objects":4090,"num_object_clones":0,"num_object_copies":8180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4090,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1953,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"40'4168","reported_seq":4277,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-15T21:21:12.335143+0000","last_change":"2024-09-15T21:21:12.352131+0000","last_active":"2024-09-15T21:21:12.352131+0000","last_peered":"2024-09-15T21:21:12.335143+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:07.321423+0000","last_became_peered":"2024-09-15T21:21:07.321423+0000","last_unstale":"2024-09-15T21:21:12.335143+0000","last_undegraded":"2024-09-15T21:21:12.335143+0000","last_fullsized":"2024-09-15T21:21:12.335143+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":4168,"log_dups_size":0,"ondisk_log_size":4168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17047552,"num_objects":4162,"num_object_clones":0,"num_object_copies":8324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1861,"num_write_kb":7444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4100","reported_seq":4228,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.970102+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:14.970102+0000","last_peered":"2024-09-15T21:21:14.970102+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:14.970102+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4100,"log_dups_size":0,"ondisk_log_size":4100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16785408,"num_objects":4098,"num_object_clones":0,"num_object_copies":8196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1044,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4154","reported_seq":4281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.015247+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:15.015247+0000","last_peered":"2024-09-15T21:21:15.015247+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:15.015247+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4154,"log_dups_size":0,"ondisk_log_size":4154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16982016,"num_objects":4146,"num_object_clones":0,"num_object_copies":8292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1063,"num_write_kb":4252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.351173+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:12.351173+0000","last_peered":"2024-09-15T21:21:12.351173+0000","last_clean":"2024-09-15T21:21:12.351173+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:12.351173+0000","last_undegraded":"2024-09-15T21:21:12.351173+0000","last_fullsized":"2024-09-15T21:21:12.351173+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4093","reported_seq":4222,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.930199+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:14.930199+0000","last_peered":"2024-09-15T21:21:14.930199+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:14.930199+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16748544,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":577,"num_write_kb":2308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4168","reported_seq":4297,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.039892+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:15.039892+0000","last_peered":"2024-09-15T21:21:15.039892+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:15.039892+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4168,"log_dups_size":0,"ondisk_log_size":4168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17023942,"num_objects":4158,"num_object_clones":0,"num_object_copies":8316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4158,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":872,"num_write_kb":3488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":11},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":270013150,"num_objects":65930,"num_object_clones":0,"num_object_copies":131860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65930,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25795,"num_write_kb":103180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":541175808,"data_stored":541020449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66002,"ondisk_log_size":66002,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399632,"kb_used_data":263804,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93972208,"statfs":{"total":96636764160,"available":96227540992,"internally_reserved":0,"allocated":270135296,"data_stored":269878287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":139062953},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412052,"kb_used_data":263812,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93959788,"statfs":{"total":96636764160,"available":96214822912,"internally_reserved":0,"allocated":270143488,"data_stored":269877397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151776939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65156,"kb_used_data":1516,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94306684,"statfs":{"total":96636764160,"available":96570044416,"internally_reserved":0,"allocated":1552384,"data_stored":1239324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":65138351},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":270139392,"data_stored":270103262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1249280,"data_stored":1166181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":269787136,"data_stored":269751006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:17.590 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:17.736 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 46 20 67548 67528 5.73356 2.26953 0.0603049 0.0136174 2024-09-15T21:21:18.141 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:18.142 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:18.165 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-15T21:21:16.690237+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":270013169,"num_objects":65932,"num_object_clones":0,"num_object_copies":131864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65932,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25797,"num_write_kb":103182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66005,"ondisk_log_size":66005,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":876840,"kb_used_data":529132,"kb_used_omap":12,"kb_used_meta":347635,"kb_avail":282238680,"statfs":{"total":289910292480,"available":289012408320,"internally_reserved":0,"allocated":541831168,"data_stored":540995008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":355978243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,4,0,0,0,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15441920,"num_objects":3770,"num_object_clones":0,"num_object_copies":7540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32862,"num_objects_misplaced":-46778,"num_objects_unfound":0,"num_objects_dirty":3770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3770,"num_write_kb":15080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000986"},"pg_stats":[{"pgid":"2.f","version":"43'4138","reported_seq":4208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.973249+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:14.973249+0000","last_peered":"2024-09-15T21:21:14.973249+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:14.973249+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16949248,"num_objects":4138,"num_object_clones":0,"num_object_copies":8276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4138,"num_write_kb":16552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"40'4049","reported_seq":4145,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.355550+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:12.355550+0000","last_peered":"2024-09-15T21:21:12.355550+0000","last_clean":"2024-09-15T21:21:12.355550+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:12.355550+0000","last_undegraded":"2024-09-15T21:21:12.355550+0000","last_fullsized":"2024-09-15T21:21:12.355550+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4049,"log_dups_size":0,"ondisk_log_size":4049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16561094,"num_objects":4045,"num_object_clones":0,"num_object_copies":8090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4045,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2330,"num_write_kb":9320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'4271","reported_seq":4383,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-15T21:21:12.349985+0000","last_change":"2024-09-15T21:21:12.352701+0000","last_active":"2024-09-15T21:21:12.352701+0000","last_peered":"2024-09-15T21:21:12.349985+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:07.341580+0000","last_became_peered":"2024-09-15T21:21:07.341580+0000","last_unstale":"2024-09-15T21:21:12.349985+0000","last_undegraded":"2024-09-15T21:21:12.349985+0000","last_fullsized":"2024-09-15T21:21:12.349985+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":4271,"log_dups_size":0,"ondisk_log_size":4271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17477632,"num_objects":4267,"num_object_clones":0,"num_object_copies":8534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2371,"num_write_kb":9484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"40'4048","reported_seq":4109,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.352096+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:12.352096+0000","last_peered":"2024-09-15T21:21:12.352096+0000","last_clean":"2024-09-15T21:21:12.352096+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:12.352096+0000","last_undegraded":"2024-09-15T21:21:12.352096+0000","last_fullsized":"2024-09-15T21:21:12.352096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4048,"log_dups_size":0,"ondisk_log_size":4048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16580608,"num_objects":4048,"num_object_clones":0,"num_object_copies":8096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4048,"num_write_kb":16192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4095","reported_seq":4222,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:15.017995+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:15.017995+0000","last_peered":"2024-09-15T21:21:15.017995+0000","last_clean":"2024-09-15T21:21:15.017995+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:15.017995+0000","last_undegraded":"2024-09-15T21:21:15.017995+0000","last_fullsized":"2024-09-15T21:21:15.017995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4095,"log_dups_size":0,"ondisk_log_size":4095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16744931,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":550,"num_write_kb":2200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4085","reported_seq":4216,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.005433+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:15.005433+0000","last_peered":"2024-09-15T21:21:15.005433+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:15.005433+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4085,"log_dups_size":0,"ondisk_log_size":4085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16712163,"num_objects":4081,"num_object_clones":0,"num_object_copies":8162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":761,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4171","reported_seq":4302,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.978720+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:14.978720+0000","last_peered":"2024-09-15T21:21:14.978720+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:14.978720+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4171,"log_dups_size":0,"ondisk_log_size":4171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17060806,"num_objects":4167,"num_object_clones":0,"num_object_copies":8334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":482,"num_write_kb":1928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4241","reported_seq":4364,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:15.005584+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:15.005584+0000","last_peered":"2024-09-15T21:21:15.005584+0000","last_clean":"2024-09-15T21:21:15.005584+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:15.005584+0000","last_undegraded":"2024-09-15T21:21:15.005584+0000","last_fullsized":"2024-09-15T21:21:15.005584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":4241,"log_dups_size":0,"ondisk_log_size":4241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17362944,"num_objects":4239,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1013,"num_write_kb":4052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"40'4093","reported_seq":4206,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-15T21:21:12.300354+0000","last_change":"2024-09-15T21:21:12.353444+0000","last_active":"2024-09-15T21:21:12.353444+0000","last_peered":"2024-09-15T21:21:12.300354+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:07.317452+0000","last_became_peered":"2024-09-15T21:21:07.317452+0000","last_unstale":"2024-09-15T21:21:12.300354+0000","last_undegraded":"2024-09-15T21:21:12.300354+0000","last_fullsized":"2024-09-15T21:21:12.300354+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16736739,"num_objects":4087,"num_object_clones":0,"num_object_copies":8174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4087,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2025,"num_write_kb":8100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.352390+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:12.352390+0000","last_peered":"2024-09-15T21:21:12.352390+0000","last_clean":"2024-09-15T21:21:12.352390+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:12.352390+0000","last_undegraded":"2024-09-15T21:21:12.352390+0000","last_fullsized":"2024-09-15T21:21:12.352390+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4034","reported_seq":4159,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.022851+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:15.022851+0000","last_peered":"2024-09-15T21:21:15.022851+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:15.022851+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4034,"log_dups_size":0,"ondisk_log_size":4034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16490496,"num_objects":4026,"num_object_clones":0,"num_object_copies":8052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":707,"num_write_kb":2828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'4094","reported_seq":4199,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-15T21:21:12.325580+0000","last_change":"2024-09-15T21:21:12.352115+0000","last_active":"2024-09-15T21:21:12.352115+0000","last_peered":"2024-09-15T21:21:12.325580+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:07.316154+0000","last_became_peered":"2024-09-15T21:21:07.316154+0000","last_unstale":"2024-09-15T21:21:12.325580+0000","last_undegraded":"2024-09-15T21:21:12.325580+0000","last_fullsized":"2024-09-15T21:21:12.325580+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4094,"log_dups_size":0,"ondisk_log_size":4094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16749027,"num_objects":4090,"num_object_clones":0,"num_object_copies":8180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4090,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1953,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"40'4168","reported_seq":4277,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-15T21:21:12.335143+0000","last_change":"2024-09-15T21:21:12.352131+0000","last_active":"2024-09-15T21:21:12.352131+0000","last_peered":"2024-09-15T21:21:12.335143+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:07.321423+0000","last_became_peered":"2024-09-15T21:21:07.321423+0000","last_unstale":"2024-09-15T21:21:12.335143+0000","last_undegraded":"2024-09-15T21:21:12.335143+0000","last_fullsized":"2024-09-15T21:21:12.335143+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":4168,"log_dups_size":0,"ondisk_log_size":4168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17047552,"num_objects":4162,"num_object_clones":0,"num_object_copies":8324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1861,"num_write_kb":7444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4100","reported_seq":4228,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.970102+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:14.970102+0000","last_peered":"2024-09-15T21:21:14.970102+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:14.970102+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4100,"log_dups_size":0,"ondisk_log_size":4100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16785408,"num_objects":4098,"num_object_clones":0,"num_object_copies":8196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1044,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4154","reported_seq":4281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.015247+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:15.015247+0000","last_peered":"2024-09-15T21:21:15.015247+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:15.015247+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4154,"log_dups_size":0,"ondisk_log_size":4154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16982016,"num_objects":4146,"num_object_clones":0,"num_object_copies":8292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1063,"num_write_kb":4252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-15T21:21:12.351173+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:12.351173+0000","last_peered":"2024-09-15T21:21:12.351173+0000","last_clean":"2024-09-15T21:21:12.351173+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:12.351173+0000","last_undegraded":"2024-09-15T21:21:12.351173+0000","last_fullsized":"2024-09-15T21:21:12.351173+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4093","reported_seq":4222,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.930199+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:14.930199+0000","last_peered":"2024-09-15T21:21:14.930199+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:14.930199+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16748544,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":577,"num_write_kb":2308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4168","reported_seq":4297,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.039892+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:15.039892+0000","last_peered":"2024-09-15T21:21:15.039892+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:15.039892+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4168,"log_dups_size":0,"ondisk_log_size":4168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17023942,"num_objects":4158,"num_object_clones":0,"num_object_copies":8316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4158,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":872,"num_write_kb":3488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":11},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":270013150,"num_objects":65930,"num_object_clones":0,"num_object_copies":131860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32862,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65930,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25795,"num_write_kb":103180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":541175808,"data_stored":541020449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66002,"ondisk_log_size":66002,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986191,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":399632,"kb_used_data":263804,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93972208,"statfs":{"total":96636764160,"available":96227540992,"internally_reserved":0,"allocated":270135296,"data_stored":269878287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":139062953},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,5,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412052,"kb_used_data":263812,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93959788,"statfs":{"total":96636764160,"available":96214822912,"internally_reserved":0,"allocated":270143488,"data_stored":269877397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151776939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65156,"kb_used_data":1516,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94306684,"statfs":{"total":96636764160,"available":96570044416,"internally_reserved":0,"allocated":1552384,"data_stored":1239324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":65138351},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":270139392,"data_stored":270103262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1249280,"data_stored":1166181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":269787136,"data_stored":269751006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:18.166 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:18.166 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'4138", 'reported_seq': 4208, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:14.973249+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:21:14.973249+0000', 'last_peered': '2024-09-15T21:21:14.973249+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:21:14.973249+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 4138, 'log_dups_size': 0, 'ondisk_log_size': 4138, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16949248, 'num_objects': 4138, 'num_object_clones': 0, 'num_object_copies': 8276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4138, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4138, 'num_write_kb': 16552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:18.167 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:21:18.167 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "40'4271", 'reported_seq': 4383, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-15T21:21:12.349985+0000', 'last_change': '2024-09-15T21:21:12.352701+0000', 'last_active': '2024-09-15T21:21:12.352701+0000', 'last_peered': '2024-09-15T21:21:12.349985+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:07.341580+0000', 'last_became_peered': '2024-09-15T21:21:07.341580+0000', 'last_unstale': '2024-09-15T21:21:12.349985+0000', 'last_undegraded': '2024-09-15T21:21:12.349985+0000', 'last_fullsized': '2024-09-15T21:21:12.349985+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 4271, 'log_dups_size': 0, 'ondisk_log_size': 4271, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17477632, 'num_objects': 4267, 'num_object_clones': 0, 'num_object_copies': 8534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4267, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2371, 'num_write_kb': 9484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:18.167 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:18.167 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'4085", 'reported_seq': 4216, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:15.005433+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:21:15.005433+0000', 'last_peered': '2024-09-15T21:21:15.005433+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:21:15.005433+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 4085, 'log_dups_size': 0, 'ondisk_log_size': 4085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16712163, 'num_objects': 4081, 'num_object_clones': 0, 'num_object_copies': 8162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4081, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 761, 'num_write_kb': 3044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:18.167 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:18.167 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'4171", 'reported_seq': 4302, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:14.978720+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:21:14.978720+0000', 'last_peered': '2024-09-15T21:21:14.978720+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:21:14.978720+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 4171, 'log_dups_size': 0, 'ondisk_log_size': 4171, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17060806, 'num_objects': 4167, 'num_object_clones': 0, 'num_object_copies': 8334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4167, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 482, 'num_write_kb': 1928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:18.167 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:18.167 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "40'4093", 'reported_seq': 4206, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-15T21:21:12.300354+0000', 'last_change': '2024-09-15T21:21:12.353444+0000', 'last_active': '2024-09-15T21:21:12.353444+0000', 'last_peered': '2024-09-15T21:21:12.300354+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:07.317452+0000', 'last_became_peered': '2024-09-15T21:21:07.317452+0000', 'last_unstale': '2024-09-15T21:21:12.300354+0000', 'last_undegraded': '2024-09-15T21:21:12.300354+0000', 'last_fullsized': '2024-09-15T21:21:12.300354+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 4093, 'log_dups_size': 0, 'ondisk_log_size': 4093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16736739, 'num_objects': 4087, 'num_object_clones': 0, 'num_object_copies': 8174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4087, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2025, 'num_write_kb': 8100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:18.167 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:18.167 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'4034", 'reported_seq': 4159, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:15.022851+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:21:15.022851+0000', 'last_peered': '2024-09-15T21:21:15.022851+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:21:15.022851+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 4034, 'log_dups_size': 0, 'ondisk_log_size': 4034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16490496, 'num_objects': 4026, 'num_object_clones': 0, 'num_object_copies': 8052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 707, 'num_write_kb': 2828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:18.168 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:18.168 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "40'4094", 'reported_seq': 4199, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-15T21:21:12.325580+0000', 'last_change': '2024-09-15T21:21:12.352115+0000', 'last_active': '2024-09-15T21:21:12.352115+0000', 'last_peered': '2024-09-15T21:21:12.325580+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:07.316154+0000', 'last_became_peered': '2024-09-15T21:21:07.316154+0000', 'last_unstale': '2024-09-15T21:21:12.325580+0000', 'last_undegraded': '2024-09-15T21:21:12.325580+0000', 'last_fullsized': '2024-09-15T21:21:12.325580+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 4094, 'log_dups_size': 0, 'ondisk_log_size': 4094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16749027, 'num_objects': 4090, 'num_object_clones': 0, 'num_object_copies': 8180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4090, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1953, 'num_write_kb': 7812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:18.168 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:18.168 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "40'4168", 'reported_seq': 4277, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-15T21:21:12.335143+0000', 'last_change': '2024-09-15T21:21:12.352131+0000', 'last_active': '2024-09-15T21:21:12.352131+0000', 'last_peered': '2024-09-15T21:21:12.335143+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:07.321423+0000', 'last_became_peered': '2024-09-15T21:21:07.321423+0000', 'last_unstale': '2024-09-15T21:21:12.335143+0000', 'last_undegraded': '2024-09-15T21:21:12.335143+0000', 'last_fullsized': '2024-09-15T21:21:12.335143+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 4168, 'log_dups_size': 0, 'ondisk_log_size': 4168, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17047552, 'num_objects': 4162, 'num_object_clones': 0, 'num_object_copies': 8324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4162, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1861, 'num_write_kb': 7444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:18.168 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:18.168 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'4100", 'reported_seq': 4228, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:14.970102+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:21:14.970102+0000', 'last_peered': '2024-09-15T21:21:14.970102+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:21:14.970102+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 4100, 'log_dups_size': 0, 'ondisk_log_size': 4100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16785408, 'num_objects': 4098, 'num_object_clones': 0, 'num_object_copies': 8196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4098, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1044, 'num_write_kb': 4176, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:18.168 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:18.168 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'4154", 'reported_seq': 4281, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:15.015247+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:21:15.015247+0000', 'last_peered': '2024-09-15T21:21:15.015247+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:21:15.015247+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 4154, 'log_dups_size': 0, 'ondisk_log_size': 4154, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16982016, 'num_objects': 4146, 'num_object_clones': 0, 'num_object_copies': 8292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4146, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1063, 'num_write_kb': 4252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:18.168 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:18.168 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'4093", 'reported_seq': 4222, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:14.930199+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:21:14.930199+0000', 'last_peered': '2024-09-15T21:21:14.930199+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:21:14.930199+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 4093, 'log_dups_size': 0, 'ondisk_log_size': 4093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16748544, 'num_objects': 4089, 'num_object_clones': 0, 'num_object_copies': 8178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4089, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 577, 'num_write_kb': 2308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:18.169 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:18.169 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'4168", 'reported_seq': 4297, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:15.039892+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:21:15.039892+0000', 'last_peered': '2024-09-15T21:21:15.039892+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:21:15.039892+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 4168, 'log_dups_size': 0, 'ondisk_log_size': 4168, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17023942, 'num_objects': 4158, 'num_object_clones': 0, 'num_object_copies': 8316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4158, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 872, 'num_write_kb': 3488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 11}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:18.169 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:18.737 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 47 20 68136 68116 5.66044 2.29688 0.0153189 0.0137937 2024-09-15T21:21:18.877 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:18.878 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32862/131864 objects degraded (24.921%), 8 pgs degraded","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":130,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"remapped+peering","count":4}],"num_pgs":24,"num_pools":2,"num_objects":65932,"data_bytes":270013169,"bytes_used":897884160,"bytes_avail":289012408320,"bytes_total":289910292480,"inactive_pgs_ratio":0.1666666716337204,"degraded_objects":32862,"degraded_total":131864,"degraded_ratio":0.24921130862100344,"write_bytes_sec":2573230,"read_op_per_sec":0,"write_op_per_sec":628},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2024-09-15T21:21:00.687659+0000","services":{"osd":{"daemons":{"summary":"","2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:21:18.902 INFO:tasks.lost_unfound.ceph_manager:no progress seen, keeping timeout for now 2024-09-15T21:21:18.902 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:19.445 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:19.470 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-15T21:21:18.692922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":272765681,"num_objects":66604,"num_object_clones":0,"num_object_copies":133208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":4085,"num_objects_unfound":0,"num_objects_dirty":66604,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":26469,"num_write_kb":105870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":319507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66677,"ondisk_log_size":66677,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":63,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":884260,"kb_used_data":532396,"kb_used_omap":12,"kb_used_meta":351795,"kb_avail":282231260,"statfs":{"total":289910292480,"available":289004810240,"internally_reserved":0,"allocated":545173504,"data_stored":544332339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":360238083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,0,3,1,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18194432,"num_objects":4442,"num_object_clones":0,"num_object_copies":8884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":-42693,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4442,"num_write_kb":17768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":315392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.355182"},"pg_stats":[{"pgid":"2.f","version":"43'4138","reported_seq":4208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.973249+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:14.973249+0000","last_peered":"2024-09-15T21:21:14.973249+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:14.973249+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4138,"log_dups_size":0,"ondisk_log_size":4138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16949248,"num_objects":4138,"num_object_clones":0,"num_object_copies":8276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4138,"num_write_kb":16552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4157","reported_seq":4257,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:17.957469+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:17.957469+0000","last_peered":"2024-09-15T21:21:17.957469+0000","last_clean":"2024-09-15T21:21:17.957469+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:17.957469+0000","last_undegraded":"2024-09-15T21:21:17.957469+0000","last_fullsized":"2024-09-15T21:21:17.957469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4157,"log_dups_size":0,"ondisk_log_size":4157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17003462,"num_objects":4153,"num_object_clones":0,"num_object_copies":8306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4153,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2438,"num_write_kb":9752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4394","reported_seq":4514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.999209+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:17.999209+0000","last_peered":"2024-09-15T21:21:17.999209+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:17.999209+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":4394,"log_dups_size":0,"ondisk_log_size":4394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17981440,"num_objects":4390,"num_object_clones":0,"num_object_copies":8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2494,"num_write_kb":9976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":123}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4167","reported_seq":4232,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:17.977231+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:17.977231+0000","last_peered":"2024-09-15T21:21:17.977231+0000","last_clean":"2024-09-15T21:21:17.977231+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:17.977231+0000","last_undegraded":"2024-09-15T21:21:17.977231+0000","last_fullsized":"2024-09-15T21:21:17.977231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4167,"log_dups_size":0,"ondisk_log_size":4167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17068032,"num_objects":4167,"num_object_clones":0,"num_object_copies":8334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4167,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4095","reported_seq":4222,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:15.017995+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:15.017995+0000","last_peered":"2024-09-15T21:21:15.017995+0000","last_clean":"2024-09-15T21:21:15.017995+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:15.017995+0000","last_undegraded":"2024-09-15T21:21:15.017995+0000","last_fullsized":"2024-09-15T21:21:15.017995+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4095,"log_dups_size":0,"ondisk_log_size":4095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16744931,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":550,"num_write_kb":2200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4085","reported_seq":4216,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.005433+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:15.005433+0000","last_peered":"2024-09-15T21:21:15.005433+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:15.005433+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4085,"log_dups_size":0,"ondisk_log_size":4085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16712163,"num_objects":4081,"num_object_clones":0,"num_object_copies":8162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":761,"num_write_kb":3044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4171","reported_seq":4302,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.978720+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:14.978720+0000","last_peered":"2024-09-15T21:21:14.978720+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:14.978720+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4171,"log_dups_size":0,"ondisk_log_size":4171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17060806,"num_objects":4167,"num_object_clones":0,"num_object_copies":8334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":482,"num_write_kb":1928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4241","reported_seq":4364,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:15.005584+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:15.005584+0000","last_peered":"2024-09-15T21:21:15.005584+0000","last_clean":"2024-09-15T21:21:15.005584+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:15.005584+0000","last_undegraded":"2024-09-15T21:21:15.005584+0000","last_fullsized":"2024-09-15T21:21:15.005584+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":4241,"log_dups_size":0,"ondisk_log_size":4241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17362944,"num_objects":4239,"num_object_clones":0,"num_object_copies":8478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1013,"num_write_kb":4052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4216","reported_seq":4335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:18.009126+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:18.009126+0000","last_peered":"2024-09-15T21:21:18.009126+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:18.009126+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4216,"log_dups_size":0,"ondisk_log_size":4216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17240547,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2148,"num_write_kb":8592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":123}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4034","reported_seq":4159,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.022851+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:15.022851+0000","last_peered":"2024-09-15T21:21:15.022851+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:15.022851+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4034,"log_dups_size":0,"ondisk_log_size":4034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16490496,"num_objects":4026,"num_object_clones":0,"num_object_copies":8052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":707,"num_write_kb":2828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4192","reported_seq":4305,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:18.008632+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:18.008632+0000","last_peered":"2024-09-15T21:21:18.008632+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:18.008632+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4192,"log_dups_size":0,"ondisk_log_size":4192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17150435,"num_objects":4188,"num_object_clones":0,"num_object_copies":8376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2051,"num_write_kb":8204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":98}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4269","reported_seq":4537,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:18.020360+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:18.020360+0000","last_peered":"2024-09-15T21:21:18.020360+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:18.020360+0000","last_undegraded":"2024-09-15T21:21:18.020360+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":4269,"log_dups_size":0,"ondisk_log_size":4269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17461248,"num_objects":4263,"num_object_clones":0,"num_object_copies":8526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4085,"num_objects_unfound":0,"num_objects_dirty":4263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1962,"num_write_kb":7848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":315392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":178}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4100","reported_seq":4228,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.970102+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:14.970102+0000","last_peered":"2024-09-15T21:21:14.970102+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:14.970102+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4100,"log_dups_size":0,"ondisk_log_size":4100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16785408,"num_objects":4098,"num_object_clones":0,"num_object_copies":8196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1044,"num_write_kb":4176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4154","reported_seq":4281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.015247+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:15.015247+0000","last_peered":"2024-09-15T21:21:15.015247+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:15.015247+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4154,"log_dups_size":0,"ondisk_log_size":4154,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16982016,"num_objects":4146,"num_object_clones":0,"num_object_copies":8292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1063,"num_write_kb":4252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4093","reported_seq":4222,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:14.930199+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:14.930199+0000","last_peered":"2024-09-15T21:21:14.930199+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:14.930199+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4093,"log_dups_size":0,"ondisk_log_size":4093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16748544,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":577,"num_write_kb":2308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4168","reported_seq":4297,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:15.039892+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:15.039892+0000","last_peered":"2024-09-15T21:21:15.039892+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:15.039892+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4168,"log_dups_size":0,"ondisk_log_size":4168,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17023942,"num_objects":4158,"num_object_clones":0,"num_object_copies":8316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4158,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":872,"num_write_kb":3488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":11},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":272765662,"num_objects":66602,"num_object_clones":0,"num_object_copies":133204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":4085,"num_objects_unfound":0,"num_objects_dirty":66602,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":26467,"num_write_kb":105868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":319488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":544874496,"data_stored":544719137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66674,"ondisk_log_size":66674,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407052,"kb_used_data":267068,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93964788,"statfs":{"total":96636764160,"available":96219942912,"internally_reserved":0,"allocated":273477632,"data_stored":273215618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":143322793},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,0,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412052,"kb_used_data":263812,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93959788,"statfs":{"total":96636764160,"available":96214822912,"internally_reserved":0,"allocated":270143488,"data_stored":269877397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151776939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,13],"upper_bound":4096},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65156,"kb_used_data":1516,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94306684,"statfs":{"total":96636764160,"available":96570044416,"internally_reserved":0,"allocated":1552384,"data_stored":1239324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":65138351},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":270139392,"data_stored":270103262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1249280,"data_stored":1166181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":273485824,"data_stored":273449694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:19.739 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 48 20 68741 68721 5.59175 2.36328 0.033573 0.0139626 2024-09-15T21:21:20.737 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 49 20 69402 69382 5.53032 2.58203 0.0448992 0.0141176 2024-09-15T21:21:21.737 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 50 20 70047 70027 5.47011 2.51953 0.0177345 0.0142725 2024-09-15T21:21:22.472 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:22.737 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 51 20 70681 70661 5.41141 2.47656 0.0173736 0.0144256 2024-09-15T21:21:23.029 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:23.031 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:23.059 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-15T21:21:22.693861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":280621809,"num_objects":68522,"num_object_clones":0,"num_object_copies":137044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":4085,"num_objects_unfound":0,"num_objects_dirty":68522,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":28387,"num_write_kb":113542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":319507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68595,"ondisk_log_size":68595,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":898052,"kb_used_data":550356,"kb_used_omap":12,"kb_used_meta":347635,"kb_avail":282217468,"statfs":{"total":289910292480,"available":288990687232,"internally_reserved":0,"allocated":563564544,"data_stored":562723379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":355978243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,1,2,6,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17776640,"num_objects":4340,"num_object_clones":0,"num_object_copies":8680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":-27447,"num_objects_unfound":0,"num_objects_dirty":4340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4340,"num_write_kb":17360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":315392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.354775"},"pg_stats":[{"pgid":"2.f","version":"43'4335","reported_seq":4405,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.048934+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:20.048934+0000","last_peered":"2024-09-15T21:21:20.048934+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:20.048934+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4335,"log_dups_size":0,"ondisk_log_size":4335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17756160,"num_objects":4335,"num_object_clones":0,"num_object_copies":8670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4335,"num_write_kb":17340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":203},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4157","reported_seq":4257,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:17.957469+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:17.957469+0000","last_peered":"2024-09-15T21:21:17.957469+0000","last_clean":"2024-09-15T21:21:17.957469+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:17.957469+0000","last_undegraded":"2024-09-15T21:21:17.957469+0000","last_fullsized":"2024-09-15T21:21:17.957469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4157,"log_dups_size":0,"ondisk_log_size":4157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17003462,"num_objects":4153,"num_object_clones":0,"num_object_copies":8306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4153,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2438,"num_write_kb":9752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4394","reported_seq":4514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.999209+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:17.999209+0000","last_peered":"2024-09-15T21:21:17.999209+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:17.999209+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":4394,"log_dups_size":0,"ondisk_log_size":4394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17981440,"num_objects":4390,"num_object_clones":0,"num_object_copies":8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2494,"num_write_kb":9976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":123}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4167","reported_seq":4232,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:17.977231+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:17.977231+0000","last_peered":"2024-09-15T21:21:17.977231+0000","last_clean":"2024-09-15T21:21:17.977231+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:17.977231+0000","last_undegraded":"2024-09-15T21:21:17.977231+0000","last_fullsized":"2024-09-15T21:21:17.977231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4167,"log_dups_size":0,"ondisk_log_size":4167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17068032,"num_objects":4167,"num_object_clones":0,"num_object_copies":8334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4167,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4288","reported_seq":4415,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:20.017597+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:20.017597+0000","last_peered":"2024-09-15T21:21:20.017597+0000","last_clean":"2024-09-15T21:21:20.017597+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:20.017597+0000","last_undegraded":"2024-09-15T21:21:20.017597+0000","last_fullsized":"2024-09-15T21:21:20.017597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17535459,"num_objects":4282,"num_object_clones":0,"num_object_copies":8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":743,"num_write_kb":2972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4275","reported_seq":4406,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.065234+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:20.065234+0000","last_peered":"2024-09-15T21:21:20.065234+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:20.065234+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4275,"log_dups_size":0,"ondisk_log_size":4275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17490403,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":951,"num_write_kb":3804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":195},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4363","reported_seq":4493,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.055917+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:20.055917+0000","last_peered":"2024-09-15T21:21:20.055917+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:20.055917+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4363,"log_dups_size":0,"ondisk_log_size":4363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17847238,"num_objects":4359,"num_object_clones":0,"num_object_copies":8718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":674,"num_write_kb":2696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":198},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4436","reported_seq":4559,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:20.031526+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:20.031526+0000","last_peered":"2024-09-15T21:21:20.031526+0000","last_clean":"2024-09-15T21:21:20.031526+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:20.031526+0000","last_undegraded":"2024-09-15T21:21:20.031526+0000","last_fullsized":"2024-09-15T21:21:20.031526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":4436,"log_dups_size":0,"ondisk_log_size":4436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18161664,"num_objects":4434,"num_object_clones":0,"num_object_copies":8868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1208,"num_write_kb":4832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4216","reported_seq":4335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:18.009126+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:18.009126+0000","last_peered":"2024-09-15T21:21:18.009126+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:18.009126+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4216,"log_dups_size":0,"ondisk_log_size":4216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17240547,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2148,"num_write_kb":8592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":123}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4216","reported_seq":4341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.014741+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:20.014741+0000","last_peered":"2024-09-15T21:21:20.014741+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:20.014741+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4216,"log_dups_size":0,"ondisk_log_size":4216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17235968,"num_objects":4208,"num_object_clones":0,"num_object_copies":8416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":889,"num_write_kb":3556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":188},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4192","reported_seq":4305,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:18.008632+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:18.008632+0000","last_peered":"2024-09-15T21:21:18.008632+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:18.008632+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4192,"log_dups_size":0,"ondisk_log_size":4192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17150435,"num_objects":4188,"num_object_clones":0,"num_object_copies":8376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2051,"num_write_kb":8204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":98}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4269","reported_seq":4537,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:18.020360+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:18.020360+0000","last_peered":"2024-09-15T21:21:18.020360+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:18.020360+0000","last_undegraded":"2024-09-15T21:21:18.020360+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":4269,"log_dups_size":0,"ondisk_log_size":4269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17461248,"num_objects":4263,"num_object_clones":0,"num_object_copies":8526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4085,"num_objects_unfound":0,"num_objects_dirty":4263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1962,"num_write_kb":7848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":315392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":178}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4287","reported_seq":4415,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:19.981012+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:19.981012+0000","last_peered":"2024-09-15T21:21:19.981012+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:19.981012+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4287,"log_dups_size":0,"ondisk_log_size":4287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17551360,"num_objects":4285,"num_object_clones":0,"num_object_copies":8570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":191},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4352","reported_seq":4479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.037252+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:20.037252+0000","last_peered":"2024-09-15T21:21:20.037252+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:20.037252+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4352,"log_dups_size":0,"ondisk_log_size":4352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17793024,"num_objects":4344,"num_object_clones":0,"num_object_copies":8688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1261,"num_write_kb":5044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":205},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4272","reported_seq":4402,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.038420+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:20.038420+0000","last_peered":"2024-09-15T21:21:20.038420+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:20.038420+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4272,"log_dups_size":0,"ondisk_log_size":4272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17481728,"num_objects":4268,"num_object_clones":0,"num_object_copies":8536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":756,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":186},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4373","reported_seq":4506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.064931+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:20.064931+0000","last_peered":"2024-09-15T21:21:20.064931+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:20.064931+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4373,"log_dups_size":0,"ondisk_log_size":4373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17863622,"num_objects":4363,"num_object_clones":0,"num_object_copies":8726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4363,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1077,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":216},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":280621790,"num_objects":68520,"num_object_clones":0,"num_object_copies":137040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":4085,"num_objects_unfound":0,"num_objects_dirty":68520,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":28385,"num_write_kb":113540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":319488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":563785728,"data_stored":563630369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68592,"ondisk_log_size":68592,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407052,"kb_used_data":267068,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93964788,"statfs":{"total":96636764160,"available":96219942912,"internally_reserved":0,"allocated":273477632,"data_stored":273215618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":143322793},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,0,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408316,"kb_used_data":272556,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93963524,"statfs":{"total":96636764160,"available":96218648576,"internally_reserved":0,"allocated":279097344,"data_stored":278831253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":138997419},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,0,4,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82684,"kb_used_data":10732,"kb_used_omap":4,"kb_used_meta":71931,"kb_avail":94289156,"statfs":{"total":96636764160,"available":96552095744,"internally_reserved":0,"allocated":10989568,"data_stored":10676508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":73658031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":279449600,"data_stored":279413470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10850304,"data_stored":10767205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":273485824,"data_stored":273449694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:23.061 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:23.625 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:23.626 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:23.652 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-09-15T21:21:22.693861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":280621809,"num_objects":68522,"num_object_clones":0,"num_object_copies":137044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":4085,"num_objects_unfound":0,"num_objects_dirty":68522,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":28387,"num_write_kb":113542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":79,"num_bytes_recovered":319507,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68595,"ondisk_log_size":68595,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":62,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":898052,"kb_used_data":550356,"kb_used_omap":12,"kb_used_meta":347635,"kb_avail":282217468,"statfs":{"total":289910292480,"available":288990687232,"internally_reserved":0,"allocated":563564544,"data_stored":562723379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":355978243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,1,2,6,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17776640,"num_objects":4340,"num_object_clones":0,"num_object_copies":8680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":-27447,"num_objects_unfound":0,"num_objects_dirty":4340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4340,"num_write_kb":17360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":315392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.354775"},"pg_stats":[{"pgid":"2.f","version":"43'4335","reported_seq":4405,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.048934+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:20.048934+0000","last_peered":"2024-09-15T21:21:20.048934+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:20.048934+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4335,"log_dups_size":0,"ondisk_log_size":4335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17756160,"num_objects":4335,"num_object_clones":0,"num_object_copies":8670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4335,"num_write_kb":17340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":203},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4157","reported_seq":4257,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:17.957469+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:17.957469+0000","last_peered":"2024-09-15T21:21:17.957469+0000","last_clean":"2024-09-15T21:21:17.957469+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:17.957469+0000","last_undegraded":"2024-09-15T21:21:17.957469+0000","last_fullsized":"2024-09-15T21:21:17.957469+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4157,"log_dups_size":0,"ondisk_log_size":4157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17003462,"num_objects":4153,"num_object_clones":0,"num_object_copies":8306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4153,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2438,"num_write_kb":9752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4394","reported_seq":4514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:17.999209+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:17.999209+0000","last_peered":"2024-09-15T21:21:17.999209+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:17.999209+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":4394,"log_dups_size":0,"ondisk_log_size":4394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17981440,"num_objects":4390,"num_object_clones":0,"num_object_copies":8780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2494,"num_write_kb":9976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":123}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4167","reported_seq":4232,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:17.977231+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:17.977231+0000","last_peered":"2024-09-15T21:21:17.977231+0000","last_clean":"2024-09-15T21:21:17.977231+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:17.977231+0000","last_undegraded":"2024-09-15T21:21:17.977231+0000","last_fullsized":"2024-09-15T21:21:17.977231+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4167,"log_dups_size":0,"ondisk_log_size":4167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17068032,"num_objects":4167,"num_object_clones":0,"num_object_copies":8334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4167,"num_write_kb":16668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4288","reported_seq":4415,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:20.017597+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:20.017597+0000","last_peered":"2024-09-15T21:21:20.017597+0000","last_clean":"2024-09-15T21:21:20.017597+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:20.017597+0000","last_undegraded":"2024-09-15T21:21:20.017597+0000","last_fullsized":"2024-09-15T21:21:20.017597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17535459,"num_objects":4282,"num_object_clones":0,"num_object_copies":8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":743,"num_write_kb":2972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4275","reported_seq":4406,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.065234+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:20.065234+0000","last_peered":"2024-09-15T21:21:20.065234+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:20.065234+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4275,"log_dups_size":0,"ondisk_log_size":4275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17490403,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":951,"num_write_kb":3804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":195},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4363","reported_seq":4493,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.055917+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:20.055917+0000","last_peered":"2024-09-15T21:21:20.055917+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:20.055917+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4363,"log_dups_size":0,"ondisk_log_size":4363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17847238,"num_objects":4359,"num_object_clones":0,"num_object_copies":8718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":674,"num_write_kb":2696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":198},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4436","reported_seq":4559,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:20.031526+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:20.031526+0000","last_peered":"2024-09-15T21:21:20.031526+0000","last_clean":"2024-09-15T21:21:20.031526+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:20.031526+0000","last_undegraded":"2024-09-15T21:21:20.031526+0000","last_fullsized":"2024-09-15T21:21:20.031526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":4436,"log_dups_size":0,"ondisk_log_size":4436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18161664,"num_objects":4434,"num_object_clones":0,"num_object_copies":8868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1208,"num_write_kb":4832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4216","reported_seq":4335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:18.009126+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:18.009126+0000","last_peered":"2024-09-15T21:21:18.009126+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:18.009126+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4216,"log_dups_size":0,"ondisk_log_size":4216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17240547,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2148,"num_write_kb":8592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":123}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4216","reported_seq":4341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.014741+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:20.014741+0000","last_peered":"2024-09-15T21:21:20.014741+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:20.014741+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4216,"log_dups_size":0,"ondisk_log_size":4216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17235968,"num_objects":4208,"num_object_clones":0,"num_object_copies":8416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":889,"num_write_kb":3556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":188},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4192","reported_seq":4305,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:18.008632+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:18.008632+0000","last_peered":"2024-09-15T21:21:18.008632+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:18.008632+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4192,"log_dups_size":0,"ondisk_log_size":4192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17150435,"num_objects":4188,"num_object_clones":0,"num_object_copies":8376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4188,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2051,"num_write_kb":8204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":98}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4269","reported_seq":4537,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:18.020360+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:18.020360+0000","last_peered":"2024-09-15T21:21:18.020360+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:18.020360+0000","last_undegraded":"2024-09-15T21:21:18.020360+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":4269,"log_dups_size":0,"ondisk_log_size":4269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17461248,"num_objects":4263,"num_object_clones":0,"num_object_copies":8526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4085,"num_objects_unfound":0,"num_objects_dirty":4263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1962,"num_write_kb":7848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":77,"num_bytes_recovered":315392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":178}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4287","reported_seq":4415,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:19.981012+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:19.981012+0000","last_peered":"2024-09-15T21:21:19.981012+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:19.981012+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4287,"log_dups_size":0,"ondisk_log_size":4287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17551360,"num_objects":4285,"num_object_clones":0,"num_object_copies":8570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":191},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4352","reported_seq":4479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.037252+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:20.037252+0000","last_peered":"2024-09-15T21:21:20.037252+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:20.037252+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4352,"log_dups_size":0,"ondisk_log_size":4352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17793024,"num_objects":4344,"num_object_clones":0,"num_object_copies":8688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1261,"num_write_kb":5044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":205},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4272","reported_seq":4402,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.038420+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:20.038420+0000","last_peered":"2024-09-15T21:21:20.038420+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:20.038420+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4272,"log_dups_size":0,"ondisk_log_size":4272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17481728,"num_objects":4268,"num_object_clones":0,"num_object_copies":8536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":756,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":186},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4373","reported_seq":4506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.064931+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:20.064931+0000","last_peered":"2024-09-15T21:21:20.064931+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:20.064931+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4373,"log_dups_size":0,"ondisk_log_size":4373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17863622,"num_objects":4363,"num_object_clones":0,"num_object_copies":8726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4363,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1077,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":216},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":280621790,"num_objects":68520,"num_object_clones":0,"num_object_copies":137040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":4085,"num_objects_unfound":0,"num_objects_dirty":68520,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":28385,"num_write_kb":113540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":78,"num_bytes_recovered":319488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":563785728,"data_stored":563630369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68592,"ondisk_log_size":68592,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407052,"kb_used_data":267068,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93964788,"statfs":{"total":96636764160,"available":96219942912,"internally_reserved":0,"allocated":273477632,"data_stored":273215618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":143322793},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,0,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408316,"kb_used_data":272556,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93963524,"statfs":{"total":96636764160,"available":96218648576,"internally_reserved":0,"allocated":279097344,"data_stored":278831253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":138997419},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,0,4,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82684,"kb_used_data":10732,"kb_used_omap":4,"kb_used_meta":71931,"kb_avail":94289156,"statfs":{"total":96636764160,"available":96552095744,"internally_reserved":0,"allocated":10989568,"data_stored":10676508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":73658031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":279449600,"data_stored":279413470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10850304,"data_stored":10767205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":273485824,"data_stored":273449694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:23.653 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:23.653 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'4335", 'reported_seq': 4405, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:20.048934+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:21:20.048934+0000', 'last_peered': '2024-09-15T21:21:20.048934+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:21:20.048934+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 4335, 'log_dups_size': 0, 'ondisk_log_size': 4335, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17756160, 'num_objects': 4335, 'num_object_clones': 0, 'num_object_copies': 8670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4335, 'num_write_kb': 17340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 203}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:23.653 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:21:23.653 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'4394", 'reported_seq': 4514, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:17.999209+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:21:17.999209+0000', 'last_peered': '2024-09-15T21:21:17.999209+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:21:17.999209+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 4394, 'log_dups_size': 0, 'ondisk_log_size': 4394, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17981440, 'num_objects': 4390, 'num_object_clones': 0, 'num_object_copies': 8780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4390, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2494, 'num_write_kb': 9976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 123}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:23.653 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:23.653 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'4275", 'reported_seq': 4406, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:20.065234+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:21:20.065234+0000', 'last_peered': '2024-09-15T21:21:20.065234+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:21:20.065234+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 4275, 'log_dups_size': 0, 'ondisk_log_size': 4275, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17490403, 'num_objects': 4271, 'num_object_clones': 0, 'num_object_copies': 8542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4271, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 951, 'num_write_kb': 3804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 195}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:23.653 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:23.653 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'4363", 'reported_seq': 4493, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:20.055917+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:21:20.055917+0000', 'last_peered': '2024-09-15T21:21:20.055917+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:21:20.055917+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 4363, 'log_dups_size': 0, 'ondisk_log_size': 4363, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17847238, 'num_objects': 4359, 'num_object_clones': 0, 'num_object_copies': 8718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4359, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 674, 'num_write_kb': 2696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 198}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:23.654 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:23.654 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'4216", 'reported_seq': 4335, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:18.009126+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:21:18.009126+0000', 'last_peered': '2024-09-15T21:21:18.009126+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:21:18.009126+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 4216, 'log_dups_size': 0, 'ondisk_log_size': 4216, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17240547, 'num_objects': 4210, 'num_object_clones': 0, 'num_object_copies': 8420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4210, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2148, 'num_write_kb': 8592, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 123}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:23.654 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:23.654 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'4216", 'reported_seq': 4341, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:20.014741+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:21:20.014741+0000', 'last_peered': '2024-09-15T21:21:20.014741+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:21:20.014741+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 4216, 'log_dups_size': 0, 'ondisk_log_size': 4216, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17235968, 'num_objects': 4208, 'num_object_clones': 0, 'num_object_copies': 8416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4208, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 889, 'num_write_kb': 3556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 188}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:23.654 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:23.654 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'4192", 'reported_seq': 4305, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:18.008632+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:21:18.008632+0000', 'last_peered': '2024-09-15T21:21:18.008632+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:21:18.008632+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 4192, 'log_dups_size': 0, 'ondisk_log_size': 4192, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17150435, 'num_objects': 4188, 'num_object_clones': 0, 'num_object_copies': 8376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4188, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2051, 'num_write_kb': 8204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 98}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:23.654 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:23.654 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'4269", 'reported_seq': 4537, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:18.020360+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:21:18.020360+0000', 'last_peered': '2024-09-15T21:21:18.020360+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:21:18.020360+0000', 'last_undegraded': '2024-09-15T21:21:18.020360+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 4269, 'log_dups_size': 0, 'ondisk_log_size': 4269, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17461248, 'num_objects': 4263, 'num_object_clones': 0, 'num_object_copies': 8526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 4085, 'num_objects_unfound': 0, 'num_objects_dirty': 4263, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1962, 'num_write_kb': 7848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 77, 'num_bytes_recovered': 315392, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 178}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:23.654 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:23.654 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'4287", 'reported_seq': 4415, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:19.981012+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:21:19.981012+0000', 'last_peered': '2024-09-15T21:21:19.981012+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:21:19.981012+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 4287, 'log_dups_size': 0, 'ondisk_log_size': 4287, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17551360, 'num_objects': 4285, 'num_object_clones': 0, 'num_object_copies': 8570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4285, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1231, 'num_write_kb': 4924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 191}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:23.654 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:23.655 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'4352", 'reported_seq': 4479, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:20.037252+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:21:20.037252+0000', 'last_peered': '2024-09-15T21:21:20.037252+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:21:20.037252+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 4352, 'log_dups_size': 0, 'ondisk_log_size': 4352, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17793024, 'num_objects': 4344, 'num_object_clones': 0, 'num_object_copies': 8688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4344, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1261, 'num_write_kb': 5044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 205}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:23.655 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:23.655 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'4272", 'reported_seq': 4402, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:20.038420+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:21:20.038420+0000', 'last_peered': '2024-09-15T21:21:20.038420+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:21:20.038420+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 4272, 'log_dups_size': 0, 'ondisk_log_size': 4272, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17481728, 'num_objects': 4268, 'num_object_clones': 0, 'num_object_copies': 8536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4268, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 756, 'num_write_kb': 3024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 186}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:23.655 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:23.655 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'4373", 'reported_seq': 4506, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:20.064931+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:21:20.064931+0000', 'last_peered': '2024-09-15T21:21:20.064931+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:21:20.064931+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 4373, 'log_dups_size': 0, 'ondisk_log_size': 4373, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17863622, 'num_objects': 4363, 'num_object_clones': 0, 'num_object_copies': 8726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4363, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1077, 'num_write_kb': 4308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 216}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:23.655 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:23.737 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 52 20 71256 71236 5.35054 2.24609 0.016056 0.0145934 2024-09-15T21:21:24.331 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:24.331 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/137044 objects degraded (33.062%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":136,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":68522,"data_bytes":280621809,"bytes_used":919605248,"bytes_avail":288990687232,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":137044,"degraded_ratio":0.33061644435363824,"misplaced_objects":4085,"misplaced_total":137044,"misplaced_ratio":0.02980794489361081,"recovering_objects_per_sec":8,"recovering_bytes_per_sec":33714,"recovering_keys_per_sec":0,"num_objects_recovered":77,"num_bytes_recovered":315392,"num_keys_recovered":0,"write_bytes_sec":1900274,"read_op_per_sec":0,"write_op_per_sec":463},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-15T21:21:18.693002+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:21:24.356 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:24.357 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:24.737 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 53 20 71821 71801 5.29122 2.20703 0.0269767 0.0147543 2024-09-15T21:21:24.897 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:24.900 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:24.927 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-15T21:21:24.694286+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":285393649,"num_objects":69687,"num_object_clones":0,"num_object_copies":139374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3970,"num_objects_unfound":0,"num_objects_dirty":69687,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":29552,"num_write_kb":118202,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":790547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69760,"ondisk_log_size":69760,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":908396,"kb_used_data":556540,"kb_used_omap":12,"kb_used_meta":351795,"kb_avail":282207124,"statfs":{"total":289910292480,"available":288980094976,"internally_reserved":0,"allocated":569896960,"data_stored":569055795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":360238083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,6,1,2,6,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22548480,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":-27562,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5505,"num_write_kb":22020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.347114"},"pg_stats":[{"pgid":"2.f","version":"43'4335","reported_seq":4405,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.048934+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:20.048934+0000","last_peered":"2024-09-15T21:21:20.048934+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:20.048934+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4335,"log_dups_size":0,"ondisk_log_size":4335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17756160,"num_objects":4335,"num_object_clones":0,"num_object_copies":8670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4335,"num_write_kb":17340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":203},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4363","reported_seq":4463,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:23.025127+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:23.025127+0000","last_peered":"2024-09-15T21:21:23.025127+0000","last_clean":"2024-09-15T21:21:23.025127+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:23.025127+0000","last_undegraded":"2024-09-15T21:21:23.025127+0000","last_fullsized":"2024-09-15T21:21:23.025127+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4363,"log_dups_size":0,"ondisk_log_size":4363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17847238,"num_objects":4359,"num_object_clones":0,"num_object_copies":8718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4359,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2644,"num_write_kb":10576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4577","reported_seq":4697,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:23.028524+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:23.028524+0000","last_peered":"2024-09-15T21:21:23.028524+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:23.028524+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18731008,"num_objects":4573,"num_object_clones":0,"num_object_copies":9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2677,"num_write_kb":10708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4352","reported_seq":4416,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:23.071810+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:23.071810+0000","last_peered":"2024-09-15T21:21:23.071810+0000","last_clean":"2024-09-15T21:21:23.071810+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:23.071810+0000","last_undegraded":"2024-09-15T21:21:23.071810+0000","last_fullsized":"2024-09-15T21:21:23.071810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4352,"log_dups_size":0,"ondisk_log_size":4352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4352,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4288","reported_seq":4415,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:20.017597+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:20.017597+0000","last_peered":"2024-09-15T21:21:20.017597+0000","last_clean":"2024-09-15T21:21:20.017597+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:20.017597+0000","last_undegraded":"2024-09-15T21:21:20.017597+0000","last_fullsized":"2024-09-15T21:21:20.017597+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17535459,"num_objects":4282,"num_object_clones":0,"num_object_copies":8564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":743,"num_write_kb":2972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4275","reported_seq":4406,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.065234+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:20.065234+0000","last_peered":"2024-09-15T21:21:20.065234+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:20.065234+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4275,"log_dups_size":0,"ondisk_log_size":4275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17490403,"num_objects":4271,"num_object_clones":0,"num_object_copies":8542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4271,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":951,"num_write_kb":3804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":195},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4363","reported_seq":4493,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.055917+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:20.055917+0000","last_peered":"2024-09-15T21:21:20.055917+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:20.055917+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4363,"log_dups_size":0,"ondisk_log_size":4363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17847238,"num_objects":4359,"num_object_clones":0,"num_object_copies":8718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":674,"num_write_kb":2696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":198},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4436","reported_seq":4559,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:20.031526+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:20.031526+0000","last_peered":"2024-09-15T21:21:20.031526+0000","last_clean":"2024-09-15T21:21:20.031526+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:20.031526+0000","last_undegraded":"2024-09-15T21:21:20.031526+0000","last_fullsized":"2024-09-15T21:21:20.031526+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":4436,"log_dups_size":0,"ondisk_log_size":4436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18161664,"num_objects":4434,"num_object_clones":0,"num_object_copies":8868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1208,"num_write_kb":4832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4412","reported_seq":4533,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:23.070237+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:23.070237+0000","last_peered":"2024-09-15T21:21:23.070237+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:23.070237+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4412,"log_dups_size":0,"ondisk_log_size":4412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18043363,"num_objects":4406,"num_object_clones":0,"num_object_copies":8812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2344,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4216","reported_seq":4341,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.014741+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:20.014741+0000","last_peered":"2024-09-15T21:21:20.014741+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:20.014741+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4216,"log_dups_size":0,"ondisk_log_size":4216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17235968,"num_objects":4208,"num_object_clones":0,"num_object_copies":8416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":889,"num_write_kb":3556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":188},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4396","reported_seq":4509,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.994064+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:22.994064+0000","last_peered":"2024-09-15T21:21:22.994064+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:22.994064+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4396,"log_dups_size":0,"ondisk_log_size":4396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17986019,"num_objects":4392,"num_object_clones":0,"num_object_copies":8784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2255,"num_write_kb":9020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":302}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4460","reported_seq":4962,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:23.059774+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:23.059774+0000","last_peered":"2024-09-15T21:21:23.059774+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:23.059774+0000","last_undegraded":"2024-09-15T21:21:23.059774+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18243584,"num_objects":4454,"num_object_clones":0,"num_object_copies":8908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3970,"num_objects_unfound":0,"num_objects_dirty":4454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2153,"num_write_kb":8612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3970},{"shards":"2","objects":484}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4287","reported_seq":4415,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:19.981012+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:19.981012+0000","last_peered":"2024-09-15T21:21:19.981012+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:19.981012+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4287,"log_dups_size":0,"ondisk_log_size":4287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17551360,"num_objects":4285,"num_object_clones":0,"num_object_copies":8570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1231,"num_write_kb":4924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":191},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4352","reported_seq":4479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.037252+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:20.037252+0000","last_peered":"2024-09-15T21:21:20.037252+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:20.037252+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4352,"log_dups_size":0,"ondisk_log_size":4352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17793024,"num_objects":4344,"num_object_clones":0,"num_object_copies":8688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1261,"num_write_kb":5044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":205},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4272","reported_seq":4402,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.038420+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:20.038420+0000","last_peered":"2024-09-15T21:21:20.038420+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:20.038420+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4272,"log_dups_size":0,"ondisk_log_size":4272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17481728,"num_objects":4268,"num_object_clones":0,"num_object_copies":8536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":756,"num_write_kb":3024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":186},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4373","reported_seq":4506,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:20.064931+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:20.064931+0000","last_peered":"2024-09-15T21:21:20.064931+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:20.064931+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4373,"log_dups_size":0,"ondisk_log_size":4373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17863622,"num_objects":4363,"num_object_clones":0,"num_object_copies":8726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4363,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1077,"num_write_kb":4308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":216},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":285393630,"num_objects":69685,"num_object_clones":0,"num_object_copies":139370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3970,"num_objects_unfound":0,"num_objects_dirty":69685,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":29550,"num_write_kb":118200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":790528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":570216448,"data_stored":570061089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69757,"ondisk_log_size":69757,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417396,"kb_used_data":273252,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93954444,"statfs":{"total":96636764160,"available":96209350656,"internally_reserved":0,"allocated":279810048,"data_stored":279548034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":147582633},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":408316,"kb_used_data":272556,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":93963524,"statfs":{"total":96636764160,"available":96218648576,"internally_reserved":0,"allocated":279097344,"data_stored":278831253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":138997419},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,0,4,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82684,"kb_used_data":10732,"kb_used_omap":4,"kb_used_meta":71931,"kb_avail":94289156,"statfs":{"total":96636764160,"available":96552095744,"internally_reserved":0,"allocated":10989568,"data_stored":10676508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":73658031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":279449600,"data_stored":279413470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10850304,"data_stored":10767205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":279916544,"data_stored":279880414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:25.737 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 54 20 72452 72432 5.23888 2.46484 0.0188552 0.0149039 2024-09-15T21:21:26.737 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 55 20 73097 73077 5.18944 2.51953 0.00749498 0.0150453 2024-09-15T21:21:27.737 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 56 20 73740 73720 5.14162 2.51172 0.0445288 0.0151854 2024-09-15T21:21:27.929 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:28.463 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:28.464 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:28.496 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-15T21:21:26.694750+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293221105,"num_objects":71598,"num_object_clones":0,"num_object_copies":143196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3970,"num_objects_unfound":0,"num_objects_dirty":71598,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":31463,"num_write_kb":125846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":790547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71671,"ondisk_log_size":71671,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":940048,"kb_used_data":575712,"kb_used_omap":12,"kb_used_meta":364275,"kb_avail":282175472,"statfs":{"total":289910292480,"available":288947683328,"internally_reserved":0,"allocated":589529088,"data_stored":588687923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":373017603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,2,7,10,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30375936,"num_objects":7416,"num_object_clones":0,"num_object_copies":14832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":-27562,"num_objects_unfound":0,"num_objects_dirty":7416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7416,"num_write_kb":29664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004818"},"pg_stats":[{"pgid":"2.f","version":"43'4516","reported_seq":4586,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.062933+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:25.062933+0000","last_peered":"2024-09-15T21:21:25.062933+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:25.062933+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4516,"log_dups_size":0,"ondisk_log_size":4516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18497536,"num_objects":4516,"num_object_clones":0,"num_object_copies":9032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4516,"num_write_kb":18064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":384},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4363","reported_seq":4463,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:23.025127+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:23.025127+0000","last_peered":"2024-09-15T21:21:23.025127+0000","last_clean":"2024-09-15T21:21:23.025127+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:23.025127+0000","last_undegraded":"2024-09-15T21:21:23.025127+0000","last_fullsized":"2024-09-15T21:21:23.025127+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4363,"log_dups_size":0,"ondisk_log_size":4363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17847238,"num_objects":4359,"num_object_clones":0,"num_object_copies":8718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4359,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2644,"num_write_kb":10576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4577","reported_seq":4697,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:23.028524+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:23.028524+0000","last_peered":"2024-09-15T21:21:23.028524+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:23.028524+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18731008,"num_objects":4573,"num_object_clones":0,"num_object_copies":9146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2677,"num_write_kb":10708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4352","reported_seq":4416,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:23.071810+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:23.071810+0000","last_peered":"2024-09-15T21:21:23.071810+0000","last_clean":"2024-09-15T21:21:23.071810+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:23.071810+0000","last_undegraded":"2024-09-15T21:21:23.071810+0000","last_fullsized":"2024-09-15T21:21:23.071810+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4352,"log_dups_size":0,"ondisk_log_size":4352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17825792,"num_objects":4352,"num_object_clones":0,"num_object_copies":8704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4352,"num_write_kb":17408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4463","reported_seq":4589,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:25.088055+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:25.088055+0000","last_peered":"2024-09-15T21:21:25.088055+0000","last_clean":"2024-09-15T21:21:25.088055+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:25.088055+0000","last_undegraded":"2024-09-15T21:21:25.088055+0000","last_fullsized":"2024-09-15T21:21:25.088055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4463,"log_dups_size":0,"ondisk_log_size":4463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18252259,"num_objects":4457,"num_object_clones":0,"num_object_copies":8914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":918,"num_write_kb":3672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4465","reported_seq":4596,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.908565+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:24.908565+0000","last_peered":"2024-09-15T21:21:24.908565+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:24.908565+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4465,"log_dups_size":0,"ondisk_log_size":4465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18268643,"num_objects":4461,"num_object_clones":0,"num_object_copies":8922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1141,"num_write_kb":4564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":385},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4548","reported_seq":4678,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.085249+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:25.085249+0000","last_peered":"2024-09-15T21:21:25.085249+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:25.085249+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4548,"log_dups_size":0,"ondisk_log_size":4548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604998,"num_objects":4544,"num_object_clones":0,"num_object_copies":9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":859,"num_write_kb":3436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":383},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4632","reported_seq":4755,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:25.081795+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:25.081795+0000","last_peered":"2024-09-15T21:21:25.081795+0000","last_clean":"2024-09-15T21:21:25.081795+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:25.081795+0000","last_undegraded":"2024-09-15T21:21:25.081795+0000","last_fullsized":"2024-09-15T21:21:25.081795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":4632,"log_dups_size":0,"ondisk_log_size":4632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18964480,"num_objects":4630,"num_object_clones":0,"num_object_copies":9260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1404,"num_write_kb":5616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4412","reported_seq":4533,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:23.070237+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:23.070237+0000","last_peered":"2024-09-15T21:21:23.070237+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:23.070237+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4412,"log_dups_size":0,"ondisk_log_size":4412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18043363,"num_objects":4406,"num_object_clones":0,"num_object_copies":8812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2344,"num_write_kb":9376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":319}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4410","reported_seq":4535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.944191+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:24.944191+0000","last_peered":"2024-09-15T21:21:24.944191+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:24.944191+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4410,"log_dups_size":0,"ondisk_log_size":4410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18030592,"num_objects":4402,"num_object_clones":0,"num_object_copies":8804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1083,"num_write_kb":4332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":382},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4396","reported_seq":4509,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:22.994064+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:22.994064+0000","last_peered":"2024-09-15T21:21:22.994064+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:22.994064+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4396,"log_dups_size":0,"ondisk_log_size":4396,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17986019,"num_objects":4392,"num_object_clones":0,"num_object_copies":8784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2255,"num_write_kb":9020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":302}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4460","reported_seq":4962,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:23.059774+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:23.059774+0000","last_peered":"2024-09-15T21:21:23.059774+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:23.059774+0000","last_undegraded":"2024-09-15T21:21:23.059774+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":4460,"log_dups_size":0,"ondisk_log_size":4460,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18243584,"num_objects":4454,"num_object_clones":0,"num_object_copies":8908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3970,"num_objects_unfound":0,"num_objects_dirty":4454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2153,"num_write_kb":8612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":786432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3970},{"shards":"2","objects":484}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4501","reported_seq":4629,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.054283+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:25.054283+0000","last_peered":"2024-09-15T21:21:25.054283+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:25.054283+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4501,"log_dups_size":0,"ondisk_log_size":4501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18427904,"num_objects":4499,"num_object_clones":0,"num_object_copies":8998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1445,"num_write_kb":5780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":405},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4546","reported_seq":4673,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.053695+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:25.053695+0000","last_peered":"2024-09-15T21:21:25.053695+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:25.053695+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4546,"log_dups_size":0,"ondisk_log_size":4546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18587648,"num_objects":4538,"num_object_clones":0,"num_object_copies":9076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1455,"num_write_kb":5820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":399},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4468","reported_seq":4598,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.091040+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:25.091040+0000","last_peered":"2024-09-15T21:21:25.091040+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:25.091040+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4468,"log_dups_size":0,"ondisk_log_size":4468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18284544,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":3808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":382},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4559","reported_seq":4692,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.057554+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:25.057554+0000","last_peered":"2024-09-15T21:21:25.057554+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:25.057554+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4559,"log_dups_size":0,"ondisk_log_size":4559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18625478,"num_objects":4549,"num_object_clones":0,"num_object_copies":9098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4549,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1263,"num_write_kb":5052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":402},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293221086,"num_objects":71596,"num_object_clones":0,"num_object_copies":143192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3970,"num_objects_unfound":0,"num_objects_dirty":71596,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":31461,"num_write_kb":125844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":790528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589639680,"data_stored":589484321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71668,"ondisk_log_size":71668,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417396,"kb_used_data":273252,"kb_used_omap":4,"kb_used_meta":144123,"kb_avail":93954444,"statfs":{"total":96636764160,"available":96209350656,"internally_reserved":0,"allocated":279810048,"data_stored":279548034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":147582633},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426040,"kb_used_data":281960,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93945800,"statfs":{"total":96636764160,"available":96200499200,"internally_reserved":0,"allocated":288727040,"data_stored":288460949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147517099},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,6,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96612,"kb_used_data":20500,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94275228,"statfs":{"total":96636764160,"available":96537833472,"internally_reserved":0,"allocated":20992000,"data_stored":20678940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":77917871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":288866304,"data_stored":288830174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20856832,"data_stored":20773733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":279916544,"data_stored":279880414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:28.497 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:28.737 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 57 20 74324 74304 5.09144 2.28125 0.0228942 0.0153352 2024-09-15T21:21:29.040 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:29.043 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:29.075 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-15T21:21:28.695196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":297902833,"num_objects":72741,"num_object_clones":0,"num_object_copies":145482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":72741,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":32606,"num_write_kb":130418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":1306643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72814,"ondisk_log_size":72814,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":950408,"kb_used_data":581912,"kb_used_omap":12,"kb_used_meta":368435,"kb_avail":282165112,"statfs":{"total":289910292480,"available":288937074688,"internally_reserved":0,"allocated":595877888,"data_stored":595036723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":377277443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,2,7,10,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27889664,"num_objects":6809,"num_object_clones":0,"num_object_copies":13618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12447,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":6809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6809,"num_write_kb":27236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004958"},"pg_stats":[{"pgid":"2.f","version":"43'4516","reported_seq":4586,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.062933+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:25.062933+0000","last_peered":"2024-09-15T21:21:25.062933+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:25.062933+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4516,"log_dups_size":0,"ondisk_log_size":4516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18497536,"num_objects":4516,"num_object_clones":0,"num_object_copies":9032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4516,"num_write_kb":18064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":384},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4554","reported_seq":4654,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:28.083072+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:28.083072+0000","last_peered":"2024-09-15T21:21:28.083072+0000","last_clean":"2024-09-15T21:21:28.083072+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:28.083072+0000","last_undegraded":"2024-09-15T21:21:28.083072+0000","last_fullsized":"2024-09-15T21:21:28.083072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4554,"log_dups_size":0,"ondisk_log_size":4554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18629574,"num_objects":4550,"num_object_clones":0,"num_object_copies":9100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4550,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2835,"num_write_kb":11340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4777","reported_seq":4894,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:28.065499+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:28.065499+0000","last_peered":"2024-09-15T21:21:28.065499+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:28.065499+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":4777,"log_dups_size":0,"ondisk_log_size":4777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550208,"num_objects":4773,"num_object_clones":0,"num_object_copies":9546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2877,"num_write_kb":11508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":506}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4533","reported_seq":4598,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:28.062630+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:28.062630+0000","last_peered":"2024-09-15T21:21:28.062630+0000","last_clean":"2024-09-15T21:21:28.062630+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:28.062630+0000","last_undegraded":"2024-09-15T21:21:28.062630+0000","last_fullsized":"2024-09-15T21:21:28.062630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4533,"log_dups_size":0,"ondisk_log_size":4533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18567168,"num_objects":4533,"num_object_clones":0,"num_object_copies":9066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4463","reported_seq":4589,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:25.088055+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:25.088055+0000","last_peered":"2024-09-15T21:21:25.088055+0000","last_clean":"2024-09-15T21:21:25.088055+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:25.088055+0000","last_undegraded":"2024-09-15T21:21:25.088055+0000","last_fullsized":"2024-09-15T21:21:25.088055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4463,"log_dups_size":0,"ondisk_log_size":4463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18252259,"num_objects":4457,"num_object_clones":0,"num_object_copies":8914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":918,"num_write_kb":3672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4465","reported_seq":4596,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.908565+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:24.908565+0000","last_peered":"2024-09-15T21:21:24.908565+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:24.908565+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4465,"log_dups_size":0,"ondisk_log_size":4465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18268643,"num_objects":4461,"num_object_clones":0,"num_object_copies":8922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1141,"num_write_kb":4564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":385},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4548","reported_seq":4678,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.085249+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:25.085249+0000","last_peered":"2024-09-15T21:21:25.085249+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:25.085249+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4548,"log_dups_size":0,"ondisk_log_size":4548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604998,"num_objects":4544,"num_object_clones":0,"num_object_copies":9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":859,"num_write_kb":3436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":383},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4632","reported_seq":4755,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:25.081795+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:25.081795+0000","last_peered":"2024-09-15T21:21:25.081795+0000","last_clean":"2024-09-15T21:21:25.081795+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:25.081795+0000","last_undegraded":"2024-09-15T21:21:25.081795+0000","last_fullsized":"2024-09-15T21:21:25.081795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":4632,"log_dups_size":0,"ondisk_log_size":4632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18964480,"num_objects":4630,"num_object_clones":0,"num_object_copies":9260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1404,"num_write_kb":5616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4602","reported_seq":4723,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:28.062074+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:28.062074+0000","last_peered":"2024-09-15T21:21:28.062074+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:28.062074+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18821603,"num_objects":4596,"num_object_clones":0,"num_object_copies":9192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":509}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4410","reported_seq":4535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.944191+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:24.944191+0000","last_peered":"2024-09-15T21:21:24.944191+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:24.944191+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4410,"log_dups_size":0,"ondisk_log_size":4410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18030592,"num_objects":4402,"num_object_clones":0,"num_object_copies":8804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1083,"num_write_kb":4332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":382},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4599","reported_seq":4712,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:28.082545+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:28.082545+0000","last_peered":"2024-09-15T21:21:28.082545+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:28.082545+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4599,"log_dups_size":0,"ondisk_log_size":4599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18817507,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2458,"num_write_kb":9832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":505}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4638","reported_seq":5392,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:28.101082+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:28.101082+0000","last_peered":"2024-09-15T21:21:28.101082+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:28.101082+0000","last_undegraded":"2024-09-15T21:21:28.101082+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":4638,"log_dups_size":0,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18972672,"num_objects":4632,"num_object_clones":0,"num_object_copies":9264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":4632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2331,"num_write_kb":9324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3844},{"shards":"2","objects":788}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4501","reported_seq":4629,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.054283+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:25.054283+0000","last_peered":"2024-09-15T21:21:25.054283+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:25.054283+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4501,"log_dups_size":0,"ondisk_log_size":4501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18427904,"num_objects":4499,"num_object_clones":0,"num_object_copies":8998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1445,"num_write_kb":5780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":405},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4546","reported_seq":4673,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.053695+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:25.053695+0000","last_peered":"2024-09-15T21:21:25.053695+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:25.053695+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4546,"log_dups_size":0,"ondisk_log_size":4546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18587648,"num_objects":4538,"num_object_clones":0,"num_object_copies":9076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1455,"num_write_kb":5820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":399},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4468","reported_seq":4598,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.091040+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:25.091040+0000","last_peered":"2024-09-15T21:21:25.091040+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:25.091040+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4468,"log_dups_size":0,"ondisk_log_size":4468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18284544,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":3808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":382},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4559","reported_seq":4692,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.057554+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:25.057554+0000","last_peered":"2024-09-15T21:21:25.057554+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:25.057554+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4559,"log_dups_size":0,"ondisk_log_size":4559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18625478,"num_objects":4549,"num_object_clones":0,"num_object_copies":9098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4549,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1263,"num_write_kb":5052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":402},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":297902814,"num_objects":72739,"num_object_clones":0,"num_object_copies":145478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":72739,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":32604,"num_write_kb":130416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":1306624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":595922944,"data_stored":595767585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72811,"ondisk_log_size":72811,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427756,"kb_used_data":279452,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93944084,"statfs":{"total":96636764160,"available":96198742016,"internally_reserved":0,"allocated":286158848,"data_stored":285896834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":151842473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426040,"kb_used_data":281960,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93945800,"statfs":{"total":96636764160,"available":96200499200,"internally_reserved":0,"allocated":288727040,"data_stored":288460949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147517099},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,6,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96612,"kb_used_data":20500,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94275228,"statfs":{"total":96636764160,"available":96537833472,"internally_reserved":0,"allocated":20992000,"data_stored":20678940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":77917871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":288866304,"data_stored":288830174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20856832,"data_stored":20773733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":286199808,"data_stored":286163678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:29.076 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:29.076 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'4516", 'reported_seq': 4586, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:25.062933+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:21:25.062933+0000', 'last_peered': '2024-09-15T21:21:25.062933+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:21:25.062933+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 4516, 'log_dups_size': 0, 'ondisk_log_size': 4516, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18497536, 'num_objects': 4516, 'num_object_clones': 0, 'num_object_copies': 9032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4516, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4516, 'num_write_kb': 18064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 384}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:29.076 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:21:29.076 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'4777", 'reported_seq': 4894, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:28.065499+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:21:28.065499+0000', 'last_peered': '2024-09-15T21:21:28.065499+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:21:28.065499+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 4777, 'log_dups_size': 0, 'ondisk_log_size': 4777, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19550208, 'num_objects': 4773, 'num_object_clones': 0, 'num_object_copies': 9546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4773, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2877, 'num_write_kb': 11508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 506}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:29.076 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:29.076 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'4465", 'reported_seq': 4596, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:24.908565+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:21:24.908565+0000', 'last_peered': '2024-09-15T21:21:24.908565+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:21:24.908565+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 4465, 'log_dups_size': 0, 'ondisk_log_size': 4465, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18268643, 'num_objects': 4461, 'num_object_clones': 0, 'num_object_copies': 8922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4461, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1141, 'num_write_kb': 4564, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 385}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:29.077 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:29.077 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'4548", 'reported_seq': 4678, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:25.085249+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:21:25.085249+0000', 'last_peered': '2024-09-15T21:21:25.085249+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:21:25.085249+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 4548, 'log_dups_size': 0, 'ondisk_log_size': 4548, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18604998, 'num_objects': 4544, 'num_object_clones': 0, 'num_object_copies': 9088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 859, 'num_write_kb': 3436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 383}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:29.077 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:29.077 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'4602", 'reported_seq': 4723, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:28.062074+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:21:28.062074+0000', 'last_peered': '2024-09-15T21:21:28.062074+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:21:28.062074+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 4602, 'log_dups_size': 0, 'ondisk_log_size': 4602, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18821603, 'num_objects': 4596, 'num_object_clones': 0, 'num_object_copies': 9192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4596, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2534, 'num_write_kb': 10136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 509}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:29.077 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:29.077 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'4410", 'reported_seq': 4535, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:24.944191+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:21:24.944191+0000', 'last_peered': '2024-09-15T21:21:24.944191+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:21:24.944191+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 4410, 'log_dups_size': 0, 'ondisk_log_size': 4410, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18030592, 'num_objects': 4402, 'num_object_clones': 0, 'num_object_copies': 8804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4402, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1083, 'num_write_kb': 4332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 382}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:29.077 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:29.077 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'4599", 'reported_seq': 4712, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:28.082545+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:21:28.082545+0000', 'last_peered': '2024-09-15T21:21:28.082545+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:21:28.082545+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 4599, 'log_dups_size': 0, 'ondisk_log_size': 4599, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18817507, 'num_objects': 4595, 'num_object_clones': 0, 'num_object_copies': 9190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4595, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2458, 'num_write_kb': 9832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 505}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:29.077 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:29.077 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'4638", 'reported_seq': 5392, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:28.101082+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:21:28.101082+0000', 'last_peered': '2024-09-15T21:21:28.101082+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:21:28.101082+0000', 'last_undegraded': '2024-09-15T21:21:28.101082+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 4638, 'log_dups_size': 0, 'ondisk_log_size': 4638, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18972672, 'num_objects': 4632, 'num_object_clones': 0, 'num_object_copies': 9264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3844, 'num_objects_unfound': 0, 'num_objects_dirty': 4632, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2331, 'num_write_kb': 9324, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 318, 'num_bytes_recovered': 1302528, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3844}, {'shards': '2', 'objects': 788}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:29.078 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:29.078 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'4501", 'reported_seq': 4629, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:25.054283+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:21:25.054283+0000', 'last_peered': '2024-09-15T21:21:25.054283+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:21:25.054283+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 4501, 'log_dups_size': 0, 'ondisk_log_size': 4501, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18427904, 'num_objects': 4499, 'num_object_clones': 0, 'num_object_copies': 8998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4499, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1445, 'num_write_kb': 5780, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 405}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:29.078 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:29.078 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'4546", 'reported_seq': 4673, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:25.053695+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:21:25.053695+0000', 'last_peered': '2024-09-15T21:21:25.053695+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:21:25.053695+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 4546, 'log_dups_size': 0, 'ondisk_log_size': 4546, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18587648, 'num_objects': 4538, 'num_object_clones': 0, 'num_object_copies': 9076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4538, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1455, 'num_write_kb': 5820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 399}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:29.078 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:29.078 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'4468", 'reported_seq': 4598, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:25.091040+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:21:25.091040+0000', 'last_peered': '2024-09-15T21:21:25.091040+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:21:25.091040+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 4468, 'log_dups_size': 0, 'ondisk_log_size': 4468, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18284544, 'num_objects': 4464, 'num_object_clones': 0, 'num_object_copies': 8928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 952, 'num_write_kb': 3808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 382}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:29.078 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:29.078 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'4559", 'reported_seq': 4692, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:25.057554+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:21:25.057554+0000', 'last_peered': '2024-09-15T21:21:25.057554+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:21:25.057554+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 4559, 'log_dups_size': 0, 'ondisk_log_size': 4559, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18625478, 'num_objects': 4549, 'num_object_clones': 0, 'num_object_copies': 9098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4549, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1263, 'num_write_kb': 5052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 402}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:29.078 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:29.737 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 58 20 74885 74865 5.04143 2.19141 0.0757244 0.0154851 2024-09-15T21:21:29.770 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:29.771 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/145482 objects degraded (31.144%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":141,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":72741,"data_bytes":297902833,"bytes_used":973217792,"bytes_avail":288937074688,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":145482,"degraded_ratio":0.31144059058852641,"misplaced_objects":3844,"misplaced_total":145482,"misplaced_ratio":0.026422512750718302,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":108499,"recovering_keys_per_sec":0,"num_objects_recovered":318,"num_bytes_recovered":1302528,"num_keys_recovered":0,"write_bytes_sec":2323178,"read_op_per_sec":0,"write_op_per_sec":567},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-15T21:21:18.693002+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:21:29.801 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:29.801 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:30.352 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:30.353 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:30.379 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-15T21:21:28.695196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":297902833,"num_objects":72741,"num_object_clones":0,"num_object_copies":145482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":72741,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":32606,"num_write_kb":130418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":1306643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72814,"ondisk_log_size":72814,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":950408,"kb_used_data":581912,"kb_used_omap":12,"kb_used_meta":368435,"kb_avail":282165112,"statfs":{"total":289910292480,"available":288937074688,"internally_reserved":0,"allocated":595877888,"data_stored":595036723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":377277443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,2,7,10,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27889664,"num_objects":6809,"num_object_clones":0,"num_object_copies":13618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12447,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":6809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6809,"num_write_kb":27236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004958"},"pg_stats":[{"pgid":"2.f","version":"43'4516","reported_seq":4586,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.062933+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:25.062933+0000","last_peered":"2024-09-15T21:21:25.062933+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:25.062933+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4516,"log_dups_size":0,"ondisk_log_size":4516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18497536,"num_objects":4516,"num_object_clones":0,"num_object_copies":9032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4516,"num_write_kb":18064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":384},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4554","reported_seq":4654,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:28.083072+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:28.083072+0000","last_peered":"2024-09-15T21:21:28.083072+0000","last_clean":"2024-09-15T21:21:28.083072+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:28.083072+0000","last_undegraded":"2024-09-15T21:21:28.083072+0000","last_fullsized":"2024-09-15T21:21:28.083072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4554,"log_dups_size":0,"ondisk_log_size":4554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18629574,"num_objects":4550,"num_object_clones":0,"num_object_copies":9100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4550,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2835,"num_write_kb":11340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4777","reported_seq":4894,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:28.065499+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:28.065499+0000","last_peered":"2024-09-15T21:21:28.065499+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:28.065499+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":4777,"log_dups_size":0,"ondisk_log_size":4777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550208,"num_objects":4773,"num_object_clones":0,"num_object_copies":9546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2877,"num_write_kb":11508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":506}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4533","reported_seq":4598,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:28.062630+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:28.062630+0000","last_peered":"2024-09-15T21:21:28.062630+0000","last_clean":"2024-09-15T21:21:28.062630+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:28.062630+0000","last_undegraded":"2024-09-15T21:21:28.062630+0000","last_fullsized":"2024-09-15T21:21:28.062630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4533,"log_dups_size":0,"ondisk_log_size":4533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18567168,"num_objects":4533,"num_object_clones":0,"num_object_copies":9066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4463","reported_seq":4589,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:25.088055+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:25.088055+0000","last_peered":"2024-09-15T21:21:25.088055+0000","last_clean":"2024-09-15T21:21:25.088055+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:25.088055+0000","last_undegraded":"2024-09-15T21:21:25.088055+0000","last_fullsized":"2024-09-15T21:21:25.088055+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4463,"log_dups_size":0,"ondisk_log_size":4463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18252259,"num_objects":4457,"num_object_clones":0,"num_object_copies":8914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":918,"num_write_kb":3672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4465","reported_seq":4596,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.908565+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:24.908565+0000","last_peered":"2024-09-15T21:21:24.908565+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:24.908565+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4465,"log_dups_size":0,"ondisk_log_size":4465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18268643,"num_objects":4461,"num_object_clones":0,"num_object_copies":8922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1141,"num_write_kb":4564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":385},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4548","reported_seq":4678,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.085249+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:25.085249+0000","last_peered":"2024-09-15T21:21:25.085249+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:25.085249+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4548,"log_dups_size":0,"ondisk_log_size":4548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604998,"num_objects":4544,"num_object_clones":0,"num_object_copies":9088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":859,"num_write_kb":3436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":383},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4632","reported_seq":4755,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:25.081795+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:25.081795+0000","last_peered":"2024-09-15T21:21:25.081795+0000","last_clean":"2024-09-15T21:21:25.081795+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:25.081795+0000","last_undegraded":"2024-09-15T21:21:25.081795+0000","last_fullsized":"2024-09-15T21:21:25.081795+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":4632,"log_dups_size":0,"ondisk_log_size":4632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18964480,"num_objects":4630,"num_object_clones":0,"num_object_copies":9260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1404,"num_write_kb":5616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4602","reported_seq":4723,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:28.062074+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:28.062074+0000","last_peered":"2024-09-15T21:21:28.062074+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:28.062074+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18821603,"num_objects":4596,"num_object_clones":0,"num_object_copies":9192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":509}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4410","reported_seq":4535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:24.944191+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:24.944191+0000","last_peered":"2024-09-15T21:21:24.944191+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:24.944191+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4410,"log_dups_size":0,"ondisk_log_size":4410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18030592,"num_objects":4402,"num_object_clones":0,"num_object_copies":8804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1083,"num_write_kb":4332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":382},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4599","reported_seq":4712,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:28.082545+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:28.082545+0000","last_peered":"2024-09-15T21:21:28.082545+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:28.082545+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4599,"log_dups_size":0,"ondisk_log_size":4599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18817507,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2458,"num_write_kb":9832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":505}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4638","reported_seq":5392,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:28.101082+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:28.101082+0000","last_peered":"2024-09-15T21:21:28.101082+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:28.101082+0000","last_undegraded":"2024-09-15T21:21:28.101082+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":4638,"log_dups_size":0,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18972672,"num_objects":4632,"num_object_clones":0,"num_object_copies":9264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":4632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2331,"num_write_kb":9324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3844},{"shards":"2","objects":788}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4501","reported_seq":4629,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.054283+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:25.054283+0000","last_peered":"2024-09-15T21:21:25.054283+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:25.054283+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4501,"log_dups_size":0,"ondisk_log_size":4501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18427904,"num_objects":4499,"num_object_clones":0,"num_object_copies":8998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1445,"num_write_kb":5780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":405},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4546","reported_seq":4673,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.053695+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:25.053695+0000","last_peered":"2024-09-15T21:21:25.053695+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:25.053695+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4546,"log_dups_size":0,"ondisk_log_size":4546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18587648,"num_objects":4538,"num_object_clones":0,"num_object_copies":9076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1455,"num_write_kb":5820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":399},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4468","reported_seq":4598,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.091040+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:25.091040+0000","last_peered":"2024-09-15T21:21:25.091040+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:25.091040+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4468,"log_dups_size":0,"ondisk_log_size":4468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18284544,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":952,"num_write_kb":3808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":382},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4559","reported_seq":4692,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:25.057554+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:25.057554+0000","last_peered":"2024-09-15T21:21:25.057554+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:25.057554+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4559,"log_dups_size":0,"ondisk_log_size":4559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18625478,"num_objects":4549,"num_object_clones":0,"num_object_copies":9098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4549,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1263,"num_write_kb":5052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":402},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":297902814,"num_objects":72739,"num_object_clones":0,"num_object_copies":145478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":72739,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":32604,"num_write_kb":130416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":1306624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":595922944,"data_stored":595767585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72811,"ondisk_log_size":72811,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427756,"kb_used_data":279452,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93944084,"statfs":{"total":96636764160,"available":96198742016,"internally_reserved":0,"allocated":286158848,"data_stored":285896834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":151842473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":426040,"kb_used_data":281960,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93945800,"statfs":{"total":96636764160,"available":96200499200,"internally_reserved":0,"allocated":288727040,"data_stored":288460949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147517099},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,6,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":96612,"kb_used_data":20500,"kb_used_omap":4,"kb_used_meta":76091,"kb_avail":94275228,"statfs":{"total":96636764160,"available":96537833472,"internally_reserved":0,"allocated":20992000,"data_stored":20678940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":77917871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":288866304,"data_stored":288830174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20856832,"data_stored":20773733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":286199808,"data_stored":286163678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:30.738 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 59 20 75480 75460 4.99538 2.32422 0.0133404 0.0156315 2024-09-15T21:21:31.738 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:2024-09-15T21:21:31.735685+0000 min lat: 0.00235499 max lat: 2.60418 avg lat: 0.0157641 2024-09-15T21:21:31.738 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:21:31.738 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 60 20 76112 76092 4.95327 2.46875 0.0223318 0.0157641 2024-09-15T21:21:32.738 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 61 20 76755 76735 4.91324 2.51172 0.0607667 0.0158923 2024-09-15T21:21:33.382 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:33.738 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 62 20 77357 77337 4.87192 2.35156 0.0331288 0.0160267 2024-09-15T21:21:33.952 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:33.952 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:33.978 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-15T21:21:32.696110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":305758961,"num_objects":74659,"num_object_clones":0,"num_object_copies":149318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":74659,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":34524,"num_write_kb":138090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":1306643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74732,"ondisk_log_size":74732,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1000988,"kb_used_data":600556,"kb_used_omap":12,"kb_used_meta":400371,"kb_avail":282114532,"statfs":{"total":289910292480,"available":288885280768,"internally_reserved":0,"allocated":614969344,"data_stored":614128179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":409979907},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,4,1,4,4,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25137152,"num_objects":6137,"num_object_clones":0,"num_object_copies":12274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-241,"num_objects_unfound":0,"num_objects_dirty":6137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6137,"num_write_kb":24548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":987136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002723"},"pg_stats":[{"pgid":"2.f","version":"43'4709","reported_seq":4779,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.008625+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:30.008625+0000","last_peered":"2024-09-15T21:21:30.008625+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:30.008625+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4709,"log_dups_size":0,"ondisk_log_size":4709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19288064,"num_objects":4709,"num_object_clones":0,"num_object_copies":9418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4709,"num_write_kb":18836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":577},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4554","reported_seq":4654,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:28.083072+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:28.083072+0000","last_peered":"2024-09-15T21:21:28.083072+0000","last_clean":"2024-09-15T21:21:28.083072+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:28.083072+0000","last_undegraded":"2024-09-15T21:21:28.083072+0000","last_fullsized":"2024-09-15T21:21:28.083072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4554,"log_dups_size":0,"ondisk_log_size":4554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18629574,"num_objects":4550,"num_object_clones":0,"num_object_copies":9100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4550,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2835,"num_write_kb":11340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4777","reported_seq":4894,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:28.065499+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:28.065499+0000","last_peered":"2024-09-15T21:21:28.065499+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:28.065499+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":4777,"log_dups_size":0,"ondisk_log_size":4777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550208,"num_objects":4773,"num_object_clones":0,"num_object_copies":9546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2877,"num_write_kb":11508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":506}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4533","reported_seq":4598,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:28.062630+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:28.062630+0000","last_peered":"2024-09-15T21:21:28.062630+0000","last_clean":"2024-09-15T21:21:28.062630+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:28.062630+0000","last_undegraded":"2024-09-15T21:21:28.062630+0000","last_fullsized":"2024-09-15T21:21:28.062630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4533,"log_dups_size":0,"ondisk_log_size":4533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18567168,"num_objects":4533,"num_object_clones":0,"num_object_copies":9066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4670","reported_seq":4797,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:30.049594+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:30.049594+0000","last_peered":"2024-09-15T21:21:30.049594+0000","last_clean":"2024-09-15T21:21:30.049594+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:30.049594+0000","last_undegraded":"2024-09-15T21:21:30.049594+0000","last_fullsized":"2024-09-15T21:21:30.049594+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4670,"log_dups_size":0,"ondisk_log_size":4670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19100131,"num_objects":4664,"num_object_clones":0,"num_object_copies":9328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1125,"num_write_kb":4500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4633","reported_seq":4764,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:29.953496+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:29.953496+0000","last_peered":"2024-09-15T21:21:29.953496+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:29.953496+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4633,"log_dups_size":0,"ondisk_log_size":4633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956771,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1309,"num_write_kb":5236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":553},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4736","reported_seq":4867,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.034546+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:30.034546+0000","last_peered":"2024-09-15T21:21:30.034546+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:30.034546+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4736,"log_dups_size":0,"ondisk_log_size":4736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19375046,"num_objects":4732,"num_object_clones":0,"num_object_copies":9464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1047,"num_write_kb":4188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":571},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4815","reported_seq":4938,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:30.057379+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:30.057379+0000","last_peered":"2024-09-15T21:21:30.057379+0000","last_clean":"2024-09-15T21:21:30.057379+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:30.057379+0000","last_undegraded":"2024-09-15T21:21:30.057379+0000","last_fullsized":"2024-09-15T21:21:30.057379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":4815,"log_dups_size":0,"ondisk_log_size":4815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19714048,"num_objects":4813,"num_object_clones":0,"num_object_copies":9626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1587,"num_write_kb":6348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4602","reported_seq":4723,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:28.062074+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:28.062074+0000","last_peered":"2024-09-15T21:21:28.062074+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:28.062074+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18821603,"num_objects":4596,"num_object_clones":0,"num_object_copies":9192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":509}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4619","reported_seq":4744,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.033397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:30.033397+0000","last_peered":"2024-09-15T21:21:30.033397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:30.033397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4619,"log_dups_size":0,"ondisk_log_size":4619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18886656,"num_objects":4611,"num_object_clones":0,"num_object_copies":9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1292,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":591},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4599","reported_seq":4712,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:28.082545+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:28.082545+0000","last_peered":"2024-09-15T21:21:28.082545+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:28.082545+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4599,"log_dups_size":0,"ondisk_log_size":4599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18817507,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2458,"num_write_kb":9832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":505}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4638","reported_seq":5392,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:28.101082+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:28.101082+0000","last_peered":"2024-09-15T21:21:28.101082+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:28.101082+0000","last_undegraded":"2024-09-15T21:21:28.101082+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":4638,"log_dups_size":0,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18972672,"num_objects":4632,"num_object_clones":0,"num_object_copies":9264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":4632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2331,"num_write_kb":9324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3844},{"shards":"2","objects":788}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4698","reported_seq":4826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.073057+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:30.073057+0000","last_peered":"2024-09-15T21:21:30.073057+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:30.073057+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4698,"log_dups_size":0,"ondisk_log_size":4698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19234816,"num_objects":4696,"num_object_clones":0,"num_object_copies":9392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1642,"num_write_kb":6568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":602},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4750","reported_seq":4877,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.060436+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:30.060436+0000","last_peered":"2024-09-15T21:21:30.060436+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:30.060436+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4750,"log_dups_size":0,"ondisk_log_size":4750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19423232,"num_objects":4742,"num_object_clones":0,"num_object_copies":9484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1659,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":603},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4647","reported_seq":4777,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.018108+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:30.018108+0000","last_peered":"2024-09-15T21:21:30.018108+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:30.018108+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4647,"log_dups_size":0,"ondisk_log_size":4647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19017728,"num_objects":4643,"num_object_clones":0,"num_object_copies":9286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1131,"num_write_kb":4524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":561},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4749","reported_seq":4882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.015357+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:30.015357+0000","last_peered":"2024-09-15T21:21:30.015357+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:30.015357+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4749,"log_dups_size":0,"ondisk_log_size":4749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19403718,"num_objects":4739,"num_object_clones":0,"num_object_copies":9478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1453,"num_write_kb":5812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":592},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":305758942,"num_objects":74657,"num_object_clones":0,"num_object_copies":149314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":74657,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":34522,"num_write_kb":138088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":1306624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":615161856,"data_stored":615006497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74729,"ondisk_log_size":74729,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427756,"kb_used_data":279452,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93944084,"statfs":{"total":96636764160,"available":96198742016,"internally_reserved":0,"allocated":286158848,"data_stored":285896834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":151842473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439380,"kb_used_data":291140,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93932460,"statfs":{"total":96636764160,"available":96186839040,"internally_reserved":0,"allocated":298127360,"data_stored":297861269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151776939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133852,"kb_used_data":29964,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94237988,"statfs":{"total":96636764160,"available":96499699712,"internally_reserved":0,"allocated":30683136,"data_stored":30370076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":106360495},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":298332160,"data_stored":298296030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30629888,"data_stored":30546789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":286199808,"data_stored":286163678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:33.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:34.515 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:34.515 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:34.538 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-15T21:21:32.696110+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":305758961,"num_objects":74659,"num_object_clones":0,"num_object_copies":149318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":74659,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":34524,"num_write_kb":138090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":1306643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74732,"ondisk_log_size":74732,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1000988,"kb_used_data":600556,"kb_used_omap":12,"kb_used_meta":400371,"kb_avail":282114532,"statfs":{"total":289910292480,"available":288885280768,"internally_reserved":0,"allocated":614969344,"data_stored":614128179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":409979907},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,4,1,4,4,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25137152,"num_objects":6137,"num_object_clones":0,"num_object_copies":12274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-241,"num_objects_unfound":0,"num_objects_dirty":6137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6137,"num_write_kb":24548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":987136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002723"},"pg_stats":[{"pgid":"2.f","version":"43'4709","reported_seq":4779,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.008625+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:30.008625+0000","last_peered":"2024-09-15T21:21:30.008625+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:30.008625+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4709,"log_dups_size":0,"ondisk_log_size":4709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19288064,"num_objects":4709,"num_object_clones":0,"num_object_copies":9418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4709,"num_write_kb":18836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":577},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4554","reported_seq":4654,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:28.083072+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:28.083072+0000","last_peered":"2024-09-15T21:21:28.083072+0000","last_clean":"2024-09-15T21:21:28.083072+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:28.083072+0000","last_undegraded":"2024-09-15T21:21:28.083072+0000","last_fullsized":"2024-09-15T21:21:28.083072+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4554,"log_dups_size":0,"ondisk_log_size":4554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18629574,"num_objects":4550,"num_object_clones":0,"num_object_copies":9100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4550,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":2835,"num_write_kb":11340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4777","reported_seq":4894,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:28.065499+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:28.065499+0000","last_peered":"2024-09-15T21:21:28.065499+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:28.065499+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":4777,"log_dups_size":0,"ondisk_log_size":4777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19550208,"num_objects":4773,"num_object_clones":0,"num_object_copies":9546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4773,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2877,"num_write_kb":11508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":506}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4533","reported_seq":4598,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:28.062630+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:28.062630+0000","last_peered":"2024-09-15T21:21:28.062630+0000","last_clean":"2024-09-15T21:21:28.062630+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:28.062630+0000","last_undegraded":"2024-09-15T21:21:28.062630+0000","last_fullsized":"2024-09-15T21:21:28.062630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4533,"log_dups_size":0,"ondisk_log_size":4533,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18567168,"num_objects":4533,"num_object_clones":0,"num_object_copies":9066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4670","reported_seq":4797,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:30.049594+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:30.049594+0000","last_peered":"2024-09-15T21:21:30.049594+0000","last_clean":"2024-09-15T21:21:30.049594+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:30.049594+0000","last_undegraded":"2024-09-15T21:21:30.049594+0000","last_fullsized":"2024-09-15T21:21:30.049594+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4670,"log_dups_size":0,"ondisk_log_size":4670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19100131,"num_objects":4664,"num_object_clones":0,"num_object_copies":9328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1125,"num_write_kb":4500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4633","reported_seq":4764,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:29.953496+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:29.953496+0000","last_peered":"2024-09-15T21:21:29.953496+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:29.953496+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4633,"log_dups_size":0,"ondisk_log_size":4633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956771,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1309,"num_write_kb":5236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":553},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4736","reported_seq":4867,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.034546+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:30.034546+0000","last_peered":"2024-09-15T21:21:30.034546+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:30.034546+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4736,"log_dups_size":0,"ondisk_log_size":4736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19375046,"num_objects":4732,"num_object_clones":0,"num_object_copies":9464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1047,"num_write_kb":4188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":571},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4815","reported_seq":4938,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:30.057379+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:30.057379+0000","last_peered":"2024-09-15T21:21:30.057379+0000","last_clean":"2024-09-15T21:21:30.057379+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:30.057379+0000","last_undegraded":"2024-09-15T21:21:30.057379+0000","last_fullsized":"2024-09-15T21:21:30.057379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":4815,"log_dups_size":0,"ondisk_log_size":4815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19714048,"num_objects":4813,"num_object_clones":0,"num_object_copies":9626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1587,"num_write_kb":6348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4602","reported_seq":4723,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:28.062074+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:28.062074+0000","last_peered":"2024-09-15T21:21:28.062074+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:28.062074+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4602,"log_dups_size":0,"ondisk_log_size":4602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18821603,"num_objects":4596,"num_object_clones":0,"num_object_copies":9192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4596,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":509}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4619","reported_seq":4744,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.033397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:30.033397+0000","last_peered":"2024-09-15T21:21:30.033397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:30.033397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4619,"log_dups_size":0,"ondisk_log_size":4619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18886656,"num_objects":4611,"num_object_clones":0,"num_object_copies":9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1292,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":591},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4599","reported_seq":4712,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:28.082545+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:28.082545+0000","last_peered":"2024-09-15T21:21:28.082545+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:28.082545+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4599,"log_dups_size":0,"ondisk_log_size":4599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18817507,"num_objects":4595,"num_object_clones":0,"num_object_copies":9190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4595,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2458,"num_write_kb":9832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":505}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4638","reported_seq":5392,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:28.101082+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:28.101082+0000","last_peered":"2024-09-15T21:21:28.101082+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:28.101082+0000","last_undegraded":"2024-09-15T21:21:28.101082+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":4638,"log_dups_size":0,"ondisk_log_size":4638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18972672,"num_objects":4632,"num_object_clones":0,"num_object_copies":9264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":4632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2331,"num_write_kb":9324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3844},{"shards":"2","objects":788}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4698","reported_seq":4826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.073057+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:30.073057+0000","last_peered":"2024-09-15T21:21:30.073057+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:30.073057+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4698,"log_dups_size":0,"ondisk_log_size":4698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19234816,"num_objects":4696,"num_object_clones":0,"num_object_copies":9392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1642,"num_write_kb":6568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":602},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4750","reported_seq":4877,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.060436+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:30.060436+0000","last_peered":"2024-09-15T21:21:30.060436+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:30.060436+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4750,"log_dups_size":0,"ondisk_log_size":4750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19423232,"num_objects":4742,"num_object_clones":0,"num_object_copies":9484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1659,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":603},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4647","reported_seq":4777,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.018108+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:30.018108+0000","last_peered":"2024-09-15T21:21:30.018108+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:30.018108+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4647,"log_dups_size":0,"ondisk_log_size":4647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19017728,"num_objects":4643,"num_object_clones":0,"num_object_copies":9286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1131,"num_write_kb":4524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":561},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4749","reported_seq":4882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.015357+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:30.015357+0000","last_peered":"2024-09-15T21:21:30.015357+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:30.015357+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4749,"log_dups_size":0,"ondisk_log_size":4749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19403718,"num_objects":4739,"num_object_clones":0,"num_object_copies":9478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1453,"num_write_kb":5812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":592},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":305758942,"num_objects":74657,"num_object_clones":0,"num_object_copies":149314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3844,"num_objects_unfound":0,"num_objects_dirty":74657,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":34522,"num_write_kb":138088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":319,"num_bytes_recovered":1306624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":615161856,"data_stored":615006497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74729,"ondisk_log_size":74729,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427756,"kb_used_data":279452,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":93944084,"statfs":{"total":96636764160,"available":96198742016,"internally_reserved":0,"allocated":286158848,"data_stored":285896834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":151842473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439380,"kb_used_data":291140,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93932460,"statfs":{"total":96636764160,"available":96186839040,"internally_reserved":0,"allocated":298127360,"data_stored":297861269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151776939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133852,"kb_used_data":29964,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94237988,"statfs":{"total":96636764160,"available":96499699712,"internally_reserved":0,"allocated":30683136,"data_stored":30370076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":106360495},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":298332160,"data_stored":298296030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30629888,"data_stored":30546789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":286199808,"data_stored":286163678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:34.539 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:34.540 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'4709", 'reported_seq': 4779, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:30.008625+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:21:30.008625+0000', 'last_peered': '2024-09-15T21:21:30.008625+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:21:30.008625+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 4709, 'log_dups_size': 0, 'ondisk_log_size': 4709, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19288064, 'num_objects': 4709, 'num_object_clones': 0, 'num_object_copies': 9418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4709, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4709, 'num_write_kb': 18836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 577}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:34.540 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:21:34.540 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'4777", 'reported_seq': 4894, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:28.065499+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:21:28.065499+0000', 'last_peered': '2024-09-15T21:21:28.065499+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:21:28.065499+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 4777, 'log_dups_size': 0, 'ondisk_log_size': 4777, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19550208, 'num_objects': 4773, 'num_object_clones': 0, 'num_object_copies': 9546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4773, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2877, 'num_write_kb': 11508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 506}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:34.540 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:34.540 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'4633", 'reported_seq': 4764, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:29.953496+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:21:29.953496+0000', 'last_peered': '2024-09-15T21:21:29.953496+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:21:29.953496+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 4633, 'log_dups_size': 0, 'ondisk_log_size': 4633, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18956771, 'num_objects': 4629, 'num_object_clones': 0, 'num_object_copies': 9258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4629, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1309, 'num_write_kb': 5236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 553}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:34.540 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:34.540 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'4736", 'reported_seq': 4867, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:30.034546+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:21:30.034546+0000', 'last_peered': '2024-09-15T21:21:30.034546+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:21:30.034546+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 4736, 'log_dups_size': 0, 'ondisk_log_size': 4736, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19375046, 'num_objects': 4732, 'num_object_clones': 0, 'num_object_copies': 9464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1047, 'num_write_kb': 4188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 571}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:34.540 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:34.541 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'4602", 'reported_seq': 4723, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:28.062074+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:21:28.062074+0000', 'last_peered': '2024-09-15T21:21:28.062074+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:21:28.062074+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 4602, 'log_dups_size': 0, 'ondisk_log_size': 4602, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18821603, 'num_objects': 4596, 'num_object_clones': 0, 'num_object_copies': 9192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4596, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2534, 'num_write_kb': 10136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 509}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:34.541 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:34.541 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'4619", 'reported_seq': 4744, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:30.033397+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:21:30.033397+0000', 'last_peered': '2024-09-15T21:21:30.033397+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:21:30.033397+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 4619, 'log_dups_size': 0, 'ondisk_log_size': 4619, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18886656, 'num_objects': 4611, 'num_object_clones': 0, 'num_object_copies': 9222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4611, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1292, 'num_write_kb': 5168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 591}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:34.541 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:34.541 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'4599", 'reported_seq': 4712, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:28.082545+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:21:28.082545+0000', 'last_peered': '2024-09-15T21:21:28.082545+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:21:28.082545+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 4599, 'log_dups_size': 0, 'ondisk_log_size': 4599, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18817507, 'num_objects': 4595, 'num_object_clones': 0, 'num_object_copies': 9190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4595, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2458, 'num_write_kb': 9832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 505}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:34.541 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:34.541 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'4638", 'reported_seq': 5392, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:28.101082+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:21:28.101082+0000', 'last_peered': '2024-09-15T21:21:28.101082+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:21:28.101082+0000', 'last_undegraded': '2024-09-15T21:21:28.101082+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 4638, 'log_dups_size': 0, 'ondisk_log_size': 4638, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18972672, 'num_objects': 4632, 'num_object_clones': 0, 'num_object_copies': 9264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3844, 'num_objects_unfound': 0, 'num_objects_dirty': 4632, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2331, 'num_write_kb': 9324, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 318, 'num_bytes_recovered': 1302528, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3844}, {'shards': '2', 'objects': 788}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:34.541 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:34.541 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'4698", 'reported_seq': 4826, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:30.073057+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:21:30.073057+0000', 'last_peered': '2024-09-15T21:21:30.073057+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:21:30.073057+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 4698, 'log_dups_size': 0, 'ondisk_log_size': 4698, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19234816, 'num_objects': 4696, 'num_object_clones': 0, 'num_object_copies': 9392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4696, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1642, 'num_write_kb': 6568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 602}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:34.541 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:34.541 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'4750", 'reported_seq': 4877, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:30.060436+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:21:30.060436+0000', 'last_peered': '2024-09-15T21:21:30.060436+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:21:30.060436+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 4750, 'log_dups_size': 0, 'ondisk_log_size': 4750, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19423232, 'num_objects': 4742, 'num_object_clones': 0, 'num_object_copies': 9484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4742, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1659, 'num_write_kb': 6636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 603}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:34.542 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:34.542 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'4647", 'reported_seq': 4777, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:30.018108+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:21:30.018108+0000', 'last_peered': '2024-09-15T21:21:30.018108+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:21:30.018108+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 4647, 'log_dups_size': 0, 'ondisk_log_size': 4647, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19017728, 'num_objects': 4643, 'num_object_clones': 0, 'num_object_copies': 9286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4643, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1131, 'num_write_kb': 4524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 561}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:34.542 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:34.542 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'4749", 'reported_seq': 4882, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:30.015357+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:21:30.015357+0000', 'last_peered': '2024-09-15T21:21:30.015357+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:21:30.015357+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 4749, 'log_dups_size': 0, 'ondisk_log_size': 4749, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19403718, 'num_objects': 4739, 'num_object_clones': 0, 'num_object_copies': 9478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4739, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1453, 'num_write_kb': 5812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 592}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:34.542 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:34.738 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 63 20 77905 77885 4.82856 2.14062 0.0291032 0.0161717 2024-09-15T21:21:35.218 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:35.219 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/149318 objects degraded (30.344%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":146,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":74659,"data_bytes":305758961,"bytes_used":1025011712,"bytes_avail":288885280768,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":149318,"degraded_ratio":0.30343963889149334,"misplaced_objects":3844,"misplaced_total":149318,"misplaced_ratio":0.025743714756425885,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":82242,"recovering_keys_per_sec":0,"num_objects_recovered":241,"num_bytes_recovered":987136,"num_keys_recovered":0,"write_bytes_sec":2094287,"read_op_per_sec":0,"write_op_per_sec":511},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-15T21:21:18.693002+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:21:35.243 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:35.243 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:35.738 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 64 20 78455 78435 4.78668 2.14844 0.0295203 0.0163119 2024-09-15T21:21:35.779 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:35.780 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:35.808 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-15T21:21:34.696527+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":310489841,"num_objects":75814,"num_object_clones":0,"num_object_copies":151628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3729,"num_objects_unfound":0,"num_objects_dirty":75814,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":35679,"num_write_kb":142710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":435,"num_bytes_recovered":1777683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75887,"ondisk_log_size":75887,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":994592,"kb_used_data":606640,"kb_used_omap":12,"kb_used_meta":387891,"kb_avail":282120928,"statfs":{"total":289910292480,"available":288891830272,"internally_reserved":0,"allocated":621199360,"data_stored":620358195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":397200387},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,1,7,6,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29868032,"num_objects":7292,"num_object_clones":0,"num_object_copies":14584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-356,"num_objects_unfound":0,"num_objects_dirty":7292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7292,"num_write_kb":29168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":1458176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002665"},"pg_stats":[{"pgid":"2.f","version":"43'4709","reported_seq":4779,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.008625+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:30.008625+0000","last_peered":"2024-09-15T21:21:30.008625+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:30.008625+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4709,"log_dups_size":0,"ondisk_log_size":4709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19288064,"num_objects":4709,"num_object_clones":0,"num_object_copies":9418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4709,"num_write_kb":18836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":577},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4757","reported_seq":4857,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:33.022270+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:33.022270+0000","last_peered":"2024-09-15T21:21:33.022270+0000","last_clean":"2024-09-15T21:21:33.022270+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:33.022270+0000","last_undegraded":"2024-09-15T21:21:33.022270+0000","last_fullsized":"2024-09-15T21:21:33.022270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4757,"log_dups_size":0,"ondisk_log_size":4757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19461062,"num_objects":4753,"num_object_clones":0,"num_object_copies":9506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4753,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3038,"num_write_kb":12152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4951","reported_seq":5071,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:33.114903+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:33.114903+0000","last_peered":"2024-09-15T21:21:33.114903+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:33.114903+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":4951,"log_dups_size":0,"ondisk_log_size":4951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20262912,"num_objects":4947,"num_object_clones":0,"num_object_copies":9894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3051,"num_write_kb":12204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":680}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4745","reported_seq":4810,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:33.090631+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:33.090631+0000","last_peered":"2024-09-15T21:21:33.090631+0000","last_clean":"2024-09-15T21:21:33.090631+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:33.090631+0000","last_undegraded":"2024-09-15T21:21:33.090631+0000","last_fullsized":"2024-09-15T21:21:33.090631+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4745,"log_dups_size":0,"ondisk_log_size":4745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19435520,"num_objects":4745,"num_object_clones":0,"num_object_copies":9490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4745,"num_write_kb":18980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4670","reported_seq":4797,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:30.049594+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:30.049594+0000","last_peered":"2024-09-15T21:21:30.049594+0000","last_clean":"2024-09-15T21:21:30.049594+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:30.049594+0000","last_undegraded":"2024-09-15T21:21:30.049594+0000","last_fullsized":"2024-09-15T21:21:30.049594+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4670,"log_dups_size":0,"ondisk_log_size":4670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19100131,"num_objects":4664,"num_object_clones":0,"num_object_copies":9328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1125,"num_write_kb":4500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4633","reported_seq":4764,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:29.953496+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:29.953496+0000","last_peered":"2024-09-15T21:21:29.953496+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:29.953496+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4633,"log_dups_size":0,"ondisk_log_size":4633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956771,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1309,"num_write_kb":5236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":553},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4736","reported_seq":4867,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.034546+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:30.034546+0000","last_peered":"2024-09-15T21:21:30.034546+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:30.034546+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4736,"log_dups_size":0,"ondisk_log_size":4736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19375046,"num_objects":4732,"num_object_clones":0,"num_object_copies":9464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1047,"num_write_kb":4188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":571},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4815","reported_seq":4938,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:30.057379+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:30.057379+0000","last_peered":"2024-09-15T21:21:30.057379+0000","last_clean":"2024-09-15T21:21:30.057379+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:30.057379+0000","last_undegraded":"2024-09-15T21:21:30.057379+0000","last_fullsized":"2024-09-15T21:21:30.057379+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":4815,"log_dups_size":0,"ondisk_log_size":4815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19714048,"num_objects":4813,"num_object_clones":0,"num_object_copies":9626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1587,"num_write_kb":6348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4778","reported_seq":4899,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:33.051229+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:33.051229+0000","last_peered":"2024-09-15T21:21:33.051229+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:33.051229+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4778,"log_dups_size":0,"ondisk_log_size":4778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19542499,"num_objects":4772,"num_object_clones":0,"num_object_copies":9544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4772,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2710,"num_write_kb":10840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":685}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4619","reported_seq":4744,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.033397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:30.033397+0000","last_peered":"2024-09-15T21:21:30.033397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:30.033397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4619,"log_dups_size":0,"ondisk_log_size":4619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18886656,"num_objects":4611,"num_object_clones":0,"num_object_copies":9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1292,"num_write_kb":5168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":591},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4797","reported_seq":4910,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:33.092739+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:33.092739+0000","last_peered":"2024-09-15T21:21:33.092739+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:33.092739+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4797,"log_dups_size":0,"ondisk_log_size":4797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19628515,"num_objects":4793,"num_object_clones":0,"num_object_copies":9586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4793,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2656,"num_write_kb":10624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":703}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4830","reported_seq":5814,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:33.100768+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:33.100768+0000","last_peered":"2024-09-15T21:21:33.100768+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:33.100768+0000","last_undegraded":"2024-09-15T21:21:33.100768+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":4830,"log_dups_size":0,"ondisk_log_size":4830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19759104,"num_objects":4824,"num_object_clones":0,"num_object_copies":9648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3729,"num_objects_unfound":0,"num_objects_dirty":4824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2523,"num_write_kb":10092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":433,"num_bytes_recovered":1773568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3729},{"shards":"2","objects":1095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4698","reported_seq":4826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.073057+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:30.073057+0000","last_peered":"2024-09-15T21:21:30.073057+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:30.073057+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4698,"log_dups_size":0,"ondisk_log_size":4698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19234816,"num_objects":4696,"num_object_clones":0,"num_object_copies":9392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1642,"num_write_kb":6568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":602},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4750","reported_seq":4877,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.060436+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:30.060436+0000","last_peered":"2024-09-15T21:21:30.060436+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:30.060436+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4750,"log_dups_size":0,"ondisk_log_size":4750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19423232,"num_objects":4742,"num_object_clones":0,"num_object_copies":9484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1659,"num_write_kb":6636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":603},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4647","reported_seq":4777,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.018108+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:30.018108+0000","last_peered":"2024-09-15T21:21:30.018108+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:30.018108+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4647,"log_dups_size":0,"ondisk_log_size":4647,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19017728,"num_objects":4643,"num_object_clones":0,"num_object_copies":9286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1131,"num_write_kb":4524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":561},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4749","reported_seq":4882,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:30.015357+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:30.015357+0000","last_peered":"2024-09-15T21:21:30.015357+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:30.015357+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4749,"log_dups_size":0,"ondisk_log_size":4749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19403718,"num_objects":4739,"num_object_clones":0,"num_object_copies":9478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1453,"num_write_kb":5812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":592},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":310489822,"num_objects":75812,"num_object_clones":0,"num_object_copies":151624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3729,"num_objects_unfound":0,"num_objects_dirty":75812,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":35677,"num_write_kb":142708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":434,"num_bytes_recovered":1777664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":621346816,"data_stored":621191457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75884,"ondisk_log_size":75884,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":421360,"kb_used_data":285536,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":93950480,"statfs":{"total":96636764160,"available":96205291520,"internally_reserved":0,"allocated":292388864,"data_stored":292126850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":139062953},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439380,"kb_used_data":291140,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93932460,"statfs":{"total":96636764160,"available":96186839040,"internally_reserved":0,"allocated":298127360,"data_stored":297861269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151776939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":133852,"kb_used_data":29964,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94237988,"statfs":{"total":96636764160,"available":96499699712,"internally_reserved":0,"allocated":30683136,"data_stored":30370076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":106360495},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":298332160,"data_stored":298296030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30629888,"data_stored":30546789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":292384768,"data_stored":292348638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:36.741 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 65 20 79073 79053 4.75018 2.41406 0.0240498 0.0164387 2024-09-15T21:21:37.738 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 66 20 79685 79665 4.71443 2.39062 0.0268644 0.0165622 2024-09-15T21:21:38.738 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 67 20 80311 80291 4.68056 2.44531 0.0267059 0.0166822 2024-09-15T21:21:38.811 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:39.367 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:39.367 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:39.395 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-15T21:21:38.697393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":322671345,"num_objects":78788,"num_object_clones":0,"num_object_copies":157576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3609,"num_objects_unfound":0,"num_objects_dirty":78788,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":38653,"num_write_kb":154606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":555,"num_bytes_recovered":2269203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78861,"ondisk_log_size":78861,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1032744,"kb_used_data":630904,"kb_used_omap":12,"kb_used_meta":401779,"kb_avail":282082776,"statfs":{"total":289910292480,"available":288852762624,"internally_reserved":0,"allocated":646045696,"data_stored":645204531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":411421699},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,7,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29450240,"num_objects":7190,"num_object_clones":0,"num_object_copies":14380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-361,"num_objects_unfound":0,"num_objects_dirty":7190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7190,"num_write_kb":28760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":1478656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002642"},"pg_stats":[{"pgid":"2.f","version":"43'4913","reported_seq":4979,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.055468+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:35.055468+0000","last_peered":"2024-09-15T21:21:35.055468+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:35.055468+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4913,"log_dups_size":0,"ondisk_log_size":4913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20123648,"num_objects":4913,"num_object_clones":0,"num_object_copies":9826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4913,"num_write_kb":19652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":781},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4940","reported_seq":5040,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:38.082402+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:38.082402+0000","last_peered":"2024-09-15T21:21:38.082402+0000","last_clean":"2024-09-15T21:21:38.082402+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:38.082402+0000","last_undegraded":"2024-09-15T21:21:38.082402+0000","last_fullsized":"2024-09-15T21:21:38.082402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4940,"log_dups_size":0,"ondisk_log_size":4940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20210630,"num_objects":4936,"num_object_clones":0,"num_object_copies":9872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3221,"num_write_kb":12884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5139","reported_seq":5259,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:38.063413+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:38.063413+0000","last_peered":"2024-09-15T21:21:38.063413+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:38.063413+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5139,"log_dups_size":0,"ondisk_log_size":5139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21032960,"num_objects":5135,"num_object_clones":0,"num_object_copies":10270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3239,"num_write_kb":12956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4913","reported_seq":4978,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:38.010586+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:38.010586+0000","last_peered":"2024-09-15T21:21:38.010586+0000","last_clean":"2024-09-15T21:21:38.010586+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:38.010586+0000","last_undegraded":"2024-09-15T21:21:38.010586+0000","last_fullsized":"2024-09-15T21:21:38.010586+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4913,"log_dups_size":0,"ondisk_log_size":4913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20123648,"num_objects":4913,"num_object_clones":0,"num_object_copies":9826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4913,"num_write_kb":19652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4835","reported_seq":4962,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:35.074242+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:35.074242+0000","last_peered":"2024-09-15T21:21:35.074242+0000","last_clean":"2024-09-15T21:21:35.074242+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:35.074242+0000","last_undegraded":"2024-09-15T21:21:35.074242+0000","last_fullsized":"2024-09-15T21:21:35.074242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4835,"log_dups_size":0,"ondisk_log_size":4835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19775971,"num_objects":4829,"num_object_clones":0,"num_object_copies":9658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1290,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4818","reported_seq":4949,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.097975+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:35.097975+0000","last_peered":"2024-09-15T21:21:35.097975+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:35.097975+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4818,"log_dups_size":0,"ondisk_log_size":4818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19714531,"num_objects":4814,"num_object_clones":0,"num_object_copies":9628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4814,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1494,"num_write_kb":5976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":738},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4917","reported_seq":5048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.028220+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:35.028220+0000","last_peered":"2024-09-15T21:21:35.028220+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:35.028220+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4917,"log_dups_size":0,"ondisk_log_size":4917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20116422,"num_objects":4913,"num_object_clones":0,"num_object_copies":9826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1228,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":752},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5000","reported_seq":5123,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:35.040844+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:35.040844+0000","last_peered":"2024-09-15T21:21:35.040844+0000","last_clean":"2024-09-15T21:21:35.040844+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:35.040844+0000","last_undegraded":"2024-09-15T21:21:35.040844+0000","last_fullsized":"2024-09-15T21:21:35.040844+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5000,"log_dups_size":0,"ondisk_log_size":5000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20471808,"num_objects":4998,"num_object_clones":0,"num_object_copies":9996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1772,"num_write_kb":7088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4953","reported_seq":5074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:38.077784+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:38.077784+0000","last_peered":"2024-09-15T21:21:38.077784+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:38.077784+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20259299,"num_objects":4947,"num_object_clones":0,"num_object_copies":9894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2885,"num_write_kb":11540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":860}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4833","reported_seq":4958,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.084180+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:35.084180+0000","last_peered":"2024-09-15T21:21:35.084180+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:35.084180+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4833,"log_dups_size":0,"ondisk_log_size":4833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19763200,"num_objects":4825,"num_object_clones":0,"num_object_copies":9650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1506,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":805},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4980","reported_seq":5093,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:38.050180+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:38.050180+0000","last_peered":"2024-09-15T21:21:38.050180+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:38.050180+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20378083,"num_objects":4976,"num_object_clones":0,"num_object_copies":9952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2839,"num_write_kb":11356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":886}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5013","reported_seq":6237,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:38.104163+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:38.104163+0000","last_peered":"2024-09-15T21:21:38.104163+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:38.104163+0000","last_undegraded":"2024-09-15T21:21:38.104163+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20508672,"num_objects":5007,"num_object_clones":0,"num_object_copies":10014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3609,"num_objects_unfound":0,"num_objects_dirty":5007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2706,"num_write_kb":10824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":553,"num_bytes_recovered":2265088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3609},{"shards":"2","objects":1398}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4891","reported_seq":5019,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.023248+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:35.023248+0000","last_peered":"2024-09-15T21:21:35.023248+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:35.023248+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4891,"log_dups_size":0,"ondisk_log_size":4891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1835,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":795},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4936","reported_seq":5063,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.110245+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:35.110245+0000","last_peered":"2024-09-15T21:21:35.110245+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:35.110245+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4936,"log_dups_size":0,"ondisk_log_size":4936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185088,"num_objects":4928,"num_object_clones":0,"num_object_copies":9856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1845,"num_write_kb":7380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":789},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4826","reported_seq":4956,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.063138+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:35.063138+0000","last_peered":"2024-09-15T21:21:35.063138+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:35.063138+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4826,"log_dups_size":0,"ondisk_log_size":4826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19750912,"num_objects":4822,"num_object_clones":0,"num_object_copies":9644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1310,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":740},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4951","reported_seq":5084,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.092694+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:35.092694+0000","last_peered":"2024-09-15T21:21:35.092694+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:35.092694+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4951,"log_dups_size":0,"ondisk_log_size":4951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20231110,"num_objects":4941,"num_object_clones":0,"num_object_copies":9882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4941,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1655,"num_write_kb":6620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":794},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":322671326,"num_objects":78786,"num_object_clones":0,"num_object_copies":157572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3609,"num_objects_unfound":0,"num_objects_dirty":78786,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":38651,"num_write_kb":154604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":554,"num_bytes_recovered":2269184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":646447104,"data_stored":646291745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78858,"ondisk_log_size":78858,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431384,"kb_used_data":291400,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93940456,"statfs":{"total":96636764160,"available":96195026944,"internally_reserved":0,"allocated":298393600,"data_stored":298131586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":143322793},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182424,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453972,"kb_used_data":300164,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93917868,"statfs":{"total":96636764160,"available":96171896832,"internally_reserved":0,"allocated":307367936,"data_stored":307101845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":157478571},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147388,"kb_used_data":39340,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94224452,"statfs":{"total":96636764160,"available":96485838848,"internally_reserved":0,"allocated":40284160,"data_stored":39971100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":110620335},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":307691520,"data_stored":307655390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40452096,"data_stored":40368997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298303488,"data_stored":298267358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:39.396 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:39.738 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 68 20 80864 80844 4.64349 2.16016 0.0400287 0.0168184 2024-09-15T21:21:39.937 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:39.937 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:39.957 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-15T21:21:38.697393+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":322671345,"num_objects":78788,"num_object_clones":0,"num_object_copies":157576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3609,"num_objects_unfound":0,"num_objects_dirty":78788,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":38653,"num_write_kb":154606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":555,"num_bytes_recovered":2269203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78861,"ondisk_log_size":78861,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1032744,"kb_used_data":630904,"kb_used_omap":12,"kb_used_meta":401779,"kb_avail":282082776,"statfs":{"total":289910292480,"available":288852762624,"internally_reserved":0,"allocated":646045696,"data_stored":645204531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":411421699},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,7,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29450240,"num_objects":7190,"num_object_clones":0,"num_object_copies":14380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-361,"num_objects_unfound":0,"num_objects_dirty":7190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7190,"num_write_kb":28760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":361,"num_bytes_recovered":1478656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002642"},"pg_stats":[{"pgid":"2.f","version":"43'4913","reported_seq":4979,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.055468+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:35.055468+0000","last_peered":"2024-09-15T21:21:35.055468+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:35.055468+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4913,"log_dups_size":0,"ondisk_log_size":4913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20123648,"num_objects":4913,"num_object_clones":0,"num_object_copies":9826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4913,"num_write_kb":19652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":781},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4940","reported_seq":5040,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:38.082402+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:38.082402+0000","last_peered":"2024-09-15T21:21:38.082402+0000","last_clean":"2024-09-15T21:21:38.082402+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:38.082402+0000","last_undegraded":"2024-09-15T21:21:38.082402+0000","last_fullsized":"2024-09-15T21:21:38.082402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4940,"log_dups_size":0,"ondisk_log_size":4940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20210630,"num_objects":4936,"num_object_clones":0,"num_object_copies":9872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3221,"num_write_kb":12884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5139","reported_seq":5259,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:38.063413+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:38.063413+0000","last_peered":"2024-09-15T21:21:38.063413+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:38.063413+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5139,"log_dups_size":0,"ondisk_log_size":5139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21032960,"num_objects":5135,"num_object_clones":0,"num_object_copies":10270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3239,"num_write_kb":12956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4913","reported_seq":4978,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:38.010586+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:38.010586+0000","last_peered":"2024-09-15T21:21:38.010586+0000","last_clean":"2024-09-15T21:21:38.010586+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:38.010586+0000","last_undegraded":"2024-09-15T21:21:38.010586+0000","last_fullsized":"2024-09-15T21:21:38.010586+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4913,"log_dups_size":0,"ondisk_log_size":4913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20123648,"num_objects":4913,"num_object_clones":0,"num_object_copies":9826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4913,"num_write_kb":19652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4835","reported_seq":4962,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:35.074242+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:35.074242+0000","last_peered":"2024-09-15T21:21:35.074242+0000","last_clean":"2024-09-15T21:21:35.074242+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:35.074242+0000","last_undegraded":"2024-09-15T21:21:35.074242+0000","last_fullsized":"2024-09-15T21:21:35.074242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":4835,"log_dups_size":0,"ondisk_log_size":4835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19775971,"num_objects":4829,"num_object_clones":0,"num_object_copies":9658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1290,"num_write_kb":5160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4818","reported_seq":4949,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.097975+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:35.097975+0000","last_peered":"2024-09-15T21:21:35.097975+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:35.097975+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":4818,"log_dups_size":0,"ondisk_log_size":4818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19714531,"num_objects":4814,"num_object_clones":0,"num_object_copies":9628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4814,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1494,"num_write_kb":5976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":738},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4917","reported_seq":5048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.028220+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:35.028220+0000","last_peered":"2024-09-15T21:21:35.028220+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:35.028220+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":4917,"log_dups_size":0,"ondisk_log_size":4917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20116422,"num_objects":4913,"num_object_clones":0,"num_object_copies":9826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1228,"num_write_kb":4912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":752},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5000","reported_seq":5123,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:35.040844+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:35.040844+0000","last_peered":"2024-09-15T21:21:35.040844+0000","last_clean":"2024-09-15T21:21:35.040844+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:35.040844+0000","last_undegraded":"2024-09-15T21:21:35.040844+0000","last_fullsized":"2024-09-15T21:21:35.040844+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5000,"log_dups_size":0,"ondisk_log_size":5000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20471808,"num_objects":4998,"num_object_clones":0,"num_object_copies":9996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1772,"num_write_kb":7088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4953","reported_seq":5074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:38.077784+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:38.077784+0000","last_peered":"2024-09-15T21:21:38.077784+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:38.077784+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20259299,"num_objects":4947,"num_object_clones":0,"num_object_copies":9894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2885,"num_write_kb":11540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":860}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4833","reported_seq":4958,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.084180+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:35.084180+0000","last_peered":"2024-09-15T21:21:35.084180+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:35.084180+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":4833,"log_dups_size":0,"ondisk_log_size":4833,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19763200,"num_objects":4825,"num_object_clones":0,"num_object_copies":9650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1506,"num_write_kb":6024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":805},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4980","reported_seq":5093,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:38.050180+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:38.050180+0000","last_peered":"2024-09-15T21:21:38.050180+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:38.050180+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20378083,"num_objects":4976,"num_object_clones":0,"num_object_copies":9952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2839,"num_write_kb":11356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":886}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5013","reported_seq":6237,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:38.104163+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:38.104163+0000","last_peered":"2024-09-15T21:21:38.104163+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:38.104163+0000","last_undegraded":"2024-09-15T21:21:38.104163+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20508672,"num_objects":5007,"num_object_clones":0,"num_object_copies":10014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3609,"num_objects_unfound":0,"num_objects_dirty":5007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2706,"num_write_kb":10824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":553,"num_bytes_recovered":2265088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3609},{"shards":"2","objects":1398}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4891","reported_seq":5019,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.023248+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:35.023248+0000","last_peered":"2024-09-15T21:21:35.023248+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:35.023248+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":4891,"log_dups_size":0,"ondisk_log_size":4891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20025344,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1835,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":795},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4936","reported_seq":5063,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.110245+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:35.110245+0000","last_peered":"2024-09-15T21:21:35.110245+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:35.110245+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":4936,"log_dups_size":0,"ondisk_log_size":4936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20185088,"num_objects":4928,"num_object_clones":0,"num_object_copies":9856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1845,"num_write_kb":7380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":789},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4826","reported_seq":4956,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.063138+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:35.063138+0000","last_peered":"2024-09-15T21:21:35.063138+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:35.063138+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":4826,"log_dups_size":0,"ondisk_log_size":4826,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19750912,"num_objects":4822,"num_object_clones":0,"num_object_copies":9644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1310,"num_write_kb":5240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":740},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4951","reported_seq":5084,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:35.092694+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:35.092694+0000","last_peered":"2024-09-15T21:21:35.092694+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:35.092694+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":4951,"log_dups_size":0,"ondisk_log_size":4951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20231110,"num_objects":4941,"num_object_clones":0,"num_object_copies":9882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4941,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1655,"num_write_kb":6620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":794},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":322671326,"num_objects":78786,"num_object_clones":0,"num_object_copies":157572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3609,"num_objects_unfound":0,"num_objects_dirty":78786,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":38651,"num_write_kb":154604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":554,"num_bytes_recovered":2269184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":646447104,"data_stored":646291745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78858,"ondisk_log_size":78858,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431384,"kb_used_data":291400,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93940456,"statfs":{"total":96636764160,"available":96195026944,"internally_reserved":0,"allocated":298393600,"data_stored":298131586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":143322793},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182424,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453972,"kb_used_data":300164,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":93917868,"statfs":{"total":96636764160,"available":96171896832,"internally_reserved":0,"allocated":307367936,"data_stored":307101845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":157478571},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":147388,"kb_used_data":39340,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94224452,"statfs":{"total":96636764160,"available":96485838848,"internally_reserved":0,"allocated":40284160,"data_stored":39971100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":110620335},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":307691520,"data_stored":307655390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40452096,"data_stored":40368997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298303488,"data_stored":298267358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:39.959 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:39.959 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'4913", 'reported_seq': 4979, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:35.055468+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:21:35.055468+0000', 'last_peered': '2024-09-15T21:21:35.055468+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:21:35.055468+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 4913, 'log_dups_size': 0, 'ondisk_log_size': 4913, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20123648, 'num_objects': 4913, 'num_object_clones': 0, 'num_object_copies': 9826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4913, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4913, 'num_write_kb': 19652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 781}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:39.959 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:21:39.959 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'5139", 'reported_seq': 5259, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:38.063413+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:21:38.063413+0000', 'last_peered': '2024-09-15T21:21:38.063413+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:21:38.063413+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 5139, 'log_dups_size': 0, 'ondisk_log_size': 5139, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21032960, 'num_objects': 5135, 'num_object_clones': 0, 'num_object_copies': 10270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5135, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3239, 'num_write_kb': 12956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:39.959 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:39.959 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'4818", 'reported_seq': 4949, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:35.097975+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:21:35.097975+0000', 'last_peered': '2024-09-15T21:21:35.097975+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:21:35.097975+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 4818, 'log_dups_size': 0, 'ondisk_log_size': 4818, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19714531, 'num_objects': 4814, 'num_object_clones': 0, 'num_object_copies': 9628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4814, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1494, 'num_write_kb': 5976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 738}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:39.959 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:39.959 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'4917", 'reported_seq': 5048, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:35.028220+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:21:35.028220+0000', 'last_peered': '2024-09-15T21:21:35.028220+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:21:35.028220+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 4917, 'log_dups_size': 0, 'ondisk_log_size': 4917, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20116422, 'num_objects': 4913, 'num_object_clones': 0, 'num_object_copies': 9826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4913, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1228, 'num_write_kb': 4912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 752}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:39.959 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:39.960 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'4953", 'reported_seq': 5074, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:38.077784+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:21:38.077784+0000', 'last_peered': '2024-09-15T21:21:38.077784+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:21:38.077784+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 4953, 'log_dups_size': 0, 'ondisk_log_size': 4953, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20259299, 'num_objects': 4947, 'num_object_clones': 0, 'num_object_copies': 9894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4947, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2885, 'num_write_kb': 11540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 860}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:39.960 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:39.960 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'4833", 'reported_seq': 4958, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:35.084180+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:21:35.084180+0000', 'last_peered': '2024-09-15T21:21:35.084180+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:21:35.084180+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 4833, 'log_dups_size': 0, 'ondisk_log_size': 4833, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19763200, 'num_objects': 4825, 'num_object_clones': 0, 'num_object_copies': 9650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4825, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1506, 'num_write_kb': 6024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 805}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:39.960 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:39.960 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'4980", 'reported_seq': 5093, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:38.050180+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:21:38.050180+0000', 'last_peered': '2024-09-15T21:21:38.050180+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:21:38.050180+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 4980, 'log_dups_size': 0, 'ondisk_log_size': 4980, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20378083, 'num_objects': 4976, 'num_object_clones': 0, 'num_object_copies': 9952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4976, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2839, 'num_write_kb': 11356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 886}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:39.960 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:39.960 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'5013", 'reported_seq': 6237, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:38.104163+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:21:38.104163+0000', 'last_peered': '2024-09-15T21:21:38.104163+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:21:38.104163+0000', 'last_undegraded': '2024-09-15T21:21:38.104163+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 5013, 'log_dups_size': 0, 'ondisk_log_size': 5013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20508672, 'num_objects': 5007, 'num_object_clones': 0, 'num_object_copies': 10014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3609, 'num_objects_unfound': 0, 'num_objects_dirty': 5007, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2706, 'num_write_kb': 10824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 553, 'num_bytes_recovered': 2265088, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3609}, {'shards': '2', 'objects': 1398}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:39.960 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:39.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'4891", 'reported_seq': 5019, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:35.023248+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:21:35.023248+0000', 'last_peered': '2024-09-15T21:21:35.023248+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:21:35.023248+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 4891, 'log_dups_size': 0, 'ondisk_log_size': 4891, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20025344, 'num_objects': 4889, 'num_object_clones': 0, 'num_object_copies': 9778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4889, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1835, 'num_write_kb': 7340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 795}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:39.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:39.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'4936", 'reported_seq': 5063, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:35.110245+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:21:35.110245+0000', 'last_peered': '2024-09-15T21:21:35.110245+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:21:35.110245+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 4936, 'log_dups_size': 0, 'ondisk_log_size': 4936, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20185088, 'num_objects': 4928, 'num_object_clones': 0, 'num_object_copies': 9856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4928, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1845, 'num_write_kb': 7380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 789}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:39.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:39.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'4826", 'reported_seq': 4956, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:35.063138+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:21:35.063138+0000', 'last_peered': '2024-09-15T21:21:35.063138+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:21:35.063138+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 4826, 'log_dups_size': 0, 'ondisk_log_size': 4826, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19750912, 'num_objects': 4822, 'num_object_clones': 0, 'num_object_copies': 9644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1310, 'num_write_kb': 5240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 740}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:39.961 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:39.961 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'4951", 'reported_seq': 5084, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:35.092694+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:21:35.092694+0000', 'last_peered': '2024-09-15T21:21:35.092694+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:21:35.092694+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 4951, 'log_dups_size': 0, 'ondisk_log_size': 4951, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20231110, 'num_objects': 4941, 'num_object_clones': 0, 'num_object_copies': 9882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4941, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1655, 'num_write_kb': 6620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 794}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:39.962 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:40.593 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:40.593 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/157576 objects degraded (28.754%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":152,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":78788,"data_bytes":322671345,"bytes_used":1057529856,"bytes_avail":288852762624,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":157576,"degraded_ratio":0.28753744225008887,"misplaced_objects":3609,"misplaced_total":157576,"misplaced_ratio":0.022903233995024627,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":123194,"recovering_keys_per_sec":0,"num_objects_recovered":361,"num_bytes_recovered":1478656,"num_keys_recovered":0,"write_bytes_sec":2453646,"read_op_per_sec":0,"write_op_per_sec":599},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-15T21:21:18.693002+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:21:40.606 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:40.606 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:40.738 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 69 20 81354 81334 4.60393 1.91406 0.0130864 0.0169587 2024-09-15T21:21:41.151 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:41.154 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:41.185 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-15T21:21:40.697837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":330187505,"num_objects":80623,"num_object_clones":0,"num_object_copies":161246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3609,"num_objects_unfound":0,"num_objects_dirty":80623,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":40488,"num_write_kb":161946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":555,"num_bytes_recovered":2269203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80696,"ondisk_log_size":80696,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1063428,"kb_used_data":649108,"kb_used_omap":12,"kb_used_meta":414259,"kb_avail":282052092,"statfs":{"total":289910292480,"available":288821342208,"internally_reserved":0,"allocated":664686592,"data_stored":663845427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":424201219},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,4,11,8,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32284672,"num_objects":7882,"num_object_clones":0,"num_object_copies":15764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-235,"num_objects_unfound":0,"num_objects_dirty":7882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7882,"num_write_kb":31528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":962560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002640"},"pg_stats":[{"pgid":"2.f","version":"43'5078","reported_seq":5148,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.119374+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:40.119374+0000","last_peered":"2024-09-15T21:21:40.119374+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:40.119374+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5078,"log_dups_size":0,"ondisk_log_size":5078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20799488,"num_objects":5078,"num_object_clones":0,"num_object_copies":10156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5078,"num_write_kb":20312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":946},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4940","reported_seq":5040,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:38.082402+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:38.082402+0000","last_peered":"2024-09-15T21:21:38.082402+0000","last_clean":"2024-09-15T21:21:38.082402+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:38.082402+0000","last_undegraded":"2024-09-15T21:21:38.082402+0000","last_fullsized":"2024-09-15T21:21:38.082402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":4940,"log_dups_size":0,"ondisk_log_size":4940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20210630,"num_objects":4936,"num_object_clones":0,"num_object_copies":9872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3221,"num_write_kb":12884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5139","reported_seq":5259,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:38.063413+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:38.063413+0000","last_peered":"2024-09-15T21:21:38.063413+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:38.063413+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5139,"log_dups_size":0,"ondisk_log_size":5139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21032960,"num_objects":5135,"num_object_clones":0,"num_object_copies":10270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3239,"num_write_kb":12956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":868}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4913","reported_seq":4978,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:38.010586+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:38.010586+0000","last_peered":"2024-09-15T21:21:38.010586+0000","last_clean":"2024-09-15T21:21:38.010586+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:38.010586+0000","last_undegraded":"2024-09-15T21:21:38.010586+0000","last_fullsized":"2024-09-15T21:21:38.010586+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":4913,"log_dups_size":0,"ondisk_log_size":4913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20123648,"num_objects":4913,"num_object_clones":0,"num_object_copies":9826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4913,"num_write_kb":19652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5032","reported_seq":5159,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:40.104906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:40.104906+0000","last_peered":"2024-09-15T21:21:40.104906+0000","last_clean":"2024-09-15T21:21:40.104906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:40.104906+0000","last_undegraded":"2024-09-15T21:21:40.104906+0000","last_fullsized":"2024-09-15T21:21:40.104906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5032,"log_dups_size":0,"ondisk_log_size":5032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20582883,"num_objects":5026,"num_object_clones":0,"num_object_copies":10052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1487,"num_write_kb":5948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5013","reported_seq":5144,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.075076+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:40.075076+0000","last_peered":"2024-09-15T21:21:40.075076+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:40.075076+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20513251,"num_objects":5009,"num_object_clones":0,"num_object_copies":10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5009,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1689,"num_write_kb":6756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":933},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5100","reported_seq":5231,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.080050+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:40.080050+0000","last_peered":"2024-09-15T21:21:40.080050+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:40.080050+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5100,"log_dups_size":0,"ondisk_log_size":5100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20865990,"num_objects":5096,"num_object_clones":0,"num_object_copies":10192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1411,"num_write_kb":5644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5169","reported_seq":5292,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:40.129880+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:40.129880+0000","last_peered":"2024-09-15T21:21:40.129880+0000","last_clean":"2024-09-15T21:21:40.129880+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:40.129880+0000","last_undegraded":"2024-09-15T21:21:40.129880+0000","last_fullsized":"2024-09-15T21:21:40.129880+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5169,"log_dups_size":0,"ondisk_log_size":5169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21164032,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1941,"num_write_kb":7764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4953","reported_seq":5074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:38.077784+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:38.077784+0000","last_peered":"2024-09-15T21:21:38.077784+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:38.077784+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20259299,"num_objects":4947,"num_object_clones":0,"num_object_copies":9894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4947,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2885,"num_write_kb":11540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":860}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5007","reported_seq":5132,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.098691+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:40.098691+0000","last_peered":"2024-09-15T21:21:40.098691+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:40.098691+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5007,"log_dups_size":0,"ondisk_log_size":5007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1680,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":979},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4980","reported_seq":5093,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:38.050180+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:38.050180+0000","last_peered":"2024-09-15T21:21:38.050180+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:38.050180+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20378083,"num_objects":4976,"num_object_clones":0,"num_object_copies":9952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2839,"num_write_kb":11356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":886}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5013","reported_seq":6237,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:38.104163+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:38.104163+0000","last_peered":"2024-09-15T21:21:38.104163+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:38.104163+0000","last_undegraded":"2024-09-15T21:21:38.104163+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20508672,"num_objects":5007,"num_object_clones":0,"num_object_copies":10014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3609,"num_objects_unfound":0,"num_objects_dirty":5007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2706,"num_write_kb":10824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":553,"num_bytes_recovered":2265088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3609},{"shards":"2","objects":1398}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5084","reported_seq":5212,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.117733+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:40.117733+0000","last_peered":"2024-09-15T21:21:40.117733+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:40.117733+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5084,"log_dups_size":0,"ondisk_log_size":5084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20815872,"num_objects":5082,"num_object_clones":0,"num_object_copies":10164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2028,"num_write_kb":8112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":988},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5115","reported_seq":5242,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.072791+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:40.072791+0000","last_peered":"2024-09-15T21:21:40.072791+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:40.072791+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5115,"log_dups_size":0,"ondisk_log_size":5115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20918272,"num_objects":5107,"num_object_clones":0,"num_object_copies":10214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2024,"num_write_kb":8096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":968},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5026","reported_seq":5156,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.015741+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:40.015741+0000","last_peered":"2024-09-15T21:21:40.015741+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:40.015741+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5026,"log_dups_size":0,"ondisk_log_size":5026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20570112,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1510,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":940},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5131","reported_seq":5264,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.133098+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:40.133098+0000","last_peered":"2024-09-15T21:21:40.133098+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:40.133098+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5131,"log_dups_size":0,"ondisk_log_size":5131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20968390,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1835,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":974},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":330187486,"num_objects":80621,"num_object_clones":0,"num_object_copies":161242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3609,"num_objects_unfound":0,"num_objects_dirty":80621,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":40486,"num_write_kb":161944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":554,"num_bytes_recovered":2269184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":665157632,"data_stored":665002273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80693,"ondisk_log_size":80693,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431384,"kb_used_data":291400,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":93940456,"statfs":{"total":96636764160,"available":96195026944,"internally_reserved":0,"allocated":298393600,"data_stored":298131586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":143322793},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471224,"kb_used_data":309096,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93900616,"statfs":{"total":96636764160,"available":96154230784,"internally_reserved":0,"allocated":316514304,"data_stored":316248213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":165998251},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":160820,"kb_used_data":48612,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94211020,"statfs":{"total":96636764160,"available":96472084480,"internally_reserved":0,"allocated":49778688,"data_stored":49465628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":114880175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":316784640,"data_stored":316748510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50069504,"data_stored":49986405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298303488,"data_stored":298267358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:41.739 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 70 20 81949 81929 4.57136 2.32422 0.0333551 0.0170829 2024-09-15T21:21:42.739 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 71 20 82564 82544 4.54081 2.40234 0.0273309 0.0171975 2024-09-15T21:21:43.739 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 72 20 83184 83164 4.51138 2.42188 0.0460196 0.0173101 2024-09-15T21:21:44.187 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:44.739 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 73 20 83771 83751 4.48099 2.29297 0.0168036 0.0174256 2024-09-15T21:21:44.776 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:44.779 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:44.801 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-15T21:21:44.698697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":334713585,"num_objects":81728,"num_object_clones":0,"num_object_copies":163456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3489,"num_objects_unfound":0,"num_objects_dirty":81728,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":41593,"num_write_kb":166366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":675,"num_bytes_recovered":2760723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81801,"ondisk_log_size":81801,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1091380,"kb_used_data":654852,"kb_used_omap":12,"kb_used_meta":436467,"kb_avail":282024140,"statfs":{"total":289910292480,"available":288792719360,"internally_reserved":0,"allocated":670568448,"data_stored":669727283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":446942211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,11,7,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-355,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":28276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":1454080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002586"},"pg_stats":[{"pgid":"2.f","version":"43'5078","reported_seq":5148,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.119374+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:40.119374+0000","last_peered":"2024-09-15T21:21:40.119374+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:40.119374+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5078,"log_dups_size":0,"ondisk_log_size":5078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20799488,"num_objects":5078,"num_object_clones":0,"num_object_copies":10156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5078,"num_write_kb":20312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":946},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5127","reported_seq":5227,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:43.043851+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:43.043851+0000","last_peered":"2024-09-15T21:21:43.043851+0000","last_clean":"2024-09-15T21:21:43.043851+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:43.043851+0000","last_undegraded":"2024-09-15T21:21:43.043851+0000","last_fullsized":"2024-09-15T21:21:43.043851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5127,"log_dups_size":0,"ondisk_log_size":5127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20976582,"num_objects":5123,"num_object_clones":0,"num_object_copies":10246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5123,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3408,"num_write_kb":13632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5329","reported_seq":5446,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.121650+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:43.121650+0000","last_peered":"2024-09-15T21:21:43.121650+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:43.121650+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5329,"log_dups_size":0,"ondisk_log_size":5329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21811200,"num_objects":5325,"num_object_clones":0,"num_object_copies":10650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3429,"num_write_kb":13716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5091","reported_seq":5155,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:43.114859+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:43.114859+0000","last_peered":"2024-09-15T21:21:43.114859+0000","last_clean":"2024-09-15T21:21:43.114859+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:43.114859+0000","last_undegraded":"2024-09-15T21:21:43.114859+0000","last_fullsized":"2024-09-15T21:21:43.114859+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5091,"log_dups_size":0,"ondisk_log_size":5091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20852736,"num_objects":5091,"num_object_clones":0,"num_object_copies":10182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5091,"num_write_kb":20364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5032","reported_seq":5159,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:40.104906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:40.104906+0000","last_peered":"2024-09-15T21:21:40.104906+0000","last_clean":"2024-09-15T21:21:40.104906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:40.104906+0000","last_undegraded":"2024-09-15T21:21:40.104906+0000","last_fullsized":"2024-09-15T21:21:40.104906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5032,"log_dups_size":0,"ondisk_log_size":5032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20582883,"num_objects":5026,"num_object_clones":0,"num_object_copies":10052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1487,"num_write_kb":5948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5013","reported_seq":5144,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.075076+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:40.075076+0000","last_peered":"2024-09-15T21:21:40.075076+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:40.075076+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20513251,"num_objects":5009,"num_object_clones":0,"num_object_copies":10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5009,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1689,"num_write_kb":6756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":933},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5100","reported_seq":5231,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.080050+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:40.080050+0000","last_peered":"2024-09-15T21:21:40.080050+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:40.080050+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5100,"log_dups_size":0,"ondisk_log_size":5100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20865990,"num_objects":5096,"num_object_clones":0,"num_object_copies":10192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1411,"num_write_kb":5644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5169","reported_seq":5292,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:40.129880+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:40.129880+0000","last_peered":"2024-09-15T21:21:40.129880+0000","last_clean":"2024-09-15T21:21:40.129880+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:40.129880+0000","last_undegraded":"2024-09-15T21:21:40.129880+0000","last_fullsized":"2024-09-15T21:21:40.129880+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5169,"log_dups_size":0,"ondisk_log_size":5169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21164032,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1941,"num_write_kb":7764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5133","reported_seq":5254,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.047806+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:43.047806+0000","last_peered":"2024-09-15T21:21:43.047806+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:43.047806+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5133,"log_dups_size":0,"ondisk_log_size":5133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20996579,"num_objects":5127,"num_object_clones":0,"num_object_copies":10254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3065,"num_write_kb":12260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1040}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5007","reported_seq":5132,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.098691+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:40.098691+0000","last_peered":"2024-09-15T21:21:40.098691+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:40.098691+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5007,"log_dups_size":0,"ondisk_log_size":5007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1680,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":979},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5165","reported_seq":5278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.079942+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:43.079942+0000","last_peered":"2024-09-15T21:21:43.079942+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:43.079942+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5165,"log_dups_size":0,"ondisk_log_size":5165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135843,"num_objects":5161,"num_object_clones":0,"num_object_copies":10322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5161,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3024,"num_write_kb":12096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1071}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5198","reported_seq":6661,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:43.128249+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:43.128249+0000","last_peered":"2024-09-15T21:21:43.128249+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:43.128249+0000","last_undegraded":"2024-09-15T21:21:43.128249+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5198,"log_dups_size":0,"ondisk_log_size":5198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21266432,"num_objects":5192,"num_object_clones":0,"num_object_copies":10384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3489,"num_objects_unfound":0,"num_objects_dirty":5192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2891,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2756608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3489},{"shards":"2","objects":1703}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5084","reported_seq":5212,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.117733+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:40.117733+0000","last_peered":"2024-09-15T21:21:40.117733+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:40.117733+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5084,"log_dups_size":0,"ondisk_log_size":5084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20815872,"num_objects":5082,"num_object_clones":0,"num_object_copies":10164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2028,"num_write_kb":8112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":988},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5115","reported_seq":5242,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.072791+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:40.072791+0000","last_peered":"2024-09-15T21:21:40.072791+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:40.072791+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5115,"log_dups_size":0,"ondisk_log_size":5115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20918272,"num_objects":5107,"num_object_clones":0,"num_object_copies":10214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2024,"num_write_kb":8096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":968},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5026","reported_seq":5156,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.015741+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:40.015741+0000","last_peered":"2024-09-15T21:21:40.015741+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:40.015741+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5026,"log_dups_size":0,"ondisk_log_size":5026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20570112,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1510,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":940},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5131","reported_seq":5264,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.133098+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:40.133098+0000","last_peered":"2024-09-15T21:21:40.133098+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:40.133098+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5131,"log_dups_size":0,"ondisk_log_size":5131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20968390,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1835,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":974},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":334713566,"num_objects":81726,"num_object_clones":0,"num_object_copies":163452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3489,"num_objects_unfound":0,"num_objects_dirty":81726,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":41591,"num_write_kb":166364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":674,"num_bytes_recovered":2760704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":671186944,"data_stored":671031585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81798,"ondisk_log_size":81798,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":459336,"kb_used_data":297144,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93912504,"statfs":{"total":96636764160,"available":96166404096,"internally_reserved":0,"allocated":304275456,"data_stored":304013442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":166063785},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471224,"kb_used_data":309096,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93900616,"statfs":{"total":96636764160,"available":96154230784,"internally_reserved":0,"allocated":316514304,"data_stored":316248213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":165998251},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":160820,"kb_used_data":48612,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94211020,"statfs":{"total":96636764160,"available":96472084480,"internally_reserved":0,"allocated":49778688,"data_stored":49465628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":114880175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":316784640,"data_stored":316748510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50069504,"data_stored":49986405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":304332800,"data_stored":304296670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:44.802 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:45.344 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:45.346 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:45.368 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-15T21:21:44.698697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":334713585,"num_objects":81728,"num_object_clones":0,"num_object_copies":163456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3489,"num_objects_unfound":0,"num_objects_dirty":81728,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":41593,"num_write_kb":166366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":675,"num_bytes_recovered":2760723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81801,"ondisk_log_size":81801,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1091380,"kb_used_data":654852,"kb_used_omap":12,"kb_used_meta":436467,"kb_avail":282024140,"statfs":{"total":289910292480,"available":288792719360,"internally_reserved":0,"allocated":670568448,"data_stored":669727283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":446942211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,11,7,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-355,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":28276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":1454080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002586"},"pg_stats":[{"pgid":"2.f","version":"43'5078","reported_seq":5148,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.119374+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:40.119374+0000","last_peered":"2024-09-15T21:21:40.119374+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:40.119374+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5078,"log_dups_size":0,"ondisk_log_size":5078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20799488,"num_objects":5078,"num_object_clones":0,"num_object_copies":10156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5078,"num_write_kb":20312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":946},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5127","reported_seq":5227,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:43.043851+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:43.043851+0000","last_peered":"2024-09-15T21:21:43.043851+0000","last_clean":"2024-09-15T21:21:43.043851+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:43.043851+0000","last_undegraded":"2024-09-15T21:21:43.043851+0000","last_fullsized":"2024-09-15T21:21:43.043851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5127,"log_dups_size":0,"ondisk_log_size":5127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20976582,"num_objects":5123,"num_object_clones":0,"num_object_copies":10246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5123,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3408,"num_write_kb":13632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5329","reported_seq":5446,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.121650+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:43.121650+0000","last_peered":"2024-09-15T21:21:43.121650+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:43.121650+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5329,"log_dups_size":0,"ondisk_log_size":5329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21811200,"num_objects":5325,"num_object_clones":0,"num_object_copies":10650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3429,"num_write_kb":13716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5091","reported_seq":5155,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:43.114859+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:43.114859+0000","last_peered":"2024-09-15T21:21:43.114859+0000","last_clean":"2024-09-15T21:21:43.114859+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:43.114859+0000","last_undegraded":"2024-09-15T21:21:43.114859+0000","last_fullsized":"2024-09-15T21:21:43.114859+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5091,"log_dups_size":0,"ondisk_log_size":5091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20852736,"num_objects":5091,"num_object_clones":0,"num_object_copies":10182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5091,"num_write_kb":20364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5032","reported_seq":5159,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:40.104906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:40.104906+0000","last_peered":"2024-09-15T21:21:40.104906+0000","last_clean":"2024-09-15T21:21:40.104906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:40.104906+0000","last_undegraded":"2024-09-15T21:21:40.104906+0000","last_fullsized":"2024-09-15T21:21:40.104906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5032,"log_dups_size":0,"ondisk_log_size":5032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20582883,"num_objects":5026,"num_object_clones":0,"num_object_copies":10052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1487,"num_write_kb":5948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5013","reported_seq":5144,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.075076+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:40.075076+0000","last_peered":"2024-09-15T21:21:40.075076+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:40.075076+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20513251,"num_objects":5009,"num_object_clones":0,"num_object_copies":10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5009,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1689,"num_write_kb":6756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":933},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5100","reported_seq":5231,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.080050+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:40.080050+0000","last_peered":"2024-09-15T21:21:40.080050+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:40.080050+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5100,"log_dups_size":0,"ondisk_log_size":5100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20865990,"num_objects":5096,"num_object_clones":0,"num_object_copies":10192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1411,"num_write_kb":5644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5169","reported_seq":5292,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:40.129880+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:40.129880+0000","last_peered":"2024-09-15T21:21:40.129880+0000","last_clean":"2024-09-15T21:21:40.129880+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:40.129880+0000","last_undegraded":"2024-09-15T21:21:40.129880+0000","last_fullsized":"2024-09-15T21:21:40.129880+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5169,"log_dups_size":0,"ondisk_log_size":5169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21164032,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1941,"num_write_kb":7764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5133","reported_seq":5254,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.047806+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:43.047806+0000","last_peered":"2024-09-15T21:21:43.047806+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:43.047806+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5133,"log_dups_size":0,"ondisk_log_size":5133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20996579,"num_objects":5127,"num_object_clones":0,"num_object_copies":10254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3065,"num_write_kb":12260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1040}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5007","reported_seq":5132,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.098691+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:40.098691+0000","last_peered":"2024-09-15T21:21:40.098691+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:40.098691+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5007,"log_dups_size":0,"ondisk_log_size":5007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1680,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":979},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5165","reported_seq":5278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.079942+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:43.079942+0000","last_peered":"2024-09-15T21:21:43.079942+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:43.079942+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5165,"log_dups_size":0,"ondisk_log_size":5165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135843,"num_objects":5161,"num_object_clones":0,"num_object_copies":10322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5161,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3024,"num_write_kb":12096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1071}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5198","reported_seq":6661,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:43.128249+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:43.128249+0000","last_peered":"2024-09-15T21:21:43.128249+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:43.128249+0000","last_undegraded":"2024-09-15T21:21:43.128249+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5198,"log_dups_size":0,"ondisk_log_size":5198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21266432,"num_objects":5192,"num_object_clones":0,"num_object_copies":10384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3489,"num_objects_unfound":0,"num_objects_dirty":5192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2891,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2756608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3489},{"shards":"2","objects":1703}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5084","reported_seq":5212,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.117733+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:40.117733+0000","last_peered":"2024-09-15T21:21:40.117733+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:40.117733+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5084,"log_dups_size":0,"ondisk_log_size":5084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20815872,"num_objects":5082,"num_object_clones":0,"num_object_copies":10164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2028,"num_write_kb":8112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":988},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5115","reported_seq":5242,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.072791+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:40.072791+0000","last_peered":"2024-09-15T21:21:40.072791+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:40.072791+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5115,"log_dups_size":0,"ondisk_log_size":5115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20918272,"num_objects":5107,"num_object_clones":0,"num_object_copies":10214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2024,"num_write_kb":8096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":968},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5026","reported_seq":5156,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.015741+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:40.015741+0000","last_peered":"2024-09-15T21:21:40.015741+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:40.015741+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5026,"log_dups_size":0,"ondisk_log_size":5026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20570112,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1510,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":940},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5131","reported_seq":5264,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.133098+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:40.133098+0000","last_peered":"2024-09-15T21:21:40.133098+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:40.133098+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5131,"log_dups_size":0,"ondisk_log_size":5131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20968390,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1835,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":974},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":334713566,"num_objects":81726,"num_object_clones":0,"num_object_copies":163452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3489,"num_objects_unfound":0,"num_objects_dirty":81726,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":41591,"num_write_kb":166364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":674,"num_bytes_recovered":2760704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":671186944,"data_stored":671031585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81798,"ondisk_log_size":81798,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":459336,"kb_used_data":297144,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93912504,"statfs":{"total":96636764160,"available":96166404096,"internally_reserved":0,"allocated":304275456,"data_stored":304013442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":166063785},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471224,"kb_used_data":309096,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93900616,"statfs":{"total":96636764160,"available":96154230784,"internally_reserved":0,"allocated":316514304,"data_stored":316248213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":165998251},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":160820,"kb_used_data":48612,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94211020,"statfs":{"total":96636764160,"available":96472084480,"internally_reserved":0,"allocated":49778688,"data_stored":49465628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":114880175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":316784640,"data_stored":316748510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50069504,"data_stored":49986405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":304332800,"data_stored":304296670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:45.370 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:45.370 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'5078", 'reported_seq': 5148, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:40.119374+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:21:40.119374+0000', 'last_peered': '2024-09-15T21:21:40.119374+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:21:40.119374+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 5078, 'log_dups_size': 0, 'ondisk_log_size': 5078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20799488, 'num_objects': 5078, 'num_object_clones': 0, 'num_object_copies': 10156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5078, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5078, 'num_write_kb': 20312, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 946}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:45.370 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:21:45.370 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'5329", 'reported_seq': 5446, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:43.121650+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:21:43.121650+0000', 'last_peered': '2024-09-15T21:21:43.121650+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:21:43.121650+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 5329, 'log_dups_size': 0, 'ondisk_log_size': 5329, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21811200, 'num_objects': 5325, 'num_object_clones': 0, 'num_object_copies': 10650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5325, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3429, 'num_write_kb': 13716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 1058}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:45.370 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:45.370 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'5013", 'reported_seq': 5144, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:40.075076+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:21:40.075076+0000', 'last_peered': '2024-09-15T21:21:40.075076+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:21:40.075076+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 5013, 'log_dups_size': 0, 'ondisk_log_size': 5013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20513251, 'num_objects': 5009, 'num_object_clones': 0, 'num_object_copies': 10018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5009, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1689, 'num_write_kb': 6756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 933}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:45.370 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:45.370 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'5100", 'reported_seq': 5231, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:40.080050+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:21:40.080050+0000', 'last_peered': '2024-09-15T21:21:40.080050+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:21:40.080050+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 5100, 'log_dups_size': 0, 'ondisk_log_size': 5100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20865990, 'num_objects': 5096, 'num_object_clones': 0, 'num_object_copies': 10192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1411, 'num_write_kb': 5644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 935}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:45.370 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:45.370 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'5133", 'reported_seq': 5254, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:43.047806+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:21:43.047806+0000', 'last_peered': '2024-09-15T21:21:43.047806+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:21:43.047806+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 5133, 'log_dups_size': 0, 'ondisk_log_size': 5133, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20996579, 'num_objects': 5127, 'num_object_clones': 0, 'num_object_copies': 10254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5127, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3065, 'num_write_kb': 12260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 1040}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:45.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:45.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'5007", 'reported_seq': 5132, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:40.098691+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:21:40.098691+0000', 'last_peered': '2024-09-15T21:21:40.098691+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:21:40.098691+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 5007, 'log_dups_size': 0, 'ondisk_log_size': 5007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20475904, 'num_objects': 4999, 'num_object_clones': 0, 'num_object_copies': 9998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4999, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1680, 'num_write_kb': 6720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 979}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:45.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:45.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'5165", 'reported_seq': 5278, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:43.079942+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:21:43.079942+0000', 'last_peered': '2024-09-15T21:21:43.079942+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:21:43.079942+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 5165, 'log_dups_size': 0, 'ondisk_log_size': 5165, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21135843, 'num_objects': 5161, 'num_object_clones': 0, 'num_object_copies': 10322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5161, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3024, 'num_write_kb': 12096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 1071}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:45.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:45.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'5198", 'reported_seq': 6661, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:43.128249+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:21:43.128249+0000', 'last_peered': '2024-09-15T21:21:43.128249+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:21:43.128249+0000', 'last_undegraded': '2024-09-15T21:21:43.128249+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 5198, 'log_dups_size': 0, 'ondisk_log_size': 5198, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21266432, 'num_objects': 5192, 'num_object_clones': 0, 'num_object_copies': 10384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3489, 'num_objects_unfound': 0, 'num_objects_dirty': 5192, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2891, 'num_write_kb': 11564, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 673, 'num_bytes_recovered': 2756608, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3489}, {'shards': '2', 'objects': 1703}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:45.371 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:45.371 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'5084", 'reported_seq': 5212, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:40.117733+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:21:40.117733+0000', 'last_peered': '2024-09-15T21:21:40.117733+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:21:40.117733+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 5084, 'log_dups_size': 0, 'ondisk_log_size': 5084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20815872, 'num_objects': 5082, 'num_object_clones': 0, 'num_object_copies': 10164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5082, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2028, 'num_write_kb': 8112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 988}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:45.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:45.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'5115", 'reported_seq': 5242, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:40.072791+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:21:40.072791+0000', 'last_peered': '2024-09-15T21:21:40.072791+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:21:40.072791+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 5115, 'log_dups_size': 0, 'ondisk_log_size': 5115, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20918272, 'num_objects': 5107, 'num_object_clones': 0, 'num_object_copies': 10214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5107, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2024, 'num_write_kb': 8096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 968}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:45.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:45.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'5026", 'reported_seq': 5156, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:40.015741+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:21:40.015741+0000', 'last_peered': '2024-09-15T21:21:40.015741+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:21:40.015741+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 5026, 'log_dups_size': 0, 'ondisk_log_size': 5026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20570112, 'num_objects': 5022, 'num_object_clones': 0, 'num_object_copies': 10044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5022, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1510, 'num_write_kb': 6040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 940}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:45.372 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:45.372 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'5131", 'reported_seq': 5264, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:40.133098+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:21:40.133098+0000', 'last_peered': '2024-09-15T21:21:40.133098+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:21:40.133098+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 5131, 'log_dups_size': 0, 'ondisk_log_size': 5131, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20968390, 'num_objects': 5121, 'num_object_clones': 0, 'num_object_copies': 10242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5121, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 1835, 'num_write_kb': 7340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 974}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:45.373 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:45.739 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 74 20 84305 84285 4.44863 2.08594 0.0358737 0.0175527 2024-09-15T21:21:46.077 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:46.077 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/163456 objects degraded (27.719%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":157,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":81728,"data_bytes":334713585,"bytes_used":1117573120,"bytes_avail":288792719360,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":163456,"degraded_ratio":0.27719386256851997,"misplaced_objects":3489,"misplaced_total":163456,"misplaced_ratio":0.021345193813625686,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":121147,"recovering_keys_per_sec":0,"num_objects_recovered":355,"num_bytes_recovered":1454080,"num_keys_recovered":0,"write_bytes_sec":2412365,"read_op_per_sec":0,"write_op_per_sec":588},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-15T21:21:18.693002+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:21:46.104 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:46.104 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:46.640 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:46.641 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:46.669 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-15T21:21:44.698697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":334713585,"num_objects":81728,"num_object_clones":0,"num_object_copies":163456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3489,"num_objects_unfound":0,"num_objects_dirty":81728,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":41593,"num_write_kb":166366,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":675,"num_bytes_recovered":2760723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81801,"ondisk_log_size":81801,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1091380,"kb_used_data":654852,"kb_used_omap":12,"kb_used_meta":436467,"kb_avail":282024140,"statfs":{"total":289910292480,"available":288792719360,"internally_reserved":0,"allocated":670568448,"data_stored":669727283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":446942211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,11,7,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-355,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7069,"num_write_kb":28276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":1454080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002586"},"pg_stats":[{"pgid":"2.f","version":"43'5078","reported_seq":5148,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.119374+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:40.119374+0000","last_peered":"2024-09-15T21:21:40.119374+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:40.119374+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5078,"log_dups_size":0,"ondisk_log_size":5078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20799488,"num_objects":5078,"num_object_clones":0,"num_object_copies":10156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5078,"num_write_kb":20312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":946},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5127","reported_seq":5227,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:43.043851+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:43.043851+0000","last_peered":"2024-09-15T21:21:43.043851+0000","last_clean":"2024-09-15T21:21:43.043851+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:43.043851+0000","last_undegraded":"2024-09-15T21:21:43.043851+0000","last_fullsized":"2024-09-15T21:21:43.043851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5127,"log_dups_size":0,"ondisk_log_size":5127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20976582,"num_objects":5123,"num_object_clones":0,"num_object_copies":10246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5123,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3408,"num_write_kb":13632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5329","reported_seq":5446,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.121650+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:43.121650+0000","last_peered":"2024-09-15T21:21:43.121650+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:43.121650+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5329,"log_dups_size":0,"ondisk_log_size":5329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21811200,"num_objects":5325,"num_object_clones":0,"num_object_copies":10650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3429,"num_write_kb":13716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1058}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5091","reported_seq":5155,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:43.114859+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:43.114859+0000","last_peered":"2024-09-15T21:21:43.114859+0000","last_clean":"2024-09-15T21:21:43.114859+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:43.114859+0000","last_undegraded":"2024-09-15T21:21:43.114859+0000","last_fullsized":"2024-09-15T21:21:43.114859+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5091,"log_dups_size":0,"ondisk_log_size":5091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20852736,"num_objects":5091,"num_object_clones":0,"num_object_copies":10182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5091,"num_write_kb":20364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5032","reported_seq":5159,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:40.104906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:40.104906+0000","last_peered":"2024-09-15T21:21:40.104906+0000","last_clean":"2024-09-15T21:21:40.104906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:40.104906+0000","last_undegraded":"2024-09-15T21:21:40.104906+0000","last_fullsized":"2024-09-15T21:21:40.104906+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5032,"log_dups_size":0,"ondisk_log_size":5032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20582883,"num_objects":5026,"num_object_clones":0,"num_object_copies":10052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1487,"num_write_kb":5948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5013","reported_seq":5144,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.075076+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:40.075076+0000","last_peered":"2024-09-15T21:21:40.075076+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:40.075076+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20513251,"num_objects":5009,"num_object_clones":0,"num_object_copies":10018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5009,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1689,"num_write_kb":6756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":933},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5100","reported_seq":5231,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.080050+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:40.080050+0000","last_peered":"2024-09-15T21:21:40.080050+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:40.080050+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5100,"log_dups_size":0,"ondisk_log_size":5100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20865990,"num_objects":5096,"num_object_clones":0,"num_object_copies":10192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1411,"num_write_kb":5644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":935},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5169","reported_seq":5292,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:40.129880+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:40.129880+0000","last_peered":"2024-09-15T21:21:40.129880+0000","last_clean":"2024-09-15T21:21:40.129880+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:40.129880+0000","last_undegraded":"2024-09-15T21:21:40.129880+0000","last_fullsized":"2024-09-15T21:21:40.129880+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5169,"log_dups_size":0,"ondisk_log_size":5169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21164032,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1941,"num_write_kb":7764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5133","reported_seq":5254,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.047806+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:43.047806+0000","last_peered":"2024-09-15T21:21:43.047806+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:43.047806+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5133,"log_dups_size":0,"ondisk_log_size":5133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20996579,"num_objects":5127,"num_object_clones":0,"num_object_copies":10254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3065,"num_write_kb":12260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1040}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5007","reported_seq":5132,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.098691+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:40.098691+0000","last_peered":"2024-09-15T21:21:40.098691+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:40.098691+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5007,"log_dups_size":0,"ondisk_log_size":5007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1680,"num_write_kb":6720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":979},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5165","reported_seq":5278,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:43.079942+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:43.079942+0000","last_peered":"2024-09-15T21:21:43.079942+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:43.079942+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5165,"log_dups_size":0,"ondisk_log_size":5165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135843,"num_objects":5161,"num_object_clones":0,"num_object_copies":10322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5161,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3024,"num_write_kb":12096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1071}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5198","reported_seq":6661,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:43.128249+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:43.128249+0000","last_peered":"2024-09-15T21:21:43.128249+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:43.128249+0000","last_undegraded":"2024-09-15T21:21:43.128249+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5198,"log_dups_size":0,"ondisk_log_size":5198,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21266432,"num_objects":5192,"num_object_clones":0,"num_object_copies":10384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3489,"num_objects_unfound":0,"num_objects_dirty":5192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2891,"num_write_kb":11564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":673,"num_bytes_recovered":2756608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3489},{"shards":"2","objects":1703}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5084","reported_seq":5212,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.117733+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:40.117733+0000","last_peered":"2024-09-15T21:21:40.117733+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:40.117733+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5084,"log_dups_size":0,"ondisk_log_size":5084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20815872,"num_objects":5082,"num_object_clones":0,"num_object_copies":10164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2028,"num_write_kb":8112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":988},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5115","reported_seq":5242,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.072791+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:40.072791+0000","last_peered":"2024-09-15T21:21:40.072791+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:40.072791+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5115,"log_dups_size":0,"ondisk_log_size":5115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20918272,"num_objects":5107,"num_object_clones":0,"num_object_copies":10214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2024,"num_write_kb":8096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":968},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5026","reported_seq":5156,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.015741+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:40.015741+0000","last_peered":"2024-09-15T21:21:40.015741+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:40.015741+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5026,"log_dups_size":0,"ondisk_log_size":5026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20570112,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1510,"num_write_kb":6040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":940},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5131","reported_seq":5264,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:40.133098+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:40.133098+0000","last_peered":"2024-09-15T21:21:40.133098+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:40.133098+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5131,"log_dups_size":0,"ondisk_log_size":5131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20968390,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1835,"num_write_kb":7340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":974},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":334713566,"num_objects":81726,"num_object_clones":0,"num_object_copies":163452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3489,"num_objects_unfound":0,"num_objects_dirty":81726,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":41591,"num_write_kb":166364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":674,"num_bytes_recovered":2760704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":671186944,"data_stored":671031585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81798,"ondisk_log_size":81798,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":459336,"kb_used_data":297144,"kb_used_omap":4,"kb_used_meta":162171,"kb_avail":93912504,"statfs":{"total":96636764160,"available":96166404096,"internally_reserved":0,"allocated":304275456,"data_stored":304013442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":166063785},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471224,"kb_used_data":309096,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93900616,"statfs":{"total":96636764160,"available":96154230784,"internally_reserved":0,"allocated":316514304,"data_stored":316248213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":165998251},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":160820,"kb_used_data":48612,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94211020,"statfs":{"total":96636764160,"available":96472084480,"internally_reserved":0,"allocated":49778688,"data_stored":49465628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":114880175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":316784640,"data_stored":316748510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50069504,"data_stored":49986405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":304332800,"data_stored":304296670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:46.739 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 75 20 84840 84820 4.41717 2.08984 0.0304852 0.0176791 2024-09-15T21:21:47.739 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 76 20 85452 85432 4.39051 2.39062 0.0403556 0.0177871 2024-09-15T21:21:48.739 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 77 20 86048 86028 4.36372 2.32812 0.0511333 0.0178952 2024-09-15T21:21:49.672 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:49.739 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 78 20 86625 86605 4.33667 2.25391 0.0187281 0.0180078 2024-09-15T21:21:50.235 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:50.238 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:50.268 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-15T21:21:48.699608+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346620657,"num_objects":84635,"num_object_clones":0,"num_object_copies":169270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3358,"num_objects_unfound":0,"num_objects_dirty":84635,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":44500,"num_write_kb":177994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":806,"num_bytes_recovered":3297299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84708,"ondisk_log_size":84708,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1133108,"kb_used_data":678532,"kb_used_omap":12,"kb_used_meta":454515,"kb_avail":281982412,"statfs":{"total":289910292480,"available":288749989888,"internally_reserved":0,"allocated":694816768,"data_stored":693975603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":465423363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,6,4,9,8,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28372992,"num_objects":6927,"num_object_clones":0,"num_object_copies":13854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-371,"num_objects_unfound":0,"num_objects_dirty":6927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6927,"num_write_kb":27708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":371,"num_bytes_recovered":1519616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002662"},"pg_stats":[{"pgid":"2.f","version":"43'5245","reported_seq":5315,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:45.142685+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:45.142685+0000","last_peered":"2024-09-15T21:21:45.142685+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:45.142685+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5245,"log_dups_size":0,"ondisk_log_size":5245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21483520,"num_objects":5245,"num_object_clones":0,"num_object_copies":10490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5245,"num_write_kb":20980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1113},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5310","reported_seq":5410,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:48.097615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:48.097615+0000","last_peered":"2024-09-15T21:21:48.097615+0000","last_clean":"2024-09-15T21:21:48.097615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:48.097615+0000","last_undegraded":"2024-09-15T21:21:48.097615+0000","last_fullsized":"2024-09-15T21:21:48.097615+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5310,"log_dups_size":0,"ondisk_log_size":5310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21726150,"num_objects":5306,"num_object_clones":0,"num_object_copies":10612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5306,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3591,"num_write_kb":14364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5516","reported_seq":5635,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:48.143143+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:48.143143+0000","last_peered":"2024-09-15T21:21:48.143143+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:48.143143+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5516,"log_dups_size":0,"ondisk_log_size":5516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22577152,"num_objects":5512,"num_object_clones":0,"num_object_copies":11024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3616,"num_write_kb":14464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5304","reported_seq":5369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:48.108566+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:48.108566+0000","last_peered":"2024-09-15T21:21:48.108566+0000","last_clean":"2024-09-15T21:21:48.108566+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:48.108566+0000","last_undegraded":"2024-09-15T21:21:48.108566+0000","last_fullsized":"2024-09-15T21:21:48.108566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5304,"log_dups_size":0,"ondisk_log_size":5304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21725184,"num_objects":5304,"num_object_clones":0,"num_object_copies":10608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5304,"num_write_kb":21216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5217","reported_seq":5344,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:45.117302+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:45.117302+0000","last_peered":"2024-09-15T21:21:45.117302+0000","last_clean":"2024-09-15T21:21:45.117302+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:45.117302+0000","last_undegraded":"2024-09-15T21:21:45.117302+0000","last_fullsized":"2024-09-15T21:21:45.117302+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5217,"log_dups_size":0,"ondisk_log_size":5217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21340643,"num_objects":5211,"num_object_clones":0,"num_object_copies":10422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1672,"num_write_kb":6688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5204","reported_seq":5335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:45.049058+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:45.049058+0000","last_peered":"2024-09-15T21:21:45.049058+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:45.049058+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5204,"log_dups_size":0,"ondisk_log_size":5204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21295587,"num_objects":5200,"num_object_clones":0,"num_object_copies":10400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5200,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1880,"num_write_kb":7520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1124},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5283","reported_seq":5414,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:45.075080+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:45.075080+0000","last_peered":"2024-09-15T21:21:45.075080+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:45.075080+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5283,"log_dups_size":0,"ondisk_log_size":5283,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21615558,"num_objects":5279,"num_object_clones":0,"num_object_copies":10558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1594,"num_write_kb":6376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1118},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5369","reported_seq":5492,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:45.124607+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:45.124607+0000","last_peered":"2024-09-15T21:21:45.124607+0000","last_clean":"2024-09-15T21:21:45.124607+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:45.124607+0000","last_undegraded":"2024-09-15T21:21:45.124607+0000","last_fullsized":"2024-09-15T21:21:45.124607+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5369,"log_dups_size":0,"ondisk_log_size":5369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21983232,"num_objects":5367,"num_object_clones":0,"num_object_copies":10734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2141,"num_write_kb":8564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5309","reported_seq":5430,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:48.135217+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:48.135217+0000","last_peered":"2024-09-15T21:21:48.135217+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:48.135217+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5309,"log_dups_size":0,"ondisk_log_size":5309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21717475,"num_objects":5303,"num_object_clones":0,"num_object_copies":10606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5303,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3241,"num_write_kb":12964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5195","reported_seq":5320,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:45.043177+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:45.043177+0000","last_peered":"2024-09-15T21:21:45.043177+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:45.043177+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5195,"log_dups_size":0,"ondisk_log_size":5195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21245952,"num_objects":5187,"num_object_clones":0,"num_object_copies":10374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1868,"num_write_kb":7472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1167},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5335","reported_seq":5448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:48.070593+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:48.070593+0000","last_peered":"2024-09-15T21:21:48.070593+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:48.070593+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21832163,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3194,"num_write_kb":12776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1241}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5370","reported_seq":7096,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:48.120174+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:48.120174+0000","last_peered":"2024-09-15T21:21:48.120174+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:48.120174+0000","last_undegraded":"2024-09-15T21:21:48.120174+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5370,"log_dups_size":0,"ondisk_log_size":5370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21970944,"num_objects":5364,"num_object_clones":0,"num_object_copies":10728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3358,"num_objects_unfound":0,"num_objects_dirty":5364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3063,"num_write_kb":12252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":804,"num_bytes_recovered":3293184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3358},{"shards":"2","objects":2006}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5248","reported_seq":5373,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:45.069508+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:45.069508+0000","last_peered":"2024-09-15T21:21:45.069508+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:45.069508+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5248,"log_dups_size":0,"ondisk_log_size":5248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21487616,"num_objects":5246,"num_object_clones":0,"num_object_copies":10492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2192,"num_write_kb":8768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1152},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5274","reported_seq":5401,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:45.144251+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:45.144251+0000","last_peered":"2024-09-15T21:21:45.144251+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:45.144251+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5274,"log_dups_size":0,"ondisk_log_size":5274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21569536,"num_objects":5266,"num_object_clones":0,"num_object_copies":10532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2183,"num_write_kb":8732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1127},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5205","reported_seq":5334,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:45.103367+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:45.103367+0000","last_peered":"2024-09-15T21:21:45.103367+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:45.103367+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5205,"log_dups_size":0,"ondisk_log_size":5205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21303296,"num_objects":5201,"num_object_clones":0,"num_object_copies":10402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1689,"num_write_kb":6756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1119},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5321","reported_seq":5454,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:45.117110+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:45.117110+0000","last_peered":"2024-09-15T21:21:45.117110+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:45.117110+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5321,"log_dups_size":0,"ondisk_log_size":5321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21746630,"num_objects":5311,"num_object_clones":0,"num_object_copies":10622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5311,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2025,"num_write_kb":8100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1164},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":346620638,"num_objects":84633,"num_object_clones":0,"num_object_copies":169266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3358,"num_objects_unfound":0,"num_objects_dirty":84633,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":44498,"num_write_kb":177992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":805,"num_bytes_recovered":3297280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":695648256,"data_stored":695492897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84705,"ondisk_log_size":84705,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469344,"kb_used_data":302992,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93902496,"statfs":{"total":96636764160,"available":96156155904,"internally_reserved":0,"allocated":310263808,"data_stored":310001794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":170323625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,3,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182426,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502128,"kb_used_data":317792,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93869712,"statfs":{"total":96636764160,"available":96122585088,"internally_reserved":0,"allocated":325419008,"data_stored":325152917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188739243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,2,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":161636,"kb_used_data":57748,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94210204,"statfs":{"total":96636764160,"available":96471248896,"internally_reserved":0,"allocated":59133952,"data_stored":58820892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":106360495},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":325689344,"data_stored":325653214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":59654144,"data_stored":59571045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310304768,"data_stored":310268638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:50.269 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:50.742 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 79 20 87156 87136 4.30803 2.07422 0.064985 0.0181259 2024-09-15T21:21:50.818 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:50.822 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:50.853 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-15T21:21:50.700043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":353960689,"num_objects":86427,"num_object_clones":0,"num_object_copies":172854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3358,"num_objects_unfound":0,"num_objects_dirty":86427,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":46292,"num_write_kb":185162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":806,"num_bytes_recovered":3297299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86500,"ondisk_log_size":86500,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1129336,"kb_used_data":687240,"kb_used_omap":12,"kb_used_meta":442035,"kb_avail":281986184,"statfs":{"total":289910292480,"available":288753852416,"internally_reserved":0,"allocated":703733760,"data_stored":702892595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":452643843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,6,4,9,8,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31289344,"num_objects":7639,"num_object_clones":0,"num_object_copies":15278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-251,"num_objects_unfound":0,"num_objects_dirty":7639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7639,"num_write_kb":30556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":1028096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002649"},"pg_stats":[{"pgid":"2.f","version":"43'5431","reported_seq":5500,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.120210+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:50.120210+0000","last_peered":"2024-09-15T21:21:50.120210+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:50.120210+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22245376,"num_objects":5431,"num_object_clones":0,"num_object_copies":10862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5431,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1299},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5310","reported_seq":5410,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:48.097615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:48.097615+0000","last_peered":"2024-09-15T21:21:48.097615+0000","last_clean":"2024-09-15T21:21:48.097615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:48.097615+0000","last_undegraded":"2024-09-15T21:21:48.097615+0000","last_fullsized":"2024-09-15T21:21:48.097615+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5310,"log_dups_size":0,"ondisk_log_size":5310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21726150,"num_objects":5306,"num_object_clones":0,"num_object_copies":10612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5306,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3591,"num_write_kb":14364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5516","reported_seq":5635,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:48.143143+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:48.143143+0000","last_peered":"2024-09-15T21:21:48.143143+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:48.143143+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5516,"log_dups_size":0,"ondisk_log_size":5516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22577152,"num_objects":5512,"num_object_clones":0,"num_object_copies":11024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3616,"num_write_kb":14464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5304","reported_seq":5369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:48.108566+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:48.108566+0000","last_peered":"2024-09-15T21:21:48.108566+0000","last_clean":"2024-09-15T21:21:48.108566+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:48.108566+0000","last_undegraded":"2024-09-15T21:21:48.108566+0000","last_fullsized":"2024-09-15T21:21:48.108566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5304,"log_dups_size":0,"ondisk_log_size":5304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21725184,"num_objects":5304,"num_object_clones":0,"num_object_copies":10608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5304,"num_write_kb":21216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5381","reported_seq":5508,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:50.057030+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:50.057030+0000","last_peered":"2024-09-15T21:21:50.057030+0000","last_clean":"2024-09-15T21:21:50.057030+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:50.057030+0000","last_undegraded":"2024-09-15T21:21:50.057030+0000","last_fullsized":"2024-09-15T21:21:50.057030+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22012387,"num_objects":5375,"num_object_clones":0,"num_object_copies":10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1836,"num_write_kb":7344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5379","reported_seq":5510,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.082894+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:50.082894+0000","last_peered":"2024-09-15T21:21:50.082894+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:50.082894+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5379,"log_dups_size":0,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22012387,"num_objects":5375,"num_object_clones":0,"num_object_copies":10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5375,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2055,"num_write_kb":8220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1299},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5468","reported_seq":5599,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.159010+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:50.159010+0000","last_peered":"2024-09-15T21:21:50.159010+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:50.159010+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5468,"log_dups_size":0,"ondisk_log_size":5468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22373318,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1779,"num_write_kb":7116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1303},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5564","reported_seq":5687,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:50.134978+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:50.134978+0000","last_peered":"2024-09-15T21:21:50.134978+0000","last_clean":"2024-09-15T21:21:50.134978+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:50.134978+0000","last_undegraded":"2024-09-15T21:21:50.134978+0000","last_fullsized":"2024-09-15T21:21:50.134978+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5564,"log_dups_size":0,"ondisk_log_size":5564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22781952,"num_objects":5562,"num_object_clones":0,"num_object_copies":11124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2336,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5309","reported_seq":5430,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:48.135217+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:48.135217+0000","last_peered":"2024-09-15T21:21:48.135217+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:48.135217+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5309,"log_dups_size":0,"ondisk_log_size":5309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21717475,"num_objects":5303,"num_object_clones":0,"num_object_copies":10606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5303,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3241,"num_write_kb":12964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5374","reported_seq":5499,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.094055+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:50.094055+0000","last_peered":"2024-09-15T21:21:50.094055+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:50.094055+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5374,"log_dups_size":0,"ondisk_log_size":5374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21979136,"num_objects":5366,"num_object_clones":0,"num_object_copies":10732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2047,"num_write_kb":8188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1346},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5335","reported_seq":5448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:48.070593+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:48.070593+0000","last_peered":"2024-09-15T21:21:48.070593+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:48.070593+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21832163,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3194,"num_write_kb":12776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1241}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5370","reported_seq":7096,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:48.120174+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:48.120174+0000","last_peered":"2024-09-15T21:21:48.120174+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:48.120174+0000","last_undegraded":"2024-09-15T21:21:48.120174+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5370,"log_dups_size":0,"ondisk_log_size":5370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21970944,"num_objects":5364,"num_object_clones":0,"num_object_copies":10728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3358,"num_objects_unfound":0,"num_objects_dirty":5364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3063,"num_write_kb":12252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":804,"num_bytes_recovered":3293184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3358},{"shards":"2","objects":2006}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5428","reported_seq":5556,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.166242+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:50.166242+0000","last_peered":"2024-09-15T21:21:50.166242+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:50.166242+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5428,"log_dups_size":0,"ondisk_log_size":5428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22224896,"num_objects":5426,"num_object_clones":0,"num_object_copies":10852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1332},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5456","reported_seq":5583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.071320+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:50.071320+0000","last_peered":"2024-09-15T21:21:50.071320+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:50.071320+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22315008,"num_objects":5448,"num_object_clones":0,"num_object_copies":10896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2365,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1309},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5374","reported_seq":5504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.117651+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:50.117651+0000","last_peered":"2024-09-15T21:21:50.117651+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:50.117651+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5374,"log_dups_size":0,"ondisk_log_size":5374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21995520,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1858,"num_write_kb":7432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1288},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5498","reported_seq":5631,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.132310+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:50.132310+0000","last_peered":"2024-09-15T21:21:50.132310+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:50.132310+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5498,"log_dups_size":0,"ondisk_log_size":5498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22471622,"num_objects":5488,"num_object_clones":0,"num_object_copies":10976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5488,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2202,"num_write_kb":8808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1341},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":353960670,"num_objects":86425,"num_object_clones":0,"num_object_copies":172850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3358,"num_objects_unfound":0,"num_objects_dirty":86425,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":46290,"num_write_kb":185160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":805,"num_bytes_recovered":3297280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704507904,"data_stored":704352545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86497,"ondisk_log_size":86497,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469344,"kb_used_data":302992,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93902496,"statfs":{"total":96636764160,"available":96156155904,"internally_reserved":0,"allocated":310263808,"data_stored":310001794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":170323625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,3,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498356,"kb_used_data":326500,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93873484,"statfs":{"total":96636764160,"available":96126447616,"internally_reserved":0,"allocated":334336000,"data_stored":334069909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":175959723},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,2,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":161636,"kb_used_data":57748,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94210204,"statfs":{"total":96636764160,"available":96471248896,"internally_reserved":0,"allocated":59133952,"data_stored":58820892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":106360495},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":334548992,"data_stored":334512862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":59654144,"data_stored":59571045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310304768,"data_stored":310268638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:50.855 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:50.855 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'5431", 'reported_seq': 5500, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.120210+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:21:50.120210+0000', 'last_peered': '2024-09-15T21:21:50.120210+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:21:50.120210+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 5431, 'log_dups_size': 0, 'ondisk_log_size': 5431, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22245376, 'num_objects': 5431, 'num_object_clones': 0, 'num_object_copies': 10862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5431, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5431, 'num_write_kb': 21724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1299}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.855 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:21:50.855 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'5516", 'reported_seq': 5635, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:48.143143+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:21:48.143143+0000', 'last_peered': '2024-09-15T21:21:48.143143+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:21:48.143143+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 5516, 'log_dups_size': 0, 'ondisk_log_size': 5516, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22577152, 'num_objects': 5512, 'num_object_clones': 0, 'num_object_copies': 11024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5512, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3616, 'num_write_kb': 14464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 1245}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:50.855 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:50.855 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'5379", 'reported_seq': 5510, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.082894+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:21:50.082894+0000', 'last_peered': '2024-09-15T21:21:50.082894+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:21:50.082894+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 5379, 'log_dups_size': 0, 'ondisk_log_size': 5379, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22012387, 'num_objects': 5375, 'num_object_clones': 0, 'num_object_copies': 10750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5375, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2055, 'num_write_kb': 8220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1299}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.855 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:50.856 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'5468", 'reported_seq': 5599, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.159010+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:21:50.159010+0000', 'last_peered': '2024-09-15T21:21:50.159010+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:21:50.159010+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 5468, 'log_dups_size': 0, 'ondisk_log_size': 5468, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22373318, 'num_objects': 5464, 'num_object_clones': 0, 'num_object_copies': 10928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1779, 'num_write_kb': 7116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1303}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.856 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:50.856 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'5309", 'reported_seq': 5430, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:48.135217+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:21:48.135217+0000', 'last_peered': '2024-09-15T21:21:48.135217+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:21:48.135217+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 5309, 'log_dups_size': 0, 'ondisk_log_size': 5309, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21717475, 'num_objects': 5303, 'num_object_clones': 0, 'num_object_copies': 10606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5303, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3241, 'num_write_kb': 12964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 1216}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:50.856 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:50.856 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'5374", 'reported_seq': 5499, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.094055+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:21:50.094055+0000', 'last_peered': '2024-09-15T21:21:50.094055+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:21:50.094055+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 5374, 'log_dups_size': 0, 'ondisk_log_size': 5374, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21979136, 'num_objects': 5366, 'num_object_clones': 0, 'num_object_copies': 10732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5366, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2047, 'num_write_kb': 8188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1346}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.856 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:50.856 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'5335", 'reported_seq': 5448, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:48.070593+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:21:48.070593+0000', 'last_peered': '2024-09-15T21:21:48.070593+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:21:48.070593+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 5335, 'log_dups_size': 0, 'ondisk_log_size': 5335, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21832163, 'num_objects': 5331, 'num_object_clones': 0, 'num_object_copies': 10662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5331, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3194, 'num_write_kb': 12776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 1241}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:50.856 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:50.856 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'5370", 'reported_seq': 7096, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:48.120174+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:21:48.120174+0000', 'last_peered': '2024-09-15T21:21:48.120174+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:21:48.120174+0000', 'last_undegraded': '2024-09-15T21:21:48.120174+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 5370, 'log_dups_size': 0, 'ondisk_log_size': 5370, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21970944, 'num_objects': 5364, 'num_object_clones': 0, 'num_object_copies': 10728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3358, 'num_objects_unfound': 0, 'num_objects_dirty': 5364, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3063, 'num_write_kb': 12252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 804, 'num_bytes_recovered': 3293184, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3358}, {'shards': '2', 'objects': 2006}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:50.856 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:50.857 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'5428", 'reported_seq': 5556, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.166242+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:21:50.166242+0000', 'last_peered': '2024-09-15T21:21:50.166242+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:21:50.166242+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 5428, 'log_dups_size': 0, 'ondisk_log_size': 5428, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22224896, 'num_objects': 5426, 'num_object_clones': 0, 'num_object_copies': 10852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2372, 'num_write_kb': 9488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1332}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.857 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:50.857 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'5456", 'reported_seq': 5583, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.071320+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:21:50.071320+0000', 'last_peered': '2024-09-15T21:21:50.071320+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:21:50.071320+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 5456, 'log_dups_size': 0, 'ondisk_log_size': 5456, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22315008, 'num_objects': 5448, 'num_object_clones': 0, 'num_object_copies': 10896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2365, 'num_write_kb': 9460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1309}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.857 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:50.857 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'5374", 'reported_seq': 5504, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.117651+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:21:50.117651+0000', 'last_peered': '2024-09-15T21:21:50.117651+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:21:50.117651+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 5374, 'log_dups_size': 0, 'ondisk_log_size': 5374, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21995520, 'num_objects': 5370, 'num_object_clones': 0, 'num_object_copies': 10740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5370, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1858, 'num_write_kb': 7432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1288}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.857 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:50.857 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'5498", 'reported_seq': 5631, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.132310+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:21:50.132310+0000', 'last_peered': '2024-09-15T21:21:50.132310+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:21:50.132310+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 5498, 'log_dups_size': 0, 'ondisk_log_size': 5498, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22471622, 'num_objects': 5488, 'num_object_clones': 0, 'num_object_copies': 10976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5488, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2202, 'num_write_kb': 8808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1341}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:50.858 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:51.545 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/172854 objects degraded (26.212%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":163,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":86427,"data_bytes":353960689,"bytes_used":1156440064,"bytes_avail":288753852416,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":172854,"degraded_ratio":0.26212294768995797,"misplaced_objects":3358,"misplaced_total":172854,"misplaced_ratio":0.019426799495528019,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":85655,"recovering_keys_per_sec":0,"num_objects_recovered":251,"num_bytes_recovered":1028096,"num_keys_recovered":0,"write_bytes_sec":2606869,"read_op_per_sec":0,"write_op_per_sec":636},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-15T21:21:18.693002+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:21:51.566 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:51.566 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:51.741 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:2024-09-15T21:21:51.737406+0000 min lat: 0.00235499 max lat: 2.60418 avg lat: 0.0182552 2024-09-15T21:21:51.741 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:21:51.741 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 80 20 87633 87613 4.27747 1.86328 0.0345972 0.0182552 2024-09-15T21:21:52.138 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:52.139 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:52.168 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":117,"stamp":"2024-09-15T21:21:50.700043+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":353960689,"num_objects":86427,"num_object_clones":0,"num_object_copies":172854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3358,"num_objects_unfound":0,"num_objects_dirty":86427,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":46292,"num_write_kb":185162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":806,"num_bytes_recovered":3297299,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86500,"ondisk_log_size":86500,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1129336,"kb_used_data":687240,"kb_used_omap":12,"kb_used_meta":442035,"kb_avail":281986184,"statfs":{"total":289910292480,"available":288753852416,"internally_reserved":0,"allocated":703733760,"data_stored":702892595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":452643843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,6,4,9,8,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31289344,"num_objects":7639,"num_object_clones":0,"num_object_copies":15278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-251,"num_objects_unfound":0,"num_objects_dirty":7639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7639,"num_write_kb":30556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":1028096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002649"},"pg_stats":[{"pgid":"2.f","version":"43'5431","reported_seq":5500,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.120210+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:50.120210+0000","last_peered":"2024-09-15T21:21:50.120210+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:50.120210+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22245376,"num_objects":5431,"num_object_clones":0,"num_object_copies":10862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5431,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1299},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5310","reported_seq":5410,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:48.097615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:48.097615+0000","last_peered":"2024-09-15T21:21:48.097615+0000","last_clean":"2024-09-15T21:21:48.097615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:48.097615+0000","last_undegraded":"2024-09-15T21:21:48.097615+0000","last_fullsized":"2024-09-15T21:21:48.097615+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5310,"log_dups_size":0,"ondisk_log_size":5310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21726150,"num_objects":5306,"num_object_clones":0,"num_object_copies":10612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5306,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3591,"num_write_kb":14364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5516","reported_seq":5635,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:48.143143+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:48.143143+0000","last_peered":"2024-09-15T21:21:48.143143+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:48.143143+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5516,"log_dups_size":0,"ondisk_log_size":5516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22577152,"num_objects":5512,"num_object_clones":0,"num_object_copies":11024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3616,"num_write_kb":14464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5304","reported_seq":5369,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:48.108566+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:48.108566+0000","last_peered":"2024-09-15T21:21:48.108566+0000","last_clean":"2024-09-15T21:21:48.108566+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:48.108566+0000","last_undegraded":"2024-09-15T21:21:48.108566+0000","last_fullsized":"2024-09-15T21:21:48.108566+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5304,"log_dups_size":0,"ondisk_log_size":5304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21725184,"num_objects":5304,"num_object_clones":0,"num_object_copies":10608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5304,"num_write_kb":21216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5381","reported_seq":5508,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:50.057030+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:50.057030+0000","last_peered":"2024-09-15T21:21:50.057030+0000","last_clean":"2024-09-15T21:21:50.057030+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:50.057030+0000","last_undegraded":"2024-09-15T21:21:50.057030+0000","last_fullsized":"2024-09-15T21:21:50.057030+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22012387,"num_objects":5375,"num_object_clones":0,"num_object_copies":10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1836,"num_write_kb":7344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5379","reported_seq":5510,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.082894+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:50.082894+0000","last_peered":"2024-09-15T21:21:50.082894+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:50.082894+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5379,"log_dups_size":0,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22012387,"num_objects":5375,"num_object_clones":0,"num_object_copies":10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5375,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2055,"num_write_kb":8220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1299},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5468","reported_seq":5599,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.159010+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:50.159010+0000","last_peered":"2024-09-15T21:21:50.159010+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:50.159010+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5468,"log_dups_size":0,"ondisk_log_size":5468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22373318,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1779,"num_write_kb":7116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1303},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5564","reported_seq":5687,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:50.134978+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:50.134978+0000","last_peered":"2024-09-15T21:21:50.134978+0000","last_clean":"2024-09-15T21:21:50.134978+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:50.134978+0000","last_undegraded":"2024-09-15T21:21:50.134978+0000","last_fullsized":"2024-09-15T21:21:50.134978+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5564,"log_dups_size":0,"ondisk_log_size":5564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22781952,"num_objects":5562,"num_object_clones":0,"num_object_copies":11124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2336,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5309","reported_seq":5430,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:48.135217+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:48.135217+0000","last_peered":"2024-09-15T21:21:48.135217+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:48.135217+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5309,"log_dups_size":0,"ondisk_log_size":5309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21717475,"num_objects":5303,"num_object_clones":0,"num_object_copies":10606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5303,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3241,"num_write_kb":12964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5374","reported_seq":5499,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.094055+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:50.094055+0000","last_peered":"2024-09-15T21:21:50.094055+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:50.094055+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5374,"log_dups_size":0,"ondisk_log_size":5374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21979136,"num_objects":5366,"num_object_clones":0,"num_object_copies":10732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2047,"num_write_kb":8188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1346},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5335","reported_seq":5448,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:48.070593+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:48.070593+0000","last_peered":"2024-09-15T21:21:48.070593+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:48.070593+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21832163,"num_objects":5331,"num_object_clones":0,"num_object_copies":10662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5331,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3194,"num_write_kb":12776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1241}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5370","reported_seq":7096,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:48.120174+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:48.120174+0000","last_peered":"2024-09-15T21:21:48.120174+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:48.120174+0000","last_undegraded":"2024-09-15T21:21:48.120174+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5370,"log_dups_size":0,"ondisk_log_size":5370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21970944,"num_objects":5364,"num_object_clones":0,"num_object_copies":10728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3358,"num_objects_unfound":0,"num_objects_dirty":5364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3063,"num_write_kb":12252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":804,"num_bytes_recovered":3293184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3358},{"shards":"2","objects":2006}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5428","reported_seq":5556,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.166242+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:50.166242+0000","last_peered":"2024-09-15T21:21:50.166242+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:50.166242+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5428,"log_dups_size":0,"ondisk_log_size":5428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22224896,"num_objects":5426,"num_object_clones":0,"num_object_copies":10852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1332},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5456","reported_seq":5583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.071320+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:50.071320+0000","last_peered":"2024-09-15T21:21:50.071320+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:50.071320+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22315008,"num_objects":5448,"num_object_clones":0,"num_object_copies":10896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2365,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1309},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5374","reported_seq":5504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.117651+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:50.117651+0000","last_peered":"2024-09-15T21:21:50.117651+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:50.117651+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5374,"log_dups_size":0,"ondisk_log_size":5374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21995520,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1858,"num_write_kb":7432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1288},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5498","reported_seq":5631,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.132310+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:50.132310+0000","last_peered":"2024-09-15T21:21:50.132310+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:50.132310+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5498,"log_dups_size":0,"ondisk_log_size":5498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22471622,"num_objects":5488,"num_object_clones":0,"num_object_copies":10976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5488,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2202,"num_write_kb":8808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1341},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":353960670,"num_objects":86425,"num_object_clones":0,"num_object_copies":172850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3358,"num_objects_unfound":0,"num_objects_dirty":86425,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":46290,"num_write_kb":185160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":805,"num_bytes_recovered":3297280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":704507904,"data_stored":704352545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86497,"ondisk_log_size":86497,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469344,"kb_used_data":302992,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93902496,"statfs":{"total":96636764160,"available":96156155904,"internally_reserved":0,"allocated":310263808,"data_stored":310001794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":170323625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,3,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498356,"kb_used_data":326500,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93873484,"statfs":{"total":96636764160,"available":96126447616,"internally_reserved":0,"allocated":334336000,"data_stored":334069909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":175959723},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,2,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":161636,"kb_used_data":57748,"kb_used_omap":4,"kb_used_meta":103867,"kb_avail":94210204,"statfs":{"total":96636764160,"available":96471248896,"internally_reserved":0,"allocated":59133952,"data_stored":58820892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":106360495},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":334548992,"data_stored":334512862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":59654144,"data_stored":59571045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310304768,"data_stored":310268638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:52.739 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 81 20 88205 88185 4.25225 2.23438 0.0477991 0.0183646 2024-09-15T21:21:53.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 82 20 88790 88770 4.22826 2.28516 0.0352271 0.0184701 2024-09-15T21:21:54.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 83 20 89372 89352 4.2047 2.27344 0.0324477 0.0185722 2024-09-15T21:21:55.171 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:55.719 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:55.722 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:55.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 84 20 89938 89918 4.18097 2.21094 0.0554456 0.0186789 2024-09-15T21:21:55.753 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-15T21:21:54.700966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":358171377,"num_objects":87455,"num_object_clones":0,"num_object_copies":174910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":87455,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":47320,"num_write_kb":189274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":930,"num_bytes_recovered":3805203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87528,"ondisk_log_size":87528,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1135376,"kb_used_data":701600,"kb_used_omap":12,"kb_used_meta":433715,"kb_avail":281980144,"statfs":{"total":289910292480,"available":288747667456,"internally_reserved":0,"allocated":718438400,"data_stored":717597235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":444124163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,6,5,10,10,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27983872,"num_objects":6832,"num_object_clones":0,"num_object_copies":13664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-375,"num_objects_unfound":0,"num_objects_dirty":6832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6832,"num_write_kb":27328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":1536000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002647"},"pg_stats":[{"pgid":"2.f","version":"43'5431","reported_seq":5500,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.120210+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:50.120210+0000","last_peered":"2024-09-15T21:21:50.120210+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:50.120210+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22245376,"num_objects":5431,"num_object_clones":0,"num_object_copies":10862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5431,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1299},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5493","reported_seq":5593,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:53.099932+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:53.099932+0000","last_peered":"2024-09-15T21:21:53.099932+0000","last_clean":"2024-09-15T21:21:53.099932+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:53.099932+0000","last_undegraded":"2024-09-15T21:21:53.099932+0000","last_fullsized":"2024-09-15T21:21:53.099932+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5493,"log_dups_size":0,"ondisk_log_size":5493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22475718,"num_objects":5489,"num_object_clones":0,"num_object_copies":10978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5489,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3774,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5694","reported_seq":5810,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:53.151960+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:53.151960+0000","last_peered":"2024-09-15T21:21:53.151960+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:53.151960+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5694,"log_dups_size":0,"ondisk_log_size":5694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23306240,"num_objects":5690,"num_object_clones":0,"num_object_copies":11380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3794,"num_write_kb":15176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1423}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5456","reported_seq":5521,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:53.157239+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:53.157239+0000","last_peered":"2024-09-15T21:21:53.157239+0000","last_clean":"2024-09-15T21:21:53.157239+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:53.157239+0000","last_undegraded":"2024-09-15T21:21:53.157239+0000","last_fullsized":"2024-09-15T21:21:53.157239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":5456,"num_object_clones":0,"num_object_copies":10912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5456,"num_write_kb":21824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5381","reported_seq":5508,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:50.057030+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:50.057030+0000","last_peered":"2024-09-15T21:21:50.057030+0000","last_clean":"2024-09-15T21:21:50.057030+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:50.057030+0000","last_undegraded":"2024-09-15T21:21:50.057030+0000","last_fullsized":"2024-09-15T21:21:50.057030+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22012387,"num_objects":5375,"num_object_clones":0,"num_object_copies":10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1836,"num_write_kb":7344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5379","reported_seq":5510,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.082894+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:50.082894+0000","last_peered":"2024-09-15T21:21:50.082894+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:50.082894+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5379,"log_dups_size":0,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22012387,"num_objects":5375,"num_object_clones":0,"num_object_copies":10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5375,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2055,"num_write_kb":8220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1299},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5468","reported_seq":5599,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.159010+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:50.159010+0000","last_peered":"2024-09-15T21:21:50.159010+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:50.159010+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5468,"log_dups_size":0,"ondisk_log_size":5468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22373318,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1779,"num_write_kb":7116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1303},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5564","reported_seq":5687,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:50.134978+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:50.134978+0000","last_peered":"2024-09-15T21:21:50.134978+0000","last_clean":"2024-09-15T21:21:50.134978+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:50.134978+0000","last_undegraded":"2024-09-15T21:21:50.134978+0000","last_fullsized":"2024-09-15T21:21:50.134978+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5564,"log_dups_size":0,"ondisk_log_size":5564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22781952,"num_objects":5562,"num_object_clones":0,"num_object_copies":11124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2336,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5487","reported_seq":5608,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:53.117929+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:53.117929+0000","last_peered":"2024-09-15T21:21:53.117929+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:53.117929+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5487,"log_dups_size":0,"ondisk_log_size":5487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22446563,"num_objects":5481,"num_object_clones":0,"num_object_copies":10962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5481,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3419,"num_write_kb":13676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1394}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5374","reported_seq":5499,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.094055+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:50.094055+0000","last_peered":"2024-09-15T21:21:50.094055+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:50.094055+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5374,"log_dups_size":0,"ondisk_log_size":5374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21979136,"num_objects":5366,"num_object_clones":0,"num_object_copies":10732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2047,"num_write_kb":8188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1346},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5500","reported_seq":5613,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:53.182039+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:53.182039+0000","last_peered":"2024-09-15T21:21:53.182039+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:53.182039+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5500,"log_dups_size":0,"ondisk_log_size":5500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22508003,"num_objects":5496,"num_object_clones":0,"num_object_copies":10992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5496,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3359,"num_write_kb":13436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1406}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5542","reported_seq":7515,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:53.163675+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:53.163675+0000","last_peered":"2024-09-15T21:21:53.163675+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:53.163675+0000","last_undegraded":"2024-09-15T21:21:53.163675+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5542,"log_dups_size":0,"ondisk_log_size":5542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22675456,"num_objects":5536,"num_object_clones":0,"num_object_copies":11072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":5536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3235,"num_write_kb":12940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":928,"num_bytes_recovered":3801088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3234},{"shards":"2","objects":2302}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5428","reported_seq":5556,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.166242+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:50.166242+0000","last_peered":"2024-09-15T21:21:50.166242+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:50.166242+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5428,"log_dups_size":0,"ondisk_log_size":5428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22224896,"num_objects":5426,"num_object_clones":0,"num_object_copies":10852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1332},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5456","reported_seq":5583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.071320+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:50.071320+0000","last_peered":"2024-09-15T21:21:50.071320+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:50.071320+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22315008,"num_objects":5448,"num_object_clones":0,"num_object_copies":10896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2365,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1309},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5374","reported_seq":5504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.117651+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:50.117651+0000","last_peered":"2024-09-15T21:21:50.117651+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:50.117651+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5374,"log_dups_size":0,"ondisk_log_size":5374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21995520,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1858,"num_write_kb":7432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1288},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5498","reported_seq":5631,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.132310+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:50.132310+0000","last_peered":"2024-09-15T21:21:50.132310+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:50.132310+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5498,"log_dups_size":0,"ondisk_log_size":5498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22471622,"num_objects":5488,"num_object_clones":0,"num_object_copies":10976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5488,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2202,"num_write_kb":8808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1341},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":358171358,"num_objects":87453,"num_object_clones":0,"num_object_copies":174906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":87453,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":47318,"num_write_kb":189272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":929,"num_bytes_recovered":3805184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":719454208,"data_stored":719298849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87525,"ondisk_log_size":87525,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462340,"kb_used_data":308468,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93909500,"statfs":{"total":96636764160,"available":96163328000,"internally_reserved":0,"allocated":315871232,"data_stored":315609218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":157544105},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498356,"kb_used_data":326500,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93873484,"statfs":{"total":96636764160,"available":96126447616,"internally_reserved":0,"allocated":334336000,"data_stored":334069909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":175959723},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,2,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":174680,"kb_used_data":66632,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94197160,"statfs":{"total":96636764160,"available":96457891840,"internally_reserved":0,"allocated":68231168,"data_stored":67918108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":110620335},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":334548992,"data_stored":334512862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":68915200,"data_stored":68832101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":315990016,"data_stored":315953886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:55.755 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:56.313 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:56.315 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:56.335 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-15T21:21:54.700966+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":358171377,"num_objects":87455,"num_object_clones":0,"num_object_copies":174910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":87455,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":47320,"num_write_kb":189274,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":930,"num_bytes_recovered":3805203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87528,"ondisk_log_size":87528,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1135376,"kb_used_data":701600,"kb_used_omap":12,"kb_used_meta":433715,"kb_avail":281980144,"statfs":{"total":289910292480,"available":288747667456,"internally_reserved":0,"allocated":718438400,"data_stored":717597235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":444124163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,6,5,10,10,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27983872,"num_objects":6832,"num_object_clones":0,"num_object_copies":13664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-375,"num_objects_unfound":0,"num_objects_dirty":6832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6832,"num_write_kb":27328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":1536000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002647"},"pg_stats":[{"pgid":"2.f","version":"43'5431","reported_seq":5500,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.120210+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:50.120210+0000","last_peered":"2024-09-15T21:21:50.120210+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:50.120210+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5431,"log_dups_size":0,"ondisk_log_size":5431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22245376,"num_objects":5431,"num_object_clones":0,"num_object_copies":10862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5431,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1299},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5493","reported_seq":5593,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:53.099932+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:53.099932+0000","last_peered":"2024-09-15T21:21:53.099932+0000","last_clean":"2024-09-15T21:21:53.099932+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:53.099932+0000","last_undegraded":"2024-09-15T21:21:53.099932+0000","last_fullsized":"2024-09-15T21:21:53.099932+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5493,"log_dups_size":0,"ondisk_log_size":5493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22475718,"num_objects":5489,"num_object_clones":0,"num_object_copies":10978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5489,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3774,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5694","reported_seq":5810,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:53.151960+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:53.151960+0000","last_peered":"2024-09-15T21:21:53.151960+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:53.151960+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5694,"log_dups_size":0,"ondisk_log_size":5694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23306240,"num_objects":5690,"num_object_clones":0,"num_object_copies":11380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3794,"num_write_kb":15176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1423}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5456","reported_seq":5521,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:53.157239+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:53.157239+0000","last_peered":"2024-09-15T21:21:53.157239+0000","last_clean":"2024-09-15T21:21:53.157239+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:53.157239+0000","last_undegraded":"2024-09-15T21:21:53.157239+0000","last_fullsized":"2024-09-15T21:21:53.157239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":5456,"num_object_clones":0,"num_object_copies":10912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5456,"num_write_kb":21824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5381","reported_seq":5508,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:50.057030+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:50.057030+0000","last_peered":"2024-09-15T21:21:50.057030+0000","last_clean":"2024-09-15T21:21:50.057030+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:50.057030+0000","last_undegraded":"2024-09-15T21:21:50.057030+0000","last_fullsized":"2024-09-15T21:21:50.057030+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5381,"log_dups_size":0,"ondisk_log_size":5381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22012387,"num_objects":5375,"num_object_clones":0,"num_object_copies":10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1836,"num_write_kb":7344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5379","reported_seq":5510,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.082894+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:50.082894+0000","last_peered":"2024-09-15T21:21:50.082894+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:50.082894+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5379,"log_dups_size":0,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22012387,"num_objects":5375,"num_object_clones":0,"num_object_copies":10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5375,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2055,"num_write_kb":8220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1299},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5468","reported_seq":5599,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.159010+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:50.159010+0000","last_peered":"2024-09-15T21:21:50.159010+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:50.159010+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5468,"log_dups_size":0,"ondisk_log_size":5468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22373318,"num_objects":5464,"num_object_clones":0,"num_object_copies":10928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1779,"num_write_kb":7116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1303},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5564","reported_seq":5687,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:50.134978+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:50.134978+0000","last_peered":"2024-09-15T21:21:50.134978+0000","last_clean":"2024-09-15T21:21:50.134978+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:50.134978+0000","last_undegraded":"2024-09-15T21:21:50.134978+0000","last_fullsized":"2024-09-15T21:21:50.134978+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5564,"log_dups_size":0,"ondisk_log_size":5564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22781952,"num_objects":5562,"num_object_clones":0,"num_object_copies":11124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2336,"num_write_kb":9344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5487","reported_seq":5608,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:53.117929+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:53.117929+0000","last_peered":"2024-09-15T21:21:53.117929+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:53.117929+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5487,"log_dups_size":0,"ondisk_log_size":5487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22446563,"num_objects":5481,"num_object_clones":0,"num_object_copies":10962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5481,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3419,"num_write_kb":13676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1394}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5374","reported_seq":5499,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.094055+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:50.094055+0000","last_peered":"2024-09-15T21:21:50.094055+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:50.094055+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5374,"log_dups_size":0,"ondisk_log_size":5374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21979136,"num_objects":5366,"num_object_clones":0,"num_object_copies":10732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2047,"num_write_kb":8188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1346},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5500","reported_seq":5613,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:53.182039+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:53.182039+0000","last_peered":"2024-09-15T21:21:53.182039+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:53.182039+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5500,"log_dups_size":0,"ondisk_log_size":5500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22508003,"num_objects":5496,"num_object_clones":0,"num_object_copies":10992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5496,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3359,"num_write_kb":13436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1406}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5542","reported_seq":7515,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:53.163675+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:53.163675+0000","last_peered":"2024-09-15T21:21:53.163675+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:53.163675+0000","last_undegraded":"2024-09-15T21:21:53.163675+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5542,"log_dups_size":0,"ondisk_log_size":5542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22675456,"num_objects":5536,"num_object_clones":0,"num_object_copies":11072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":5536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3235,"num_write_kb":12940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":928,"num_bytes_recovered":3801088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3234},{"shards":"2","objects":2302}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5428","reported_seq":5556,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.166242+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:50.166242+0000","last_peered":"2024-09-15T21:21:50.166242+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:50.166242+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5428,"log_dups_size":0,"ondisk_log_size":5428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22224896,"num_objects":5426,"num_object_clones":0,"num_object_copies":10852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1332},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5456","reported_seq":5583,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.071320+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:50.071320+0000","last_peered":"2024-09-15T21:21:50.071320+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:50.071320+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22315008,"num_objects":5448,"num_object_clones":0,"num_object_copies":10896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2365,"num_write_kb":9460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1309},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5374","reported_seq":5504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.117651+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:50.117651+0000","last_peered":"2024-09-15T21:21:50.117651+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:50.117651+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5374,"log_dups_size":0,"ondisk_log_size":5374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21995520,"num_objects":5370,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1858,"num_write_kb":7432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1288},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5498","reported_seq":5631,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:50.132310+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:50.132310+0000","last_peered":"2024-09-15T21:21:50.132310+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:50.132310+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5498,"log_dups_size":0,"ondisk_log_size":5498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22471622,"num_objects":5488,"num_object_clones":0,"num_object_copies":10976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5488,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2202,"num_write_kb":8808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1341},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":358171358,"num_objects":87453,"num_object_clones":0,"num_object_copies":174906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":87453,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":47318,"num_write_kb":189272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":929,"num_bytes_recovered":3805184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":719454208,"data_stored":719298849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":87525,"ondisk_log_size":87525,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462340,"kb_used_data":308468,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93909500,"statfs":{"total":96636764160,"available":96163328000,"internally_reserved":0,"allocated":315871232,"data_stored":315609218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":157544105},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498356,"kb_used_data":326500,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93873484,"statfs":{"total":96636764160,"available":96126447616,"internally_reserved":0,"allocated":334336000,"data_stored":334069909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":175959723},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,2,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":174680,"kb_used_data":66632,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94197160,"statfs":{"total":96636764160,"available":96457891840,"internally_reserved":0,"allocated":68231168,"data_stored":67918108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":110620335},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,5,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":334548992,"data_stored":334512862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":68915200,"data_stored":68832101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":315990016,"data_stored":315953886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:56.336 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:21:56.336 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'5431", 'reported_seq': 5500, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.120210+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:21:50.120210+0000', 'last_peered': '2024-09-15T21:21:50.120210+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:21:50.120210+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 5431, 'log_dups_size': 0, 'ondisk_log_size': 5431, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22245376, 'num_objects': 5431, 'num_object_clones': 0, 'num_object_copies': 10862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5431, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5431, 'num_write_kb': 21724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1299}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.336 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:21:56.336 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'5694", 'reported_seq': 5810, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:53.151960+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:21:53.151960+0000', 'last_peered': '2024-09-15T21:21:53.151960+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:21:53.151960+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 5694, 'log_dups_size': 0, 'ondisk_log_size': 5694, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23306240, 'num_objects': 5690, 'num_object_clones': 0, 'num_object_copies': 11380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3794, 'num_write_kb': 15176, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 1423}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:56.336 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:21:56.336 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'5379", 'reported_seq': 5510, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.082894+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:21:50.082894+0000', 'last_peered': '2024-09-15T21:21:50.082894+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:21:50.082894+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 5379, 'log_dups_size': 0, 'ondisk_log_size': 5379, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22012387, 'num_objects': 5375, 'num_object_clones': 0, 'num_object_copies': 10750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5375, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2055, 'num_write_kb': 8220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1299}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.337 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:21:56.337 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'5468", 'reported_seq': 5599, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.159010+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:21:50.159010+0000', 'last_peered': '2024-09-15T21:21:50.159010+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:21:50.159010+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 5468, 'log_dups_size': 0, 'ondisk_log_size': 5468, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22373318, 'num_objects': 5464, 'num_object_clones': 0, 'num_object_copies': 10928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1779, 'num_write_kb': 7116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1303}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.337 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:21:56.337 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'5487", 'reported_seq': 5608, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:53.117929+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:21:53.117929+0000', 'last_peered': '2024-09-15T21:21:53.117929+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:21:53.117929+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 5487, 'log_dups_size': 0, 'ondisk_log_size': 5487, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22446563, 'num_objects': 5481, 'num_object_clones': 0, 'num_object_copies': 10962, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5481, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3419, 'num_write_kb': 13676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 1394}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:56.337 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:21:56.337 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'5374", 'reported_seq': 5499, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.094055+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:21:50.094055+0000', 'last_peered': '2024-09-15T21:21:50.094055+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:21:50.094055+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 5374, 'log_dups_size': 0, 'ondisk_log_size': 5374, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21979136, 'num_objects': 5366, 'num_object_clones': 0, 'num_object_copies': 10732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5366, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2047, 'num_write_kb': 8188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1346}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.337 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:21:56.337 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'5500", 'reported_seq': 5613, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:53.182039+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:21:53.182039+0000', 'last_peered': '2024-09-15T21:21:53.182039+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:21:53.182039+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 5500, 'log_dups_size': 0, 'ondisk_log_size': 5500, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22508003, 'num_objects': 5496, 'num_object_clones': 0, 'num_object_copies': 10992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5496, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3359, 'num_write_kb': 13436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 1406}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:56.337 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:21:56.337 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'5542", 'reported_seq': 7515, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:53.163675+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:21:53.163675+0000', 'last_peered': '2024-09-15T21:21:53.163675+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:21:53.163675+0000', 'last_undegraded': '2024-09-15T21:21:53.163675+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 5542, 'log_dups_size': 0, 'ondisk_log_size': 5542, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22675456, 'num_objects': 5536, 'num_object_clones': 0, 'num_object_copies': 11072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3234, 'num_objects_unfound': 0, 'num_objects_dirty': 5536, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3235, 'num_write_kb': 12940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 928, 'num_bytes_recovered': 3801088, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3234}, {'shards': '2', 'objects': 2302}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:21:56.337 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:21:56.338 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'5428", 'reported_seq': 5556, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.166242+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:21:50.166242+0000', 'last_peered': '2024-09-15T21:21:50.166242+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:21:50.166242+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 5428, 'log_dups_size': 0, 'ondisk_log_size': 5428, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22224896, 'num_objects': 5426, 'num_object_clones': 0, 'num_object_copies': 10852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2372, 'num_write_kb': 9488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1332}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.338 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:21:56.338 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'5456", 'reported_seq': 5583, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.071320+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:21:50.071320+0000', 'last_peered': '2024-09-15T21:21:50.071320+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:21:50.071320+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 5456, 'log_dups_size': 0, 'ondisk_log_size': 5456, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22315008, 'num_objects': 5448, 'num_object_clones': 0, 'num_object_copies': 10896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5448, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2365, 'num_write_kb': 9460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1309}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.338 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:21:56.338 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'5374", 'reported_seq': 5504, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.117651+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:21:50.117651+0000', 'last_peered': '2024-09-15T21:21:50.117651+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:21:50.117651+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 5374, 'log_dups_size': 0, 'ondisk_log_size': 5374, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21995520, 'num_objects': 5370, 'num_object_clones': 0, 'num_object_copies': 10740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5370, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1858, 'num_write_kb': 7432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1288}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.338 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:21:56.338 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'5498", 'reported_seq': 5631, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:50.132310+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:21:50.132310+0000', 'last_peered': '2024-09-15T21:21:50.132310+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:21:50.132310+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 5498, 'log_dups_size': 0, 'ondisk_log_size': 5498, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22471622, 'num_objects': 5488, 'num_object_clones': 0, 'num_object_copies': 10976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5488, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2202, 'num_write_kb': 8808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1341}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:21:56.338 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:21:56.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 85 20 90454 90434 4.15549 2.01562 0.0397127 0.0187923 2024-09-15T21:21:57.015 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:57.015 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/178422 objects degraded (25.394%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":168,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":89211,"data_bytes":365363953,"bytes_used":1193066496,"bytes_avail":288717225984,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":178422,"degraded_ratio":0.25394289941823317,"misplaced_objects":3234,"misplaced_total":178422,"misplaced_ratio":0.018125567474862964,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":87020,"recovering_keys_per_sec":0,"num_objects_recovered":255,"num_bytes_recovered":1044480,"num_keys_recovered":0,"write_bytes_sec":2553627,"read_op_per_sec":0,"write_op_per_sec":623},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-15T21:21:18.693002+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:21:57.038 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:21:57.039 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:21:57.589 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:21:57.636 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:21:57.636 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-15T21:21:56.701373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":365363953,"num_objects":89211,"num_object_clones":0,"num_object_copies":178422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":89211,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":49076,"num_write_kb":196298,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":930,"num_bytes_recovered":3805203,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89284,"ondisk_log_size":89284,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1165104,"kb_used_data":718844,"kb_used_omap":12,"kb_used_meta":446195,"kb_avail":281950416,"statfs":{"total":289910292480,"available":288717225984,"internally_reserved":0,"allocated":736096256,"data_stored":735255091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":456903683},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,6,11,12,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30650368,"num_objects":7483,"num_object_clones":0,"num_object_copies":14966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-255,"num_objects_unfound":0,"num_objects_dirty":7483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7483,"num_write_kb":29932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":255,"num_bytes_recovered":1044480,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002676"},"pg_stats":[{"pgid":"2.f","version":"43'5622","reported_seq":5692,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:55.124743+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:21:55.124743+0000","last_peered":"2024-09-15T21:21:55.124743+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:21:55.124743+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5622,"log_dups_size":0,"ondisk_log_size":5622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23027712,"num_objects":5622,"num_object_clones":0,"num_object_copies":11244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5622,"num_write_kb":22488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1490},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5493","reported_seq":5593,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:53.099932+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:53.099932+0000","last_peered":"2024-09-15T21:21:53.099932+0000","last_clean":"2024-09-15T21:21:53.099932+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:53.099932+0000","last_undegraded":"2024-09-15T21:21:53.099932+0000","last_fullsized":"2024-09-15T21:21:53.099932+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5493,"log_dups_size":0,"ondisk_log_size":5493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22475718,"num_objects":5489,"num_object_clones":0,"num_object_copies":10978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5489,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3774,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5694","reported_seq":5810,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:53.151960+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:53.151960+0000","last_peered":"2024-09-15T21:21:53.151960+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:53.151960+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5694,"log_dups_size":0,"ondisk_log_size":5694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23306240,"num_objects":5690,"num_object_clones":0,"num_object_copies":11380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3794,"num_write_kb":15176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1423}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5456","reported_seq":5521,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:53.157239+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:53.157239+0000","last_peered":"2024-09-15T21:21:53.157239+0000","last_clean":"2024-09-15T21:21:53.157239+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:53.157239+0000","last_undegraded":"2024-09-15T21:21:53.157239+0000","last_fullsized":"2024-09-15T21:21:53.157239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5456,"log_dups_size":0,"ondisk_log_size":5456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22347776,"num_objects":5456,"num_object_clones":0,"num_object_copies":10912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5456,"num_write_kb":21824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5544","reported_seq":5671,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:55.109583+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:21:55.109583+0000","last_peered":"2024-09-15T21:21:55.109583+0000","last_clean":"2024-09-15T21:21:55.109583+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:21:55.109583+0000","last_undegraded":"2024-09-15T21:21:55.109583+0000","last_fullsized":"2024-09-15T21:21:55.109583+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5544,"log_dups_size":0,"ondisk_log_size":5544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22680035,"num_objects":5538,"num_object_clones":0,"num_object_copies":11076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1999,"num_write_kb":7996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5548","reported_seq":5679,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:55.109781+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:21:55.109781+0000","last_peered":"2024-09-15T21:21:55.109781+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:21:55.109781+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5548,"log_dups_size":0,"ondisk_log_size":5548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22704611,"num_objects":5544,"num_object_clones":0,"num_object_copies":11088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2224,"num_write_kb":8896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1468},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5646","reported_seq":5777,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:55.145076+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:21:55.145076+0000","last_peered":"2024-09-15T21:21:55.145076+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:21:55.145076+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5646,"log_dups_size":0,"ondisk_log_size":5646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23102406,"num_objects":5642,"num_object_clones":0,"num_object_copies":11284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1957,"num_write_kb":7828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1481},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5735","reported_seq":5858,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:55.090004+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:21:55.090004+0000","last_peered":"2024-09-15T21:21:55.090004+0000","last_clean":"2024-09-15T21:21:55.090004+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:21:55.090004+0000","last_undegraded":"2024-09-15T21:21:55.090004+0000","last_fullsized":"2024-09-15T21:21:55.090004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5735,"log_dups_size":0,"ondisk_log_size":5735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23482368,"num_objects":5733,"num_object_clones":0,"num_object_copies":11466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2507,"num_write_kb":10028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5487","reported_seq":5608,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:53.117929+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:53.117929+0000","last_peered":"2024-09-15T21:21:53.117929+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:53.117929+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5487,"log_dups_size":0,"ondisk_log_size":5487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22446563,"num_objects":5481,"num_object_clones":0,"num_object_copies":10962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5481,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3419,"num_write_kb":13676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1394}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5543","reported_seq":5668,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:55.124169+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:21:55.124169+0000","last_peered":"2024-09-15T21:21:55.124169+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:21:55.124169+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5543,"log_dups_size":0,"ondisk_log_size":5543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22671360,"num_objects":5535,"num_object_clones":0,"num_object_copies":11070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2216,"num_write_kb":8864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1515},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5500","reported_seq":5613,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:53.182039+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:53.182039+0000","last_peered":"2024-09-15T21:21:53.182039+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:53.182039+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5500,"log_dups_size":0,"ondisk_log_size":5500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22508003,"num_objects":5496,"num_object_clones":0,"num_object_copies":10992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5496,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3359,"num_write_kb":13436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1406}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5542","reported_seq":7515,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:53.163675+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:53.163675+0000","last_peered":"2024-09-15T21:21:53.163675+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:53.163675+0000","last_undegraded":"2024-09-15T21:21:53.163675+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5542,"log_dups_size":0,"ondisk_log_size":5542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22675456,"num_objects":5536,"num_object_clones":0,"num_object_copies":11072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":5536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3235,"num_write_kb":12940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":928,"num_bytes_recovered":3801088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3234},{"shards":"2","objects":2302}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5626","reported_seq":5754,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:55.168656+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:21:55.168656+0000","last_peered":"2024-09-15T21:21:55.168656+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:21:55.168656+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5626,"log_dups_size":0,"ondisk_log_size":5626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23035904,"num_objects":5624,"num_object_clones":0,"num_object_copies":11248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2570,"num_write_kb":10280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1530},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5630","reported_seq":5757,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:55.050114+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:21:55.050114+0000","last_peered":"2024-09-15T21:21:55.050114+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:21:55.050114+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5630,"log_dups_size":0,"ondisk_log_size":5630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23027712,"num_objects":5622,"num_object_clones":0,"num_object_copies":11244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2539,"num_write_kb":10156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1483},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5543","reported_seq":5673,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:54.901269+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:21:54.901269+0000","last_peered":"2024-09-15T21:21:54.901269+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:21:54.901269+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5543,"log_dups_size":0,"ondisk_log_size":5543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22687744,"num_objects":5539,"num_object_clones":0,"num_object_copies":11078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2027,"num_write_kb":8108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1457},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5672","reported_seq":5805,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:55.169437+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:21:55.169437+0000","last_peered":"2024-09-15T21:21:55.169437+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:21:55.169437+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5672,"log_dups_size":0,"ondisk_log_size":5672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23184326,"num_objects":5662,"num_object_clones":0,"num_object_copies":11324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5662,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2376,"num_write_kb":9504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1515},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":365363934,"num_objects":89209,"num_object_clones":0,"num_object_copies":178418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3234,"num_objects_unfound":0,"num_objects_dirty":89209,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":49074,"num_write_kb":196296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":929,"num_bytes_recovered":3805184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":737210368,"data_stored":737055009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":89281,"ondisk_log_size":89281,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462340,"kb_used_data":308468,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":93909500,"statfs":{"total":96636764160,"available":96163328000,"internally_reserved":0,"allocated":315871232,"data_stored":315609218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":157544105},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182428,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510692,"kb_used_data":334676,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93861148,"statfs":{"total":96636764160,"available":96113815552,"internally_reserved":0,"allocated":342708224,"data_stored":342442133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180219563},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":192072,"kb_used_data":75700,"kb_used_omap":4,"kb_used_meta":116347,"kb_avail":94179768,"statfs":{"total":96636764160,"available":96440082432,"internally_reserved":0,"allocated":77516800,"data_stored":77203740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":119140015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343117824,"data_stored":343081694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":78102528,"data_stored":78019429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":315990016,"data_stored":315953886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:21:57.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 86 20 91000 90980 4.13197 2.13281 0.0149749 0.0189003 2024-09-15T21:21:58.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 87 20 91606 91586 4.11168 2.36719 0.0456162 0.0189921 2024-09-15T21:21:59.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 88 20 92192 92172 4.09097 2.28906 0.0397433 0.0190882 2024-09-15T21:22:00.638 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:00.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 89 20 92785 92765 4.07103 2.31641 0.0147901 0.0191817 2024-09-15T21:22:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:01.188 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:01.212 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-15T21:22:00.702254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":376906481,"num_objects":92029,"num_object_clones":0,"num_object_copies":184058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3113,"num_objects_unfound":0,"num_objects_dirty":92029,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":51894,"num_write_kb":207570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1051,"num_bytes_recovered":4300819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92102,"ondisk_log_size":92102,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1191744,"kb_used_data":733004,"kb_used_omap":12,"kb_used_meta":458675,"kb_avail":281923776,"statfs":{"total":289910292480,"available":288689946624,"internally_reserved":0,"allocated":750596096,"data_stored":749754931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":469683203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,9,4,12,12,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30285824,"num_objects":7394,"num_object_clones":0,"num_object_copies":14788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-245,"num_objects_unfound":0,"num_objects_dirty":7394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7394,"num_write_kb":29576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002646"},"pg_stats":[{"pgid":"2.f","version":"43'5837","reported_seq":5907,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.208766+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:00.208766+0000","last_peered":"2024-09-15T21:22:00.208766+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:00.208766+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908352,"num_objects":5837,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5837,"num_write_kb":23348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1705},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5705","reported_seq":5804,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:58.174164+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:58.174164+0000","last_peered":"2024-09-15T21:21:58.174164+0000","last_clean":"2024-09-15T21:21:58.174164+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:58.174164+0000","last_undegraded":"2024-09-15T21:21:58.174164+0000","last_fullsized":"2024-09-15T21:21:58.174164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5705,"log_dups_size":0,"ondisk_log_size":5705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23344070,"num_objects":5701,"num_object_clones":0,"num_object_copies":11402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5701,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3986,"num_write_kb":15944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5879","reported_seq":5999,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:58.185655+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:58.185655+0000","last_peered":"2024-09-15T21:21:58.185655+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:58.185655+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5879,"log_dups_size":0,"ondisk_log_size":5879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24064000,"num_objects":5875,"num_object_clones":0,"num_object_copies":11750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3979,"num_write_kb":15916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5625","reported_seq":5690,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:58.145185+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:58.145185+0000","last_peered":"2024-09-15T21:21:58.145185+0000","last_clean":"2024-09-15T21:21:58.145185+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:58.145185+0000","last_undegraded":"2024-09-15T21:21:58.145185+0000","last_fullsized":"2024-09-15T21:21:58.145185+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5625,"log_dups_size":0,"ondisk_log_size":5625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23040000,"num_objects":5625,"num_object_clones":0,"num_object_copies":11250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5625,"num_write_kb":22500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5707","reported_seq":5834,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.175426+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:00.175426+0000","last_peered":"2024-09-15T21:22:00.175426+0000","last_clean":"2024-09-15T21:22:00.175426+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:00.175426+0000","last_undegraded":"2024-09-15T21:22:00.175426+0000","last_fullsized":"2024-09-15T21:22:00.175426+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5707,"log_dups_size":0,"ondisk_log_size":5707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347683,"num_objects":5701,"num_object_clones":0,"num_object_copies":11402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5724","reported_seq":5855,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.185136+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:00.185136+0000","last_peered":"2024-09-15T21:22:00.185136+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:00.185136+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5724,"log_dups_size":0,"ondisk_log_size":5724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23425507,"num_objects":5720,"num_object_clones":0,"num_object_copies":11440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2400,"num_write_kb":9600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1644},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5839","reported_seq":5970,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.160518+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:00.160518+0000","last_peered":"2024-09-15T21:22:00.160518+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:00.160518+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5839,"log_dups_size":0,"ondisk_log_size":5839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23892934,"num_objects":5835,"num_object_clones":0,"num_object_copies":11670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2150,"num_write_kb":8600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1674},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5928","reported_seq":6051,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.186489+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:00.186489+0000","last_peered":"2024-09-15T21:22:00.186489+0000","last_clean":"2024-09-15T21:22:00.186489+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:00.186489+0000","last_undegraded":"2024-09-15T21:22:00.186489+0000","last_fullsized":"2024-09-15T21:22:00.186489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5928,"log_dups_size":0,"ondisk_log_size":5928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24272896,"num_objects":5926,"num_object_clones":0,"num_object_copies":11852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2700,"num_write_kb":10800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5655","reported_seq":5776,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:58.086605+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:58.086605+0000","last_peered":"2024-09-15T21:21:58.086605+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:58.086605+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5655,"log_dups_size":0,"ondisk_log_size":5655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23134691,"num_objects":5649,"num_object_clones":0,"num_object_copies":11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3587,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1562}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5699","reported_seq":5824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.121985+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:00.121985+0000","last_peered":"2024-09-15T21:22:00.121985+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:00.121985+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5699,"log_dups_size":0,"ondisk_log_size":5699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23310336,"num_objects":5691,"num_object_clones":0,"num_object_copies":11382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1671},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5661","reported_seq":5774,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:58.105112+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:58.105112+0000","last_peered":"2024-09-15T21:21:58.105112+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:58.105112+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5661,"log_dups_size":0,"ondisk_log_size":5661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23167459,"num_objects":5657,"num_object_clones":0,"num_object_copies":11314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5657,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3520,"num_write_kb":14080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1567}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5722","reported_seq":7936,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:58.184014+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:58.184014+0000","last_peered":"2024-09-15T21:21:58.184014+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:58.184014+0000","last_undegraded":"2024-09-15T21:21:58.184014+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5722,"log_dups_size":0,"ondisk_log_size":5722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23412736,"num_objects":5716,"num_object_clones":0,"num_object_copies":11432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3113,"num_objects_unfound":0,"num_objects_dirty":5716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3415,"num_write_kb":13660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1049,"num_bytes_recovered":4296704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3113},{"shards":"2","objects":2603}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5800","reported_seq":5927,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.203302+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:00.203302+0000","last_peered":"2024-09-15T21:22:00.203302+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:00.203302+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5800,"log_dups_size":0,"ondisk_log_size":5800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23748608,"num_objects":5798,"num_object_clones":0,"num_object_copies":11596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2744,"num_write_kb":10976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1704},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5778","reported_seq":5903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.191014+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:00.191014+0000","last_peered":"2024-09-15T21:22:00.191014+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:00.191014+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5778,"log_dups_size":0,"ondisk_log_size":5778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23633920,"num_objects":5770,"num_object_clones":0,"num_object_copies":11540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1631},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5710","reported_seq":5838,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.216695+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:00.216695+0000","last_peered":"2024-09-15T21:22:00.216695+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:00.216695+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5710,"log_dups_size":0,"ondisk_log_size":5710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2194,"num_write_kb":8776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1624},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5830","reported_seq":5963,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.193912+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:00.193912+0000","last_peered":"2024-09-15T21:22:00.193912+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:00.193912+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5830,"log_dups_size":0,"ondisk_log_size":5830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23831494,"num_objects":5820,"num_object_clones":0,"num_object_copies":11640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5820,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1673},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":376906462,"num_objects":92027,"num_object_clones":0,"num_object_copies":184054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3113,"num_objects_unfound":0,"num_objects_dirty":92027,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":51892,"num_write_kb":207568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1050,"num_bytes_recovered":4300800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":751742976,"data_stored":751587617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92099,"ondisk_log_size":92099,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472100,"kb_used_data":314068,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93899740,"statfs":{"total":96636764160,"available":96153333760,"internally_reserved":0,"allocated":321605632,"data_stored":321343618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":161803945},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527572,"kb_used_data":343236,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93844268,"statfs":{"total":96636764160,"available":96096530432,"internally_reserved":0,"allocated":351473664,"data_stored":351207573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188739243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,1,5,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":192072,"kb_used_data":75700,"kb_used_omap":4,"kb_used_meta":116347,"kb_avail":94179768,"statfs":{"total":96636764160,"available":96440082432,"internally_reserved":0,"allocated":77516800,"data_stored":77203740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":119140015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":351858688,"data_stored":351822558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":78102528,"data_stored":78019429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321781760,"data_stored":321745630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:01.213 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:01.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 90 20 93299 93279 4.0481 2.00781 0.0574511 0.0192915 2024-09-15T21:22:01.780 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:01.780 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:01.811 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-15T21:22:00.702254+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":376906481,"num_objects":92029,"num_object_clones":0,"num_object_copies":184058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3113,"num_objects_unfound":0,"num_objects_dirty":92029,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":51894,"num_write_kb":207570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1051,"num_bytes_recovered":4300819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92102,"ondisk_log_size":92102,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1191744,"kb_used_data":733004,"kb_used_omap":12,"kb_used_meta":458675,"kb_avail":281923776,"statfs":{"total":289910292480,"available":288689946624,"internally_reserved":0,"allocated":750596096,"data_stored":749754931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":469683203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,9,4,12,12,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30285824,"num_objects":7394,"num_object_clones":0,"num_object_copies":14788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-245,"num_objects_unfound":0,"num_objects_dirty":7394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7394,"num_write_kb":29576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002646"},"pg_stats":[{"pgid":"2.f","version":"43'5837","reported_seq":5907,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.208766+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:00.208766+0000","last_peered":"2024-09-15T21:22:00.208766+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:00.208766+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908352,"num_objects":5837,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5837,"num_write_kb":23348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1705},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5705","reported_seq":5804,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:58.174164+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:58.174164+0000","last_peered":"2024-09-15T21:21:58.174164+0000","last_clean":"2024-09-15T21:21:58.174164+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:58.174164+0000","last_undegraded":"2024-09-15T21:21:58.174164+0000","last_fullsized":"2024-09-15T21:21:58.174164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5705,"log_dups_size":0,"ondisk_log_size":5705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23344070,"num_objects":5701,"num_object_clones":0,"num_object_copies":11402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5701,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3986,"num_write_kb":15944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5879","reported_seq":5999,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:58.185655+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:58.185655+0000","last_peered":"2024-09-15T21:21:58.185655+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:58.185655+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5879,"log_dups_size":0,"ondisk_log_size":5879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24064000,"num_objects":5875,"num_object_clones":0,"num_object_copies":11750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3979,"num_write_kb":15916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5625","reported_seq":5690,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:58.145185+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:58.145185+0000","last_peered":"2024-09-15T21:21:58.145185+0000","last_clean":"2024-09-15T21:21:58.145185+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:58.145185+0000","last_undegraded":"2024-09-15T21:21:58.145185+0000","last_fullsized":"2024-09-15T21:21:58.145185+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5625,"log_dups_size":0,"ondisk_log_size":5625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23040000,"num_objects":5625,"num_object_clones":0,"num_object_copies":11250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5625,"num_write_kb":22500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5707","reported_seq":5834,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.175426+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:00.175426+0000","last_peered":"2024-09-15T21:22:00.175426+0000","last_clean":"2024-09-15T21:22:00.175426+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:00.175426+0000","last_undegraded":"2024-09-15T21:22:00.175426+0000","last_fullsized":"2024-09-15T21:22:00.175426+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5707,"log_dups_size":0,"ondisk_log_size":5707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347683,"num_objects":5701,"num_object_clones":0,"num_object_copies":11402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5724","reported_seq":5855,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.185136+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:00.185136+0000","last_peered":"2024-09-15T21:22:00.185136+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:00.185136+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5724,"log_dups_size":0,"ondisk_log_size":5724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23425507,"num_objects":5720,"num_object_clones":0,"num_object_copies":11440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2400,"num_write_kb":9600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1644},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5839","reported_seq":5970,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.160518+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:00.160518+0000","last_peered":"2024-09-15T21:22:00.160518+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:00.160518+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5839,"log_dups_size":0,"ondisk_log_size":5839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23892934,"num_objects":5835,"num_object_clones":0,"num_object_copies":11670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2150,"num_write_kb":8600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1674},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5928","reported_seq":6051,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.186489+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:00.186489+0000","last_peered":"2024-09-15T21:22:00.186489+0000","last_clean":"2024-09-15T21:22:00.186489+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:00.186489+0000","last_undegraded":"2024-09-15T21:22:00.186489+0000","last_fullsized":"2024-09-15T21:22:00.186489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5928,"log_dups_size":0,"ondisk_log_size":5928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24272896,"num_objects":5926,"num_object_clones":0,"num_object_copies":11852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2700,"num_write_kb":10800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5655","reported_seq":5776,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:58.086605+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:58.086605+0000","last_peered":"2024-09-15T21:21:58.086605+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:58.086605+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5655,"log_dups_size":0,"ondisk_log_size":5655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23134691,"num_objects":5649,"num_object_clones":0,"num_object_copies":11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3587,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1562}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5699","reported_seq":5824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.121985+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:00.121985+0000","last_peered":"2024-09-15T21:22:00.121985+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:00.121985+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5699,"log_dups_size":0,"ondisk_log_size":5699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23310336,"num_objects":5691,"num_object_clones":0,"num_object_copies":11382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1671},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5661","reported_seq":5774,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:58.105112+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:58.105112+0000","last_peered":"2024-09-15T21:21:58.105112+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:58.105112+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5661,"log_dups_size":0,"ondisk_log_size":5661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23167459,"num_objects":5657,"num_object_clones":0,"num_object_copies":11314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5657,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3520,"num_write_kb":14080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1567}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5722","reported_seq":7936,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:58.184014+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:58.184014+0000","last_peered":"2024-09-15T21:21:58.184014+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:58.184014+0000","last_undegraded":"2024-09-15T21:21:58.184014+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5722,"log_dups_size":0,"ondisk_log_size":5722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23412736,"num_objects":5716,"num_object_clones":0,"num_object_copies":11432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3113,"num_objects_unfound":0,"num_objects_dirty":5716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3415,"num_write_kb":13660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1049,"num_bytes_recovered":4296704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3113},{"shards":"2","objects":2603}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5800","reported_seq":5927,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.203302+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:00.203302+0000","last_peered":"2024-09-15T21:22:00.203302+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:00.203302+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5800,"log_dups_size":0,"ondisk_log_size":5800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23748608,"num_objects":5798,"num_object_clones":0,"num_object_copies":11596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2744,"num_write_kb":10976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1704},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5778","reported_seq":5903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.191014+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:00.191014+0000","last_peered":"2024-09-15T21:22:00.191014+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:00.191014+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5778,"log_dups_size":0,"ondisk_log_size":5778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23633920,"num_objects":5770,"num_object_clones":0,"num_object_copies":11540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1631},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5710","reported_seq":5838,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.216695+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:00.216695+0000","last_peered":"2024-09-15T21:22:00.216695+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:00.216695+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5710,"log_dups_size":0,"ondisk_log_size":5710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2194,"num_write_kb":8776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1624},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5830","reported_seq":5963,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.193912+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:00.193912+0000","last_peered":"2024-09-15T21:22:00.193912+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:00.193912+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5830,"log_dups_size":0,"ondisk_log_size":5830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23831494,"num_objects":5820,"num_object_clones":0,"num_object_copies":11640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5820,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1673},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":376906462,"num_objects":92027,"num_object_clones":0,"num_object_copies":184054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3113,"num_objects_unfound":0,"num_objects_dirty":92027,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":51892,"num_write_kb":207568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1050,"num_bytes_recovered":4300800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":751742976,"data_stored":751587617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92099,"ondisk_log_size":92099,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472100,"kb_used_data":314068,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93899740,"statfs":{"total":96636764160,"available":96153333760,"internally_reserved":0,"allocated":321605632,"data_stored":321343618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":161803945},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527572,"kb_used_data":343236,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93844268,"statfs":{"total":96636764160,"available":96096530432,"internally_reserved":0,"allocated":351473664,"data_stored":351207573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188739243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,1,5,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":192072,"kb_used_data":75700,"kb_used_omap":4,"kb_used_meta":116347,"kb_avail":94179768,"statfs":{"total":96636764160,"available":96440082432,"internally_reserved":0,"allocated":77516800,"data_stored":77203740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":119140015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":351858688,"data_stored":351822558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":78102528,"data_stored":78019429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321781760,"data_stored":321745630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:01.812 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:01.812 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'5837", 'reported_seq': 5907, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:00.208766+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:22:00.208766+0000', 'last_peered': '2024-09-15T21:22:00.208766+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:22:00.208766+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 5837, 'log_dups_size': 0, 'ondisk_log_size': 5837, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23908352, 'num_objects': 5837, 'num_object_clones': 0, 'num_object_copies': 11674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5837, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5837, 'num_write_kb': 23348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1705}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:01.812 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:22:01.812 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'5879", 'reported_seq': 5999, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:58.185655+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:21:58.185655+0000', 'last_peered': '2024-09-15T21:21:58.185655+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:21:58.185655+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 5879, 'log_dups_size': 0, 'ondisk_log_size': 5879, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24064000, 'num_objects': 5875, 'num_object_clones': 0, 'num_object_copies': 11750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5875, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3979, 'num_write_kb': 15916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 1608}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:01.812 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:01.812 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'5724", 'reported_seq': 5855, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:00.185136+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:22:00.185136+0000', 'last_peered': '2024-09-15T21:22:00.185136+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:22:00.185136+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 5724, 'log_dups_size': 0, 'ondisk_log_size': 5724, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23425507, 'num_objects': 5720, 'num_object_clones': 0, 'num_object_copies': 11440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5720, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2400, 'num_write_kb': 9600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1644}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:01.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:01.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'5839", 'reported_seq': 5970, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:00.160518+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:22:00.160518+0000', 'last_peered': '2024-09-15T21:22:00.160518+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:22:00.160518+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 5839, 'log_dups_size': 0, 'ondisk_log_size': 5839, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23892934, 'num_objects': 5835, 'num_object_clones': 0, 'num_object_copies': 11670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5835, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2150, 'num_write_kb': 8600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1674}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:01.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:01.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'5655", 'reported_seq': 5776, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:58.086605+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:21:58.086605+0000', 'last_peered': '2024-09-15T21:21:58.086605+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:21:58.086605+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 5655, 'log_dups_size': 0, 'ondisk_log_size': 5655, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23134691, 'num_objects': 5649, 'num_object_clones': 0, 'num_object_copies': 11298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5649, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3587, 'num_write_kb': 14348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 1562}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:01.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:01.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'5699", 'reported_seq': 5824, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:00.121985+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:22:00.121985+0000', 'last_peered': '2024-09-15T21:22:00.121985+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:22:00.121985+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 5699, 'log_dups_size': 0, 'ondisk_log_size': 5699, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23310336, 'num_objects': 5691, 'num_object_clones': 0, 'num_object_copies': 11382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5691, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2372, 'num_write_kb': 9488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1671}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:01.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:01.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'5661", 'reported_seq': 5774, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:21:58.105112+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:21:58.105112+0000', 'last_peered': '2024-09-15T21:21:58.105112+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:21:58.105112+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 5661, 'log_dups_size': 0, 'ondisk_log_size': 5661, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23167459, 'num_objects': 5657, 'num_object_clones': 0, 'num_object_copies': 11314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5657, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3520, 'num_write_kb': 14080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 1567}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:01.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:01.813 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'5722", 'reported_seq': 7936, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:21:58.184014+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:21:58.184014+0000', 'last_peered': '2024-09-15T21:21:58.184014+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:21:58.184014+0000', 'last_undegraded': '2024-09-15T21:21:58.184014+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 5722, 'log_dups_size': 0, 'ondisk_log_size': 5722, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23412736, 'num_objects': 5716, 'num_object_clones': 0, 'num_object_copies': 11432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3113, 'num_objects_unfound': 0, 'num_objects_dirty': 5716, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3415, 'num_write_kb': 13660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1049, 'num_bytes_recovered': 4296704, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3113}, {'shards': '2', 'objects': 2603}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:01.813 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:01.814 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'5800", 'reported_seq': 5927, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:00.203302+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:22:00.203302+0000', 'last_peered': '2024-09-15T21:22:00.203302+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:22:00.203302+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 5800, 'log_dups_size': 0, 'ondisk_log_size': 5800, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23748608, 'num_objects': 5798, 'num_object_clones': 0, 'num_object_copies': 11596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5798, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2744, 'num_write_kb': 10976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1704}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:01.814 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:01.814 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'5778", 'reported_seq': 5903, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:00.191014+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:22:00.191014+0000', 'last_peered': '2024-09-15T21:22:00.191014+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:22:00.191014+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 5778, 'log_dups_size': 0, 'ondisk_log_size': 5778, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23633920, 'num_objects': 5770, 'num_object_clones': 0, 'num_object_copies': 11540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2687, 'num_write_kb': 10748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1631}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:01.814 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:01.814 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'5710", 'reported_seq': 5838, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:00.216695+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:22:00.216695+0000', 'last_peered': '2024-09-15T21:22:00.216695+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:22:00.216695+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 5710, 'log_dups_size': 0, 'ondisk_log_size': 5710, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23371776, 'num_objects': 5706, 'num_object_clones': 0, 'num_object_copies': 11412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5706, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2194, 'num_write_kb': 8776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1624}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:01.814 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:22:01.814 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'5830", 'reported_seq': 5963, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:00.193912+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:22:00.193912+0000', 'last_peered': '2024-09-15T21:22:00.193912+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:22:00.193912+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 5830, 'log_dups_size': 0, 'ondisk_log_size': 5830, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23831494, 'num_objects': 5820, 'num_object_clones': 0, 'num_object_copies': 11640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5820, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2534, 'num_write_kb': 10136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1673}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:01.814 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:02.477 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:02.477 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/184058 objects degraded (24.617%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":174,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":92029,"data_bytes":376906481,"bytes_used":1220345856,"bytes_avail":288689946624,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":184058,"degraded_ratio":0.24616696910756392,"misplaced_objects":3113,"misplaced_total":184058,"misplaced_ratio":0.016913146942811506,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":83608,"recovering_keys_per_sec":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"write_bytes_sec":2523262,"read_op_per_sec":0,"write_op_per_sec":616},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-15T21:21:18.693002+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:22:02.504 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:02.504 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:02.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 91 20 93818 93798 4.0259 2.02734 0.0182849 0.0193971 2024-09-15T21:22:03.034 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:03.034 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:03.057 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-15T21:22:02.702705+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":376906481,"num_objects":92029,"num_object_clones":0,"num_object_copies":184058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3113,"num_objects_unfound":0,"num_objects_dirty":92029,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":51894,"num_write_kb":207570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1051,"num_bytes_recovered":4300819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92102,"ondisk_log_size":92102,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1206372,"kb_used_data":742068,"kb_used_omap":12,"kb_used_meta":464243,"kb_avail":281909148,"statfs":{"total":289910292480,"available":288674967552,"internally_reserved":0,"allocated":759877632,"data_stored":759036467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":475384835},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,9,7,11,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22945792,"num_objects":5602,"num_object_clones":0,"num_object_copies":11204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-245,"num_objects_unfound":0,"num_objects_dirty":5602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5602,"num_write_kb":22408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":1003520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002662"},"pg_stats":[{"pgid":"2.f","version":"43'5837","reported_seq":5907,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.208766+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:00.208766+0000","last_peered":"2024-09-15T21:22:00.208766+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:00.208766+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908352,"num_objects":5837,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5837,"num_write_kb":23348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1705},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5705","reported_seq":5804,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:58.174164+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:21:58.174164+0000","last_peered":"2024-09-15T21:21:58.174164+0000","last_clean":"2024-09-15T21:21:58.174164+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:21:58.174164+0000","last_undegraded":"2024-09-15T21:21:58.174164+0000","last_fullsized":"2024-09-15T21:21:58.174164+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5705,"log_dups_size":0,"ondisk_log_size":5705,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23344070,"num_objects":5701,"num_object_clones":0,"num_object_copies":11402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5701,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":3986,"num_write_kb":15944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5879","reported_seq":5999,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:58.185655+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:21:58.185655+0000","last_peered":"2024-09-15T21:21:58.185655+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:21:58.185655+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":5879,"log_dups_size":0,"ondisk_log_size":5879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24064000,"num_objects":5875,"num_object_clones":0,"num_object_copies":11750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3979,"num_write_kb":15916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1608}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5625","reported_seq":5690,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:58.145185+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:21:58.145185+0000","last_peered":"2024-09-15T21:21:58.145185+0000","last_clean":"2024-09-15T21:21:58.145185+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:21:58.145185+0000","last_undegraded":"2024-09-15T21:21:58.145185+0000","last_fullsized":"2024-09-15T21:21:58.145185+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5625,"log_dups_size":0,"ondisk_log_size":5625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23040000,"num_objects":5625,"num_object_clones":0,"num_object_copies":11250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5625,"num_write_kb":22500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5707","reported_seq":5834,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.175426+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:00.175426+0000","last_peered":"2024-09-15T21:22:00.175426+0000","last_clean":"2024-09-15T21:22:00.175426+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:00.175426+0000","last_undegraded":"2024-09-15T21:22:00.175426+0000","last_fullsized":"2024-09-15T21:22:00.175426+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5707,"log_dups_size":0,"ondisk_log_size":5707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347683,"num_objects":5701,"num_object_clones":0,"num_object_copies":11402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5724","reported_seq":5855,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.185136+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:00.185136+0000","last_peered":"2024-09-15T21:22:00.185136+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:00.185136+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5724,"log_dups_size":0,"ondisk_log_size":5724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23425507,"num_objects":5720,"num_object_clones":0,"num_object_copies":11440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2400,"num_write_kb":9600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1644},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5839","reported_seq":5970,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.160518+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:00.160518+0000","last_peered":"2024-09-15T21:22:00.160518+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:00.160518+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5839,"log_dups_size":0,"ondisk_log_size":5839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23892934,"num_objects":5835,"num_object_clones":0,"num_object_copies":11670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2150,"num_write_kb":8600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1674},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5928","reported_seq":6051,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.186489+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:00.186489+0000","last_peered":"2024-09-15T21:22:00.186489+0000","last_clean":"2024-09-15T21:22:00.186489+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:00.186489+0000","last_undegraded":"2024-09-15T21:22:00.186489+0000","last_fullsized":"2024-09-15T21:22:00.186489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5928,"log_dups_size":0,"ondisk_log_size":5928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24272896,"num_objects":5926,"num_object_clones":0,"num_object_copies":11852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2700,"num_write_kb":10800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5655","reported_seq":5776,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:58.086605+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:21:58.086605+0000","last_peered":"2024-09-15T21:21:58.086605+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:21:58.086605+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5655,"log_dups_size":0,"ondisk_log_size":5655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23134691,"num_objects":5649,"num_object_clones":0,"num_object_copies":11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3587,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1562}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5699","reported_seq":5824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.121985+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:00.121985+0000","last_peered":"2024-09-15T21:22:00.121985+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:00.121985+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5699,"log_dups_size":0,"ondisk_log_size":5699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23310336,"num_objects":5691,"num_object_clones":0,"num_object_copies":11382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1671},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5661","reported_seq":5774,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:21:58.105112+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:21:58.105112+0000","last_peered":"2024-09-15T21:21:58.105112+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:21:58.105112+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5661,"log_dups_size":0,"ondisk_log_size":5661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23167459,"num_objects":5657,"num_object_clones":0,"num_object_copies":11314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5657,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3520,"num_write_kb":14080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1567}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5722","reported_seq":7936,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:21:58.184014+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:21:58.184014+0000","last_peered":"2024-09-15T21:21:58.184014+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:21:58.184014+0000","last_undegraded":"2024-09-15T21:21:58.184014+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5722,"log_dups_size":0,"ondisk_log_size":5722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23412736,"num_objects":5716,"num_object_clones":0,"num_object_copies":11432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3113,"num_objects_unfound":0,"num_objects_dirty":5716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3415,"num_write_kb":13660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1049,"num_bytes_recovered":4296704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3113},{"shards":"2","objects":2603}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5800","reported_seq":5927,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.203302+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:00.203302+0000","last_peered":"2024-09-15T21:22:00.203302+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:00.203302+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5800,"log_dups_size":0,"ondisk_log_size":5800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23748608,"num_objects":5798,"num_object_clones":0,"num_object_copies":11596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2744,"num_write_kb":10976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1704},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5778","reported_seq":5903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.191014+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:00.191014+0000","last_peered":"2024-09-15T21:22:00.191014+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:00.191014+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5778,"log_dups_size":0,"ondisk_log_size":5778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23633920,"num_objects":5770,"num_object_clones":0,"num_object_copies":11540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1631},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5710","reported_seq":5838,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.216695+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:00.216695+0000","last_peered":"2024-09-15T21:22:00.216695+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:00.216695+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5710,"log_dups_size":0,"ondisk_log_size":5710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2194,"num_write_kb":8776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1624},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5830","reported_seq":5963,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.193912+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:00.193912+0000","last_peered":"2024-09-15T21:22:00.193912+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:00.193912+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5830,"log_dups_size":0,"ondisk_log_size":5830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23831494,"num_objects":5820,"num_object_clones":0,"num_object_copies":11640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5820,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1673},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":376906462,"num_objects":92027,"num_object_clones":0,"num_object_copies":184054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":3113,"num_objects_unfound":0,"num_objects_dirty":92027,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":51892,"num_write_kb":207568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1050,"num_bytes_recovered":4300800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":760950784,"data_stored":760795425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92099,"ondisk_log_size":92099,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472100,"kb_used_data":314068,"kb_used_omap":4,"kb_used_meta":158011,"kb_avail":93899740,"statfs":{"total":96636764160,"available":96153333760,"internally_reserved":0,"allocated":321605632,"data_stored":321343618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":161803945},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527572,"kb_used_data":343236,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93844268,"statfs":{"total":96636764160,"available":96096530432,"internally_reserved":0,"allocated":351473664,"data_stored":351207573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188739243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,1,5,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206700,"kb_used_data":84764,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94165140,"statfs":{"total":96636764160,"available":96425103360,"internally_reserved":0,"allocated":86798336,"data_stored":86485276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":124841647},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":351858688,"data_stored":351822558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":87310336,"data_stored":87227237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321781760,"data_stored":321745630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:03.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 92 20 94365 94345 4.00536 2.13672 0.0206005 0.019495 2024-09-15T21:22:04.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 93 20 94951 94931 3.98691 2.28906 0.0132556 0.0195857 2024-09-15T21:22:05.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 94 20 95545 95525 3.96917 2.32031 0.0643331 0.0196739 2024-09-15T21:22:06.059 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:06.625 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:06.653 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":124,"stamp":"2024-09-15T21:22:04.703117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":381256433,"num_objects":93091,"num_object_clones":0,"num_object_copies":186182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2988,"num_objects_unfound":0,"num_objects_dirty":93091,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":52956,"num_write_kb":211818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1176,"num_bytes_recovered":4812819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93164,"ondisk_log_size":93164,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1220504,"kb_used_data":747880,"kb_used_omap":12,"kb_used_meta":472563,"kb_avail":281895016,"statfs":{"total":289910292480,"available":288660496384,"internally_reserved":0,"allocated":765829120,"data_stored":764987955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":483904515},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,7,5,15,11,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27295744,"num_objects":6664,"num_object_clones":0,"num_object_copies":13328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-370,"num_objects_unfound":0,"num_objects_dirty":6664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6664,"num_write_kb":26656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":370,"num_bytes_recovered":1515520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002624"},"pg_stats":[{"pgid":"2.f","version":"43'5837","reported_seq":5907,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.208766+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:00.208766+0000","last_peered":"2024-09-15T21:22:00.208766+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:00.208766+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5837,"log_dups_size":0,"ondisk_log_size":5837,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908352,"num_objects":5837,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5837,"num_write_kb":23348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1705},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5895","reported_seq":5995,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:03.137081+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:03.137081+0000","last_peered":"2024-09-15T21:22:03.137081+0000","last_clean":"2024-09-15T21:22:03.137081+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:03.137081+0000","last_undegraded":"2024-09-15T21:22:03.137081+0000","last_fullsized":"2024-09-15T21:22:03.137081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5895,"log_dups_size":0,"ondisk_log_size":5895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24122310,"num_objects":5891,"num_object_clones":0,"num_object_copies":11782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5891,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4176,"num_write_kb":16704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6062","reported_seq":6182,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:03.132188+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:03.132188+0000","last_peered":"2024-09-15T21:22:03.132188+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:03.132188+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6062,"log_dups_size":0,"ondisk_log_size":6062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24813568,"num_objects":6058,"num_object_clones":0,"num_object_copies":12116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4162,"num_write_kb":16648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1791}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5798","reported_seq":5863,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:03.152592+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:03.152592+0000","last_peered":"2024-09-15T21:22:03.152592+0000","last_clean":"2024-09-15T21:22:03.152592+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:03.152592+0000","last_undegraded":"2024-09-15T21:22:03.152592+0000","last_fullsized":"2024-09-15T21:22:03.152592+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23748608,"num_objects":5798,"num_object_clones":0,"num_object_copies":11596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5798,"num_write_kb":23192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5707","reported_seq":5834,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.175426+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:00.175426+0000","last_peered":"2024-09-15T21:22:00.175426+0000","last_clean":"2024-09-15T21:22:00.175426+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:00.175426+0000","last_undegraded":"2024-09-15T21:22:00.175426+0000","last_fullsized":"2024-09-15T21:22:00.175426+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5707,"log_dups_size":0,"ondisk_log_size":5707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347683,"num_objects":5701,"num_object_clones":0,"num_object_copies":11402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5724","reported_seq":5855,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.185136+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:00.185136+0000","last_peered":"2024-09-15T21:22:00.185136+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:00.185136+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5724,"log_dups_size":0,"ondisk_log_size":5724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23425507,"num_objects":5720,"num_object_clones":0,"num_object_copies":11440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5720,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2400,"num_write_kb":9600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1644},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5839","reported_seq":5970,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.160518+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:00.160518+0000","last_peered":"2024-09-15T21:22:00.160518+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:00.160518+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":5839,"log_dups_size":0,"ondisk_log_size":5839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23892934,"num_objects":5835,"num_object_clones":0,"num_object_copies":11670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2150,"num_write_kb":8600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1674},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5928","reported_seq":6051,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:00.186489+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:00.186489+0000","last_peered":"2024-09-15T21:22:00.186489+0000","last_clean":"2024-09-15T21:22:00.186489+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:00.186489+0000","last_undegraded":"2024-09-15T21:22:00.186489+0000","last_fullsized":"2024-09-15T21:22:00.186489+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":5928,"log_dups_size":0,"ondisk_log_size":5928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24272896,"num_objects":5926,"num_object_clones":0,"num_object_copies":11852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2700,"num_write_kb":10800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5831","reported_seq":5947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:03.206111+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:03.206111+0000","last_peered":"2024-09-15T21:22:03.206111+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:03.206111+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5831,"log_dups_size":0,"ondisk_log_size":5831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23855587,"num_objects":5825,"num_object_clones":0,"num_object_copies":11650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5825,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3763,"num_write_kb":15052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1738}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5699","reported_seq":5824,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.121985+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:00.121985+0000","last_peered":"2024-09-15T21:22:00.121985+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:00.121985+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5699,"log_dups_size":0,"ondisk_log_size":5699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23310336,"num_objects":5691,"num_object_clones":0,"num_object_copies":11382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2372,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1671},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.551112+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.551112+0000","last_peered":"2024-09-15T21:21:14.551112+0000","last_clean":"2024-09-15T21:21:14.551112+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.551112+0000","last_undegraded":"2024-09-15T21:21:14.551112+0000","last_fullsized":"2024-09-15T21:21:14.551112+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5830","reported_seq":5943,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:03.160266+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:03.160266+0000","last_peered":"2024-09-15T21:22:03.160266+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:03.160266+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5830,"log_dups_size":0,"ondisk_log_size":5830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23859683,"num_objects":5826,"num_object_clones":0,"num_object_copies":11652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3689,"num_write_kb":14756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1736}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5893","reported_seq":8359,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:03.182309+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:03.182309+0000","last_peered":"2024-09-15T21:22:03.182309+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:03.182309+0000","last_undegraded":"2024-09-15T21:22:03.182309+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5893,"log_dups_size":0,"ondisk_log_size":5893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2988,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3586,"num_write_kb":14344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1174,"num_bytes_recovered":4808704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2988},{"shards":"2","objects":2899}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5800","reported_seq":5927,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.203302+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:00.203302+0000","last_peered":"2024-09-15T21:22:00.203302+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:00.203302+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5800,"log_dups_size":0,"ondisk_log_size":5800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23748608,"num_objects":5798,"num_object_clones":0,"num_object_copies":11596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2744,"num_write_kb":10976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1704},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5778","reported_seq":5903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.191014+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:00.191014+0000","last_peered":"2024-09-15T21:22:00.191014+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:00.191014+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5778,"log_dups_size":0,"ondisk_log_size":5778,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23633920,"num_objects":5770,"num_object_clones":0,"num_object_copies":11540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1631},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5710","reported_seq":5838,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.216695+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:00.216695+0000","last_peered":"2024-09-15T21:22:00.216695+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:00.216695+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5710,"log_dups_size":0,"ondisk_log_size":5710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23371776,"num_objects":5706,"num_object_clones":0,"num_object_copies":11412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2194,"num_write_kb":8776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1624},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5830","reported_seq":5963,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:00.193912+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:00.193912+0000","last_peered":"2024-09-15T21:22:00.193912+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:00.193912+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5830,"log_dups_size":0,"ondisk_log_size":5830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23831494,"num_objects":5820,"num_object_clones":0,"num_object_copies":11640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5820,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1673},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":381256414,"num_objects":93089,"num_object_clones":0,"num_object_copies":186178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2988,"num_objects_unfound":0,"num_objects_dirty":93089,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":52954,"num_write_kb":211816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1175,"num_bytes_recovered":4812800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":766722048,"data_stored":766566689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":93161,"ondisk_log_size":93161,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486232,"kb_used_data":319880,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93885608,"statfs":{"total":96636764160,"available":96138862592,"internally_reserved":0,"allocated":327557120,"data_stored":327295106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":170323625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,6,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527572,"kb_used_data":343236,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93844268,"statfs":{"total":96636764160,"available":96096530432,"internally_reserved":0,"allocated":351473664,"data_stored":351207573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188739243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,1,5,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":206700,"kb_used_data":84764,"kb_used_omap":4,"kb_used_meta":121915,"kb_avail":94165140,"statfs":{"total":96636764160,"available":96425103360,"internally_reserved":0,"allocated":86798336,"data_stored":86485276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":124841647},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":351858688,"data_stored":351822558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":87310336,"data_stored":87227237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327553024,"data_stored":327516894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:06.655 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:06.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 95 20 96078 96058 3.94931 2.08203 0.065754 0.0197742 2024-09-15T21:22:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:07.200 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:07.226 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-15T21:22:06.703566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":388215537,"num_objects":94790,"num_object_clones":0,"num_object_copies":189580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2988,"num_objects_unfound":0,"num_objects_dirty":94790,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":54655,"num_write_kb":218614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1176,"num_bytes_recovered":4812819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94863,"ondisk_log_size":94863,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1251388,"kb_used_data":764872,"kb_used_omap":12,"kb_used_meta":486451,"kb_avail":281864132,"statfs":{"total":289910292480,"available":288628871168,"internally_reserved":0,"allocated":783228928,"data_stored":782387763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":498125827},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,15,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30044160,"num_objects":7335,"num_object_clones":0,"num_object_copies":14670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-246,"num_objects_unfound":0,"num_objects_dirty":7335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7335,"num_write_kb":29340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1007616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002600"},"pg_stats":[{"pgid":"2.f","version":"43'6004","reported_seq":6074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.230353+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:05.230353+0000","last_peered":"2024-09-15T21:22:05.230353+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:05.230353+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6004,"log_dups_size":0,"ondisk_log_size":6004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24592384,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6004,"num_write_kb":24016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1872},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5895","reported_seq":5995,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:03.137081+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:03.137081+0000","last_peered":"2024-09-15T21:22:03.137081+0000","last_clean":"2024-09-15T21:22:03.137081+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:03.137081+0000","last_undegraded":"2024-09-15T21:22:03.137081+0000","last_fullsized":"2024-09-15T21:22:03.137081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5895,"log_dups_size":0,"ondisk_log_size":5895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24122310,"num_objects":5891,"num_object_clones":0,"num_object_copies":11782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5891,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4176,"num_write_kb":16704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6062","reported_seq":6182,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:03.132188+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:03.132188+0000","last_peered":"2024-09-15T21:22:03.132188+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:03.132188+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6062,"log_dups_size":0,"ondisk_log_size":6062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24813568,"num_objects":6058,"num_object_clones":0,"num_object_copies":12116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4162,"num_write_kb":16648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1791}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5798","reported_seq":5863,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:03.152592+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:03.152592+0000","last_peered":"2024-09-15T21:22:03.152592+0000","last_clean":"2024-09-15T21:22:03.152592+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:03.152592+0000","last_undegraded":"2024-09-15T21:22:03.152592+0000","last_fullsized":"2024-09-15T21:22:03.152592+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23748608,"num_objects":5798,"num_object_clones":0,"num_object_copies":11596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5798,"num_write_kb":23192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5887","reported_seq":6013,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:05.183816+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:05.183816+0000","last_peered":"2024-09-15T21:22:05.183816+0000","last_clean":"2024-09-15T21:22:05.183816+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:05.183816+0000","last_undegraded":"2024-09-15T21:22:05.183816+0000","last_fullsized":"2024-09-15T21:22:05.183816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5887,"log_dups_size":0,"ondisk_log_size":5887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24084963,"num_objects":5881,"num_object_clones":0,"num_object_copies":11762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":9368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5901","reported_seq":6031,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.195926+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:05.195926+0000","last_peered":"2024-09-15T21:22:05.195926+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:05.195926+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5901,"log_dups_size":0,"ondisk_log_size":5901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24150499,"num_objects":5897,"num_object_clones":0,"num_object_copies":11794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2577,"num_write_kb":10308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1821},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6010","reported_seq":6140,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.227730+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:05.227730+0000","last_peered":"2024-09-15T21:22:05.227730+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:05.227730+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6010,"log_dups_size":0,"ondisk_log_size":6010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24593350,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2321,"num_write_kb":9284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1845},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6097","reported_seq":6220,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:05.215384+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:05.215384+0000","last_peered":"2024-09-15T21:22:05.215384+0000","last_clean":"2024-09-15T21:22:05.215384+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:05.215384+0000","last_undegraded":"2024-09-15T21:22:05.215384+0000","last_fullsized":"2024-09-15T21:22:05.215384+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6097,"log_dups_size":0,"ondisk_log_size":6097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24965120,"num_objects":6095,"num_object_clones":0,"num_object_copies":12190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2869,"num_write_kb":11476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5831","reported_seq":5947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:03.206111+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:03.206111+0000","last_peered":"2024-09-15T21:22:03.206111+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:03.206111+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5831,"log_dups_size":0,"ondisk_log_size":5831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23855587,"num_objects":5825,"num_object_clones":0,"num_object_copies":11650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5825,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3763,"num_write_kb":15052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1738}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5861","reported_seq":5985,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.218805+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:05.218805+0000","last_peered":"2024-09-15T21:22:05.218805+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:05.218805+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23973888,"num_objects":5853,"num_object_clones":0,"num_object_copies":11706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1833},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5830","reported_seq":5943,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:03.160266+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:03.160266+0000","last_peered":"2024-09-15T21:22:03.160266+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:03.160266+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5830,"log_dups_size":0,"ondisk_log_size":5830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23859683,"num_objects":5826,"num_object_clones":0,"num_object_copies":11652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3689,"num_write_kb":14756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1736}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5893","reported_seq":8359,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:03.182309+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:03.182309+0000","last_peered":"2024-09-15T21:22:03.182309+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:03.182309+0000","last_undegraded":"2024-09-15T21:22:03.182309+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5893,"log_dups_size":0,"ondisk_log_size":5893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2988,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3586,"num_write_kb":14344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1174,"num_bytes_recovered":4808704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2988},{"shards":"2","objects":2899}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5970","reported_seq":6098,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.208644+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:05.208644+0000","last_peered":"2024-09-15T21:22:05.208644+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:05.208644+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5970,"log_dups_size":0,"ondisk_log_size":5970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24444928,"num_objects":5968,"num_object_clones":0,"num_object_copies":11936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2914,"num_write_kb":11656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1874},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5938","reported_seq":6064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.227564+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:05.227564+0000","last_peered":"2024-09-15T21:22:05.227564+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:05.227564+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5938,"log_dups_size":0,"ondisk_log_size":5938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24289280,"num_objects":5930,"num_object_clones":0,"num_object_copies":11860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2847,"num_write_kb":11388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1791},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5885","reported_seq":6015,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.179180+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:05.179180+0000","last_peered":"2024-09-15T21:22:05.179180+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:05.179180+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5885,"log_dups_size":0,"ondisk_log_size":5885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24088576,"num_objects":5881,"num_object_clones":0,"num_object_copies":11762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2369,"num_write_kb":9476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1799},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5998","reported_seq":6130,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.227883+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:05.227883+0000","last_peered":"2024-09-15T21:22:05.227883+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:05.227883+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5998,"log_dups_size":0,"ondisk_log_size":5998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24519622,"num_objects":5988,"num_object_clones":0,"num_object_copies":11976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2702,"num_write_kb":10808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1841},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":388215518,"num_objects":94788,"num_object_clones":0,"num_object_copies":189576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2988,"num_objects_unfound":0,"num_objects_dirty":94788,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":54653,"num_write_kb":218612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1175,"num_bytes_recovered":4812800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":784216064,"data_stored":784060705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94860,"ondisk_log_size":94860,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486232,"kb_used_data":319880,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93885608,"statfs":{"total":96636764160,"available":96138862592,"internally_reserved":0,"allocated":327557120,"data_stored":327295106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":170323625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,6,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523160,"kb_used_data":351304,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93848680,"statfs":{"total":96636764160,"available":96101048320,"internally_reserved":0,"allocated":359735296,"data_stored":359469205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":175959723},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241996,"kb_used_data":93688,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94129844,"statfs":{"total":96636764160,"available":96388960256,"internally_reserved":0,"allocated":95936512,"data_stored":95623452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":151842479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":360222720,"data_stored":360186590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96440320,"data_stored":96357221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327553024,"data_stored":327516894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:07.227 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:07.227 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'6004", 'reported_seq': 6074, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:05.230353+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:22:05.230353+0000', 'last_peered': '2024-09-15T21:22:05.230353+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:22:05.230353+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 6004, 'log_dups_size': 0, 'ondisk_log_size': 6004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24592384, 'num_objects': 6004, 'num_object_clones': 0, 'num_object_copies': 12008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6004, 'num_write_kb': 24016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1872}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:07.227 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:22:07.227 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'6062", 'reported_seq': 6182, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:03.132188+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:22:03.132188+0000', 'last_peered': '2024-09-15T21:22:03.132188+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:22:03.132188+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 6062, 'log_dups_size': 0, 'ondisk_log_size': 6062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24813568, 'num_objects': 6058, 'num_object_clones': 0, 'num_object_copies': 12116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6058, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4162, 'num_write_kb': 16648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 1791}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:07.227 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:07.227 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'5901", 'reported_seq': 6031, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:05.195926+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:22:05.195926+0000', 'last_peered': '2024-09-15T21:22:05.195926+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:22:05.195926+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 5901, 'log_dups_size': 0, 'ondisk_log_size': 5901, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24150499, 'num_objects': 5897, 'num_object_clones': 0, 'num_object_copies': 11794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5897, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2577, 'num_write_kb': 10308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1821}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:07.228 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:07.228 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'6010", 'reported_seq': 6140, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:05.227730+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:22:05.227730+0000', 'last_peered': '2024-09-15T21:22:05.227730+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:22:05.227730+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 6010, 'log_dups_size': 0, 'ondisk_log_size': 6010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24593350, 'num_objects': 6006, 'num_object_clones': 0, 'num_object_copies': 12012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6006, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2321, 'num_write_kb': 9284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1845}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:07.228 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:07.228 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'5831", 'reported_seq': 5947, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:03.206111+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:22:03.206111+0000', 'last_peered': '2024-09-15T21:22:03.206111+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:22:03.206111+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 5831, 'log_dups_size': 0, 'ondisk_log_size': 5831, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23855587, 'num_objects': 5825, 'num_object_clones': 0, 'num_object_copies': 11650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5825, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3763, 'num_write_kb': 15052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 1738}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:07.228 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:07.228 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'5861", 'reported_seq': 5985, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:05.218805+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:22:05.218805+0000', 'last_peered': '2024-09-15T21:22:05.218805+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:22:05.218805+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 5861, 'log_dups_size': 0, 'ondisk_log_size': 5861, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23973888, 'num_objects': 5853, 'num_object_clones': 0, 'num_object_copies': 11706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2534, 'num_write_kb': 10136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1833}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:07.228 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:07.228 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'5830", 'reported_seq': 5943, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:03.160266+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:22:03.160266+0000', 'last_peered': '2024-09-15T21:22:03.160266+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:22:03.160266+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 5830, 'log_dups_size': 0, 'ondisk_log_size': 5830, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23859683, 'num_objects': 5826, 'num_object_clones': 0, 'num_object_copies': 11652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5826, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3689, 'num_write_kb': 14756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 1736}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:07.228 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:07.228 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'5893", 'reported_seq': 8359, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:22:03.182309+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:22:03.182309+0000', 'last_peered': '2024-09-15T21:22:03.182309+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:22:03.182309+0000', 'last_undegraded': '2024-09-15T21:22:03.182309+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 5893, 'log_dups_size': 0, 'ondisk_log_size': 5893, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24113152, 'num_objects': 5887, 'num_object_clones': 0, 'num_object_copies': 11774, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2988, 'num_objects_unfound': 0, 'num_objects_dirty': 5887, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3586, 'num_write_kb': 14344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1174, 'num_bytes_recovered': 4808704, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2988}, {'shards': '2', 'objects': 2899}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:07.229 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:07.229 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'5970", 'reported_seq': 6098, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:05.208644+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:22:05.208644+0000', 'last_peered': '2024-09-15T21:22:05.208644+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:22:05.208644+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 5970, 'log_dups_size': 0, 'ondisk_log_size': 5970, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24444928, 'num_objects': 5968, 'num_object_clones': 0, 'num_object_copies': 11936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5968, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2914, 'num_write_kb': 11656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1874}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:07.229 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:07.229 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'5938", 'reported_seq': 6064, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:05.227564+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:22:05.227564+0000', 'last_peered': '2024-09-15T21:22:05.227564+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:22:05.227564+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 5938, 'log_dups_size': 0, 'ondisk_log_size': 5938, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24289280, 'num_objects': 5930, 'num_object_clones': 0, 'num_object_copies': 11860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5930, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2847, 'num_write_kb': 11388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1791}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:07.229 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:07.229 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'5885", 'reported_seq': 6015, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:05.179180+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:22:05.179180+0000', 'last_peered': '2024-09-15T21:22:05.179180+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:22:05.179180+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 5885, 'log_dups_size': 0, 'ondisk_log_size': 5885, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24088576, 'num_objects': 5881, 'num_object_clones': 0, 'num_object_copies': 11762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5881, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2369, 'num_write_kb': 9476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1799}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:07.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:22:07.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'5998", 'reported_seq': 6130, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:05.227883+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:22:05.227883+0000', 'last_peered': '2024-09-15T21:22:05.227883+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:22:05.227883+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 5998, 'log_dups_size': 0, 'ondisk_log_size': 5998, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24519622, 'num_objects': 5988, 'num_object_clones': 0, 'num_object_copies': 11976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5988, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2702, 'num_write_kb': 10808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1841}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:07.230 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:07.740 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 96 20 96584 96564 3.92876 1.97656 0.0522277 0.0198779 2024-09-15T21:22:07.907 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:07.907 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/189580 objects degraded (23.900%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":179,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":94790,"data_bytes":388215537,"bytes_used":1281421312,"bytes_avail":288628871168,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":189580,"degraded_ratio":0.23899672961282836,"misplaced_objects":2988,"misplaced_total":189580,"misplaced_ratio":0.015761156240109715,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":83949,"recovering_keys_per_sec":0,"num_objects_recovered":246,"num_bytes_recovered":1007616,"num_keys_recovered":0,"write_bytes_sec":2503137,"read_op_per_sec":0,"write_op_per_sec":611},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-15T21:21:18.693002+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-15T21:22:07.933 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:07.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:08.483 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:08.484 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:08.509 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-15T21:22:06.703566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":388215537,"num_objects":94790,"num_object_clones":0,"num_object_copies":189580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2988,"num_objects_unfound":0,"num_objects_dirty":94790,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":54655,"num_write_kb":218614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1176,"num_bytes_recovered":4812819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94863,"ondisk_log_size":94863,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1251388,"kb_used_data":764872,"kb_used_omap":12,"kb_used_meta":486451,"kb_avail":281864132,"statfs":{"total":289910292480,"available":288628871168,"internally_reserved":0,"allocated":783228928,"data_stored":782387763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":498125827},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,15,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30044160,"num_objects":7335,"num_object_clones":0,"num_object_copies":14670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-246,"num_objects_unfound":0,"num_objects_dirty":7335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7335,"num_write_kb":29340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":1007616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002600"},"pg_stats":[{"pgid":"2.f","version":"43'6004","reported_seq":6074,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.230353+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:05.230353+0000","last_peered":"2024-09-15T21:22:05.230353+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:05.230353+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6004,"log_dups_size":0,"ondisk_log_size":6004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24592384,"num_objects":6004,"num_object_clones":0,"num_object_copies":12008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6004,"num_write_kb":24016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1872},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5895","reported_seq":5995,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:03.137081+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:03.137081+0000","last_peered":"2024-09-15T21:22:03.137081+0000","last_clean":"2024-09-15T21:22:03.137081+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:03.137081+0000","last_undegraded":"2024-09-15T21:22:03.137081+0000","last_fullsized":"2024-09-15T21:22:03.137081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":5895,"log_dups_size":0,"ondisk_log_size":5895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24122310,"num_objects":5891,"num_object_clones":0,"num_object_copies":11782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5891,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4176,"num_write_kb":16704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6062","reported_seq":6182,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:03.132188+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:03.132188+0000","last_peered":"2024-09-15T21:22:03.132188+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:03.132188+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6062,"log_dups_size":0,"ondisk_log_size":6062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24813568,"num_objects":6058,"num_object_clones":0,"num_object_copies":12116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4162,"num_write_kb":16648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1791}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5798","reported_seq":5863,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:03.152592+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:03.152592+0000","last_peered":"2024-09-15T21:22:03.152592+0000","last_clean":"2024-09-15T21:22:03.152592+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:03.152592+0000","last_undegraded":"2024-09-15T21:22:03.152592+0000","last_fullsized":"2024-09-15T21:22:03.152592+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5798,"log_dups_size":0,"ondisk_log_size":5798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23748608,"num_objects":5798,"num_object_clones":0,"num_object_copies":11596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5798,"num_write_kb":23192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5887","reported_seq":6013,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:05.183816+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:05.183816+0000","last_peered":"2024-09-15T21:22:05.183816+0000","last_clean":"2024-09-15T21:22:05.183816+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:05.183816+0000","last_undegraded":"2024-09-15T21:22:05.183816+0000","last_fullsized":"2024-09-15T21:22:05.183816+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":5887,"log_dups_size":0,"ondisk_log_size":5887,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24084963,"num_objects":5881,"num_object_clones":0,"num_object_copies":11762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2342,"num_write_kb":9368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5901","reported_seq":6031,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.195926+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:05.195926+0000","last_peered":"2024-09-15T21:22:05.195926+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:05.195926+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":5901,"log_dups_size":0,"ondisk_log_size":5901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24150499,"num_objects":5897,"num_object_clones":0,"num_object_copies":11794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2577,"num_write_kb":10308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1821},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6010","reported_seq":6140,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.227730+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:05.227730+0000","last_peered":"2024-09-15T21:22:05.227730+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:05.227730+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6010,"log_dups_size":0,"ondisk_log_size":6010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24593350,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2321,"num_write_kb":9284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1845},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6097","reported_seq":6220,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:05.215384+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:05.215384+0000","last_peered":"2024-09-15T21:22:05.215384+0000","last_clean":"2024-09-15T21:22:05.215384+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:05.215384+0000","last_undegraded":"2024-09-15T21:22:05.215384+0000","last_fullsized":"2024-09-15T21:22:05.215384+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6097,"log_dups_size":0,"ondisk_log_size":6097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24965120,"num_objects":6095,"num_object_clones":0,"num_object_copies":12190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2869,"num_write_kb":11476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5831","reported_seq":5947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:03.206111+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:03.206111+0000","last_peered":"2024-09-15T21:22:03.206111+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:03.206111+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":5831,"log_dups_size":0,"ondisk_log_size":5831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23855587,"num_objects":5825,"num_object_clones":0,"num_object_copies":11650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5825,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3763,"num_write_kb":15052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1738}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5861","reported_seq":5985,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.218805+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:05.218805+0000","last_peered":"2024-09-15T21:22:05.218805+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:05.218805+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":5861,"log_dups_size":0,"ondisk_log_size":5861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23973888,"num_objects":5853,"num_object_clones":0,"num_object_copies":11706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2534,"num_write_kb":10136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1833},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5830","reported_seq":5943,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:03.160266+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:03.160266+0000","last_peered":"2024-09-15T21:22:03.160266+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:03.160266+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":5830,"log_dups_size":0,"ondisk_log_size":5830,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23859683,"num_objects":5826,"num_object_clones":0,"num_object_copies":11652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3689,"num_write_kb":14756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1736}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5893","reported_seq":8359,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:03.182309+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:03.182309+0000","last_peered":"2024-09-15T21:22:03.182309+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:03.182309+0000","last_undegraded":"2024-09-15T21:22:03.182309+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":5893,"log_dups_size":0,"ondisk_log_size":5893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24113152,"num_objects":5887,"num_object_clones":0,"num_object_copies":11774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2988,"num_objects_unfound":0,"num_objects_dirty":5887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3586,"num_write_kb":14344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1174,"num_bytes_recovered":4808704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2988},{"shards":"2","objects":2899}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5970","reported_seq":6098,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.208644+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:05.208644+0000","last_peered":"2024-09-15T21:22:05.208644+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:05.208644+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":5970,"log_dups_size":0,"ondisk_log_size":5970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24444928,"num_objects":5968,"num_object_clones":0,"num_object_copies":11936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2914,"num_write_kb":11656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1874},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5938","reported_seq":6064,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.227564+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:05.227564+0000","last_peered":"2024-09-15T21:22:05.227564+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:05.227564+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":5938,"log_dups_size":0,"ondisk_log_size":5938,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24289280,"num_objects":5930,"num_object_clones":0,"num_object_copies":11860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2847,"num_write_kb":11388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1791},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5885","reported_seq":6015,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.179180+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:05.179180+0000","last_peered":"2024-09-15T21:22:05.179180+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:05.179180+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":5885,"log_dups_size":0,"ondisk_log_size":5885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24088576,"num_objects":5881,"num_object_clones":0,"num_object_copies":11762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2369,"num_write_kb":9476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1799},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5998","reported_seq":6130,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:05.227883+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:05.227883+0000","last_peered":"2024-09-15T21:22:05.227883+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:05.227883+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":5998,"log_dups_size":0,"ondisk_log_size":5998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24519622,"num_objects":5988,"num_object_clones":0,"num_object_copies":11976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2702,"num_write_kb":10808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1841},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":388215518,"num_objects":94788,"num_object_clones":0,"num_object_copies":189576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2988,"num_objects_unfound":0,"num_objects_dirty":94788,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":54653,"num_write_kb":218612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1175,"num_bytes_recovered":4812800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":784216064,"data_stored":784060705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94860,"ondisk_log_size":94860,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486232,"kb_used_data":319880,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":93885608,"statfs":{"total":96636764160,"available":96138862592,"internally_reserved":0,"allocated":327557120,"data_stored":327295106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":170323625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,6,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523160,"kb_used_data":351304,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93848680,"statfs":{"total":96636764160,"available":96101048320,"internally_reserved":0,"allocated":359735296,"data_stored":359469205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":175959723},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241996,"kb_used_data":93688,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94129844,"statfs":{"total":96636764160,"available":96388960256,"internally_reserved":0,"allocated":95936512,"data_stored":95623452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":151842479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":360222720,"data_stored":360186590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96440320,"data_stored":96357221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":327553024,"data_stored":327516894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:08.741 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 97 20 97111 97091 3.90948 2.05859 0.0408814 0.0199741 2024-09-15T21:22:09.743 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 98 20 97679 97659 3.89216 2.21875 0.0350657 0.0200657 2024-09-15T21:22:10.743 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 99 20 98244 98224 3.87513 2.20703 0.0192744 0.0201531 2024-09-15T21:22:11.511 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:11.743 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:2024-09-15T21:22:11.740754+0000 min lat: 0.00235499 max lat: 2.60418 avg lat: 0.0202433 2024-09-15T21:22:11.743 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:22:11.743 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 100 20 98797 98777 3.85798 2.16016 0.0321079 0.0202433 2024-09-15T21:22:12.065 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:12.066 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:12.091 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-15T21:22:10.704623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":399659761,"num_objects":97584,"num_object_clones":0,"num_object_copies":195168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2864,"num_objects_unfound":0,"num_objects_dirty":97584,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":57449,"num_write_kb":229790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1300,"num_bytes_recovered":5320723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97657,"ondisk_log_size":97657,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1291524,"kb_used_data":778640,"kb_used_omap":12,"kb_used_meta":512819,"kb_avail":281823996,"statfs":{"total":289910292480,"available":288587771904,"internally_reserved":0,"allocated":797327360,"data_stored":796486195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":525126659},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,2,15,2,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29892608,"num_objects":7298,"num_object_clones":0,"num_object_copies":14596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":7298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7298,"num_write_kb":29192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002799"},"pg_stats":[{"pgid":"2.f","version":"43'6216","reported_seq":6284,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.288215+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:10.288215+0000","last_peered":"2024-09-15T21:22:10.288215+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:10.288215+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6216,"log_dups_size":0,"ondisk_log_size":6216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25460736,"num_objects":6216,"num_object_clones":0,"num_object_copies":12432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6216,"num_write_kb":24864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2084},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6073","reported_seq":6173,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:08.184557+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:08.184557+0000","last_peered":"2024-09-15T21:22:08.184557+0000","last_clean":"2024-09-15T21:22:08.184557+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:08.184557+0000","last_undegraded":"2024-09-15T21:22:08.184557+0000","last_fullsized":"2024-09-15T21:22:08.184557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6073,"log_dups_size":0,"ondisk_log_size":6073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24851398,"num_objects":6069,"num_object_clones":0,"num_object_copies":12138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6069,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4354,"num_write_kb":17416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6220","reported_seq":6340,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:08.200443+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:08.200443+0000","last_peered":"2024-09-15T21:22:08.200443+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:08.200443+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6220,"log_dups_size":0,"ondisk_log_size":6220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25460736,"num_objects":6216,"num_object_clones":0,"num_object_copies":12432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4320,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1949}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5952","reported_seq":6017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:08.182454+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:08.182454+0000","last_peered":"2024-09-15T21:22:08.182454+0000","last_clean":"2024-09-15T21:22:08.182454+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:08.182454+0000","last_undegraded":"2024-09-15T21:22:08.182454+0000","last_fullsized":"2024-09-15T21:22:08.182454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5952,"log_dups_size":0,"ondisk_log_size":5952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24379392,"num_objects":5952,"num_object_clones":0,"num_object_copies":11904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5952,"num_write_kb":23808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6073","reported_seq":6199,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:10.266573+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:10.266573+0000","last_peered":"2024-09-15T21:22:10.266573+0000","last_clean":"2024-09-15T21:22:10.266573+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:10.266573+0000","last_undegraded":"2024-09-15T21:22:10.266573+0000","last_fullsized":"2024-09-15T21:22:10.266573+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6073,"log_dups_size":0,"ondisk_log_size":6073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24846819,"num_objects":6067,"num_object_clones":0,"num_object_copies":12134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2528,"num_write_kb":10112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6044","reported_seq":6175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.262513+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:10.262513+0000","last_peered":"2024-09-15T21:22:10.262513+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:10.262513+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6044,"log_dups_size":0,"ondisk_log_size":6044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24736227,"num_objects":6040,"num_object_clones":0,"num_object_copies":12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2720,"num_write_kb":10880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1964},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6180","reported_seq":6310,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.175287+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:10.175287+0000","last_peered":"2024-09-15T21:22:10.175287+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:10.175287+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6180,"log_dups_size":0,"ondisk_log_size":6180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25289670,"num_objects":6176,"num_object_clones":0,"num_object_copies":12352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2491,"num_write_kb":9964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2015},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6251","reported_seq":6374,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:10.278005+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:10.278005+0000","last_peered":"2024-09-15T21:22:10.278005+0000","last_clean":"2024-09-15T21:22:10.278005+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:10.278005+0000","last_undegraded":"2024-09-15T21:22:10.278005+0000","last_fullsized":"2024-09-15T21:22:10.278005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6251,"log_dups_size":0,"ondisk_log_size":6251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25595904,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3023,"num_write_kb":12092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6012","reported_seq":6133,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:08.201558+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:08.201558+0000","last_peered":"2024-09-15T21:22:08.201558+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:08.201558+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6012,"log_dups_size":0,"ondisk_log_size":6012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24596963,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3944,"num_write_kb":15776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1919}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6028","reported_seq":6153,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.244005+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:10.244005+0000","last_peered":"2024-09-15T21:22:10.244005+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:10.244005+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6028,"log_dups_size":0,"ondisk_log_size":6028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24657920,"num_objects":6020,"num_object_clones":0,"num_object_copies":12040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2701,"num_write_kb":10804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2000},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6015","reported_seq":6128,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:08.194684+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:08.194684+0000","last_peered":"2024-09-15T21:22:08.194684+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:08.194684+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6015,"log_dups_size":0,"ondisk_log_size":6015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24617443,"num_objects":6011,"num_object_clones":0,"num_object_copies":12022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6011,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3874,"num_write_kb":15496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1921}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6104","reported_seq":8818,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:08.196014+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:08.196014+0000","last_peered":"2024-09-15T21:22:08.196014+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:08.196014+0000","last_undegraded":"2024-09-15T21:22:08.196014+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6104,"log_dups_size":0,"ondisk_log_size":6104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24977408,"num_objects":6098,"num_object_clones":0,"num_object_copies":12196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2864,"num_objects_unfound":0,"num_objects_dirty":6098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3797,"num_write_kb":15188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1298,"num_bytes_recovered":5316608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2864},{"shards":"2","objects":3234}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6135","reported_seq":6263,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.212874+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:10.212874+0000","last_peered":"2024-09-15T21:22:10.212874+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:10.212874+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6135,"log_dups_size":0,"ondisk_log_size":6135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25120768,"num_objects":6133,"num_object_clones":0,"num_object_copies":12266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3079,"num_write_kb":12316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2039},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6140","reported_seq":6266,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.272892+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:10.272892+0000","last_peered":"2024-09-15T21:22:10.272892+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:10.272892+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6140,"log_dups_size":0,"ondisk_log_size":6140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25116672,"num_objects":6132,"num_object_clones":0,"num_object_copies":12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3049,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1993},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6063","reported_seq":6191,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.264857+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:10.264857+0000","last_peered":"2024-09-15T21:22:10.264857+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:10.264857+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6063,"log_dups_size":0,"ondisk_log_size":6063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24817664,"num_objects":6059,"num_object_clones":0,"num_object_copies":12118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2547,"num_write_kb":10188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1977},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6148","reported_seq":6281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.156351+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:10.156351+0000","last_peered":"2024-09-15T21:22:10.156351+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:10.156351+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6148,"log_dups_size":0,"ondisk_log_size":6148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25134022,"num_objects":6138,"num_object_clones":0,"num_object_copies":12276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2852,"num_write_kb":11408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1991},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":399659742,"num_objects":97582,"num_object_clones":0,"num_object_copies":195164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2864,"num_objects_unfound":0,"num_objects_dirty":97582,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":57447,"num_write_kb":229788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1299,"num_bytes_recovered":5320704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798351360,"data_stored":798196001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97654,"ondisk_log_size":97654,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514008,"kb_used_data":325448,"kb_used_omap":4,"kb_used_meta":188539,"kb_avail":93857832,"statfs":{"total":96636764160,"available":96110419968,"internally_reserved":0,"allocated":333258752,"data_stored":332996738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":193064617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535520,"kb_used_data":359504,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93836320,"statfs":{"total":96636764160,"available":96088391680,"internally_reserved":0,"allocated":368132096,"data_stored":367866005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180219563},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,7,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241996,"kb_used_data":93688,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94129844,"statfs":{"total":96636764160,"available":96388960256,"internally_reserved":0,"allocated":95936512,"data_stored":95623452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":151842479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":368578560,"data_stored":368542430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96440320,"data_stored":96357221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333332480,"data_stored":333296350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:12.093 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:12.661 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:12.683 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-15T21:22:10.704623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":399659761,"num_objects":97584,"num_object_clones":0,"num_object_copies":195168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2864,"num_objects_unfound":0,"num_objects_dirty":97584,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":57449,"num_write_kb":229790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1300,"num_bytes_recovered":5320723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97657,"ondisk_log_size":97657,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1291524,"kb_used_data":778640,"kb_used_omap":12,"kb_used_meta":512819,"kb_avail":281823996,"statfs":{"total":289910292480,"available":288587771904,"internally_reserved":0,"allocated":797327360,"data_stored":796486195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":525126659},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,2,15,2,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29892608,"num_objects":7298,"num_object_clones":0,"num_object_copies":14596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":7298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7298,"num_write_kb":29192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002799"},"pg_stats":[{"pgid":"2.f","version":"43'6216","reported_seq":6284,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.288215+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:10.288215+0000","last_peered":"2024-09-15T21:22:10.288215+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:10.288215+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6216,"log_dups_size":0,"ondisk_log_size":6216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25460736,"num_objects":6216,"num_object_clones":0,"num_object_copies":12432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6216,"num_write_kb":24864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2084},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6073","reported_seq":6173,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:08.184557+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:08.184557+0000","last_peered":"2024-09-15T21:22:08.184557+0000","last_clean":"2024-09-15T21:22:08.184557+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:08.184557+0000","last_undegraded":"2024-09-15T21:22:08.184557+0000","last_fullsized":"2024-09-15T21:22:08.184557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6073,"log_dups_size":0,"ondisk_log_size":6073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24851398,"num_objects":6069,"num_object_clones":0,"num_object_copies":12138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6069,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4354,"num_write_kb":17416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6220","reported_seq":6340,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:08.200443+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:08.200443+0000","last_peered":"2024-09-15T21:22:08.200443+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:08.200443+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6220,"log_dups_size":0,"ondisk_log_size":6220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25460736,"num_objects":6216,"num_object_clones":0,"num_object_copies":12432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4320,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1949}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5952","reported_seq":6017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:08.182454+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:08.182454+0000","last_peered":"2024-09-15T21:22:08.182454+0000","last_clean":"2024-09-15T21:22:08.182454+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:08.182454+0000","last_undegraded":"2024-09-15T21:22:08.182454+0000","last_fullsized":"2024-09-15T21:22:08.182454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5952,"log_dups_size":0,"ondisk_log_size":5952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24379392,"num_objects":5952,"num_object_clones":0,"num_object_copies":11904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5952,"num_write_kb":23808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6073","reported_seq":6199,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:10.266573+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:10.266573+0000","last_peered":"2024-09-15T21:22:10.266573+0000","last_clean":"2024-09-15T21:22:10.266573+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:10.266573+0000","last_undegraded":"2024-09-15T21:22:10.266573+0000","last_fullsized":"2024-09-15T21:22:10.266573+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6073,"log_dups_size":0,"ondisk_log_size":6073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24846819,"num_objects":6067,"num_object_clones":0,"num_object_copies":12134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2528,"num_write_kb":10112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6044","reported_seq":6175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.262513+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:10.262513+0000","last_peered":"2024-09-15T21:22:10.262513+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:10.262513+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6044,"log_dups_size":0,"ondisk_log_size":6044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24736227,"num_objects":6040,"num_object_clones":0,"num_object_copies":12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2720,"num_write_kb":10880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1964},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6180","reported_seq":6310,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.175287+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:10.175287+0000","last_peered":"2024-09-15T21:22:10.175287+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:10.175287+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6180,"log_dups_size":0,"ondisk_log_size":6180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25289670,"num_objects":6176,"num_object_clones":0,"num_object_copies":12352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2491,"num_write_kb":9964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2015},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6251","reported_seq":6374,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:10.278005+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:10.278005+0000","last_peered":"2024-09-15T21:22:10.278005+0000","last_clean":"2024-09-15T21:22:10.278005+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:10.278005+0000","last_undegraded":"2024-09-15T21:22:10.278005+0000","last_fullsized":"2024-09-15T21:22:10.278005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6251,"log_dups_size":0,"ondisk_log_size":6251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25595904,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3023,"num_write_kb":12092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6012","reported_seq":6133,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:08.201558+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:08.201558+0000","last_peered":"2024-09-15T21:22:08.201558+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:08.201558+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6012,"log_dups_size":0,"ondisk_log_size":6012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24596963,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3944,"num_write_kb":15776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1919}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6028","reported_seq":6153,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.244005+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:10.244005+0000","last_peered":"2024-09-15T21:22:10.244005+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:10.244005+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6028,"log_dups_size":0,"ondisk_log_size":6028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24657920,"num_objects":6020,"num_object_clones":0,"num_object_copies":12040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2701,"num_write_kb":10804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2000},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6015","reported_seq":6128,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:08.194684+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:08.194684+0000","last_peered":"2024-09-15T21:22:08.194684+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:08.194684+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6015,"log_dups_size":0,"ondisk_log_size":6015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24617443,"num_objects":6011,"num_object_clones":0,"num_object_copies":12022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6011,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3874,"num_write_kb":15496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1921}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6104","reported_seq":8818,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:08.196014+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:08.196014+0000","last_peered":"2024-09-15T21:22:08.196014+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:08.196014+0000","last_undegraded":"2024-09-15T21:22:08.196014+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6104,"log_dups_size":0,"ondisk_log_size":6104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24977408,"num_objects":6098,"num_object_clones":0,"num_object_copies":12196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2864,"num_objects_unfound":0,"num_objects_dirty":6098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3797,"num_write_kb":15188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1298,"num_bytes_recovered":5316608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2864},{"shards":"2","objects":3234}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6135","reported_seq":6263,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.212874+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:10.212874+0000","last_peered":"2024-09-15T21:22:10.212874+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:10.212874+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6135,"log_dups_size":0,"ondisk_log_size":6135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25120768,"num_objects":6133,"num_object_clones":0,"num_object_copies":12266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3079,"num_write_kb":12316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2039},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6140","reported_seq":6266,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.272892+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:10.272892+0000","last_peered":"2024-09-15T21:22:10.272892+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:10.272892+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6140,"log_dups_size":0,"ondisk_log_size":6140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25116672,"num_objects":6132,"num_object_clones":0,"num_object_copies":12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3049,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1993},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6063","reported_seq":6191,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.264857+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:10.264857+0000","last_peered":"2024-09-15T21:22:10.264857+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:10.264857+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6063,"log_dups_size":0,"ondisk_log_size":6063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24817664,"num_objects":6059,"num_object_clones":0,"num_object_copies":12118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2547,"num_write_kb":10188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1977},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6148","reported_seq":6281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.156351+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:10.156351+0000","last_peered":"2024-09-15T21:22:10.156351+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:10.156351+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6148,"log_dups_size":0,"ondisk_log_size":6148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25134022,"num_objects":6138,"num_object_clones":0,"num_object_copies":12276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2852,"num_write_kb":11408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1991},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":399659742,"num_objects":97582,"num_object_clones":0,"num_object_copies":195164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2864,"num_objects_unfound":0,"num_objects_dirty":97582,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":57447,"num_write_kb":229788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1299,"num_bytes_recovered":5320704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":798351360,"data_stored":798196001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97654,"ondisk_log_size":97654,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514008,"kb_used_data":325448,"kb_used_omap":4,"kb_used_meta":188539,"kb_avail":93857832,"statfs":{"total":96636764160,"available":96110419968,"internally_reserved":0,"allocated":333258752,"data_stored":332996738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":193064617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535520,"kb_used_data":359504,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93836320,"statfs":{"total":96636764160,"available":96088391680,"internally_reserved":0,"allocated":368132096,"data_stored":367866005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180219563},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,7,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":241996,"kb_used_data":93688,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94129844,"statfs":{"total":96636764160,"available":96388960256,"internally_reserved":0,"allocated":95936512,"data_stored":95623452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":151842479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":368578560,"data_stored":368542430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96440320,"data_stored":96357221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333332480,"data_stored":333296350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:12.684 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:12.684 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'6216", 'reported_seq': 6284, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:10.288215+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:22:10.288215+0000', 'last_peered': '2024-09-15T21:22:10.288215+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:22:10.288215+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 6216, 'log_dups_size': 0, 'ondisk_log_size': 6216, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25460736, 'num_objects': 6216, 'num_object_clones': 0, 'num_object_copies': 12432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6216, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6216, 'num_write_kb': 24864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2084}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:12.684 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:22:12.684 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'6220", 'reported_seq': 6340, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:08.200443+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:22:08.200443+0000', 'last_peered': '2024-09-15T21:22:08.200443+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:22:08.200443+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 6220, 'log_dups_size': 0, 'ondisk_log_size': 6220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25460736, 'num_objects': 6216, 'num_object_clones': 0, 'num_object_copies': 12432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6216, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4320, 'num_write_kb': 17280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 1949}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:12.685 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:12.685 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'6044", 'reported_seq': 6175, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:10.262513+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:22:10.262513+0000', 'last_peered': '2024-09-15T21:22:10.262513+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:22:10.262513+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 6044, 'log_dups_size': 0, 'ondisk_log_size': 6044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24736227, 'num_objects': 6040, 'num_object_clones': 0, 'num_object_copies': 12080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6040, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2720, 'num_write_kb': 10880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1964}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:12.685 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:12.685 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'6180", 'reported_seq': 6310, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:10.175287+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:22:10.175287+0000', 'last_peered': '2024-09-15T21:22:10.175287+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:22:10.175287+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 6180, 'log_dups_size': 0, 'ondisk_log_size': 6180, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25289670, 'num_objects': 6176, 'num_object_clones': 0, 'num_object_copies': 12352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6176, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2491, 'num_write_kb': 9964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2015}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:12.685 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:12.685 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'6012", 'reported_seq': 6133, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:08.201558+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:22:08.201558+0000', 'last_peered': '2024-09-15T21:22:08.201558+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:22:08.201558+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 6012, 'log_dups_size': 0, 'ondisk_log_size': 6012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24596963, 'num_objects': 6006, 'num_object_clones': 0, 'num_object_copies': 12012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6006, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3944, 'num_write_kb': 15776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 1919}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:12.685 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:12.685 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'6028", 'reported_seq': 6153, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:10.244005+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:22:10.244005+0000', 'last_peered': '2024-09-15T21:22:10.244005+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:22:10.244005+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 6028, 'log_dups_size': 0, 'ondisk_log_size': 6028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24657920, 'num_objects': 6020, 'num_object_clones': 0, 'num_object_copies': 12040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6020, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2701, 'num_write_kb': 10804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2000}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:12.685 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:12.685 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'6015", 'reported_seq': 6128, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:08.194684+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:22:08.194684+0000', 'last_peered': '2024-09-15T21:22:08.194684+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:22:08.194684+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 6015, 'log_dups_size': 0, 'ondisk_log_size': 6015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24617443, 'num_objects': 6011, 'num_object_clones': 0, 'num_object_copies': 12022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6011, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3874, 'num_write_kb': 15496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 1921}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:12.685 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:12.686 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'6104", 'reported_seq': 8818, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:22:08.196014+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:22:08.196014+0000', 'last_peered': '2024-09-15T21:22:08.196014+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:22:08.196014+0000', 'last_undegraded': '2024-09-15T21:22:08.196014+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 6104, 'log_dups_size': 0, 'ondisk_log_size': 6104, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24977408, 'num_objects': 6098, 'num_object_clones': 0, 'num_object_copies': 12196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2864, 'num_objects_unfound': 0, 'num_objects_dirty': 6098, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3797, 'num_write_kb': 15188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1298, 'num_bytes_recovered': 5316608, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2864}, {'shards': '2', 'objects': 3234}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:12.686 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:12.686 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'6135", 'reported_seq': 6263, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:10.212874+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:22:10.212874+0000', 'last_peered': '2024-09-15T21:22:10.212874+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:22:10.212874+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 6135, 'log_dups_size': 0, 'ondisk_log_size': 6135, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25120768, 'num_objects': 6133, 'num_object_clones': 0, 'num_object_copies': 12266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6133, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3079, 'num_write_kb': 12316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2039}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:12.686 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:12.686 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'6140", 'reported_seq': 6266, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:10.272892+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:22:10.272892+0000', 'last_peered': '2024-09-15T21:22:10.272892+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:22:10.272892+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 6140, 'log_dups_size': 0, 'ondisk_log_size': 6140, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25116672, 'num_objects': 6132, 'num_object_clones': 0, 'num_object_copies': 12264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6132, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3049, 'num_write_kb': 12196, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1993}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:12.686 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:12.686 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'6063", 'reported_seq': 6191, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:10.264857+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:22:10.264857+0000', 'last_peered': '2024-09-15T21:22:10.264857+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:22:10.264857+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 6063, 'log_dups_size': 0, 'ondisk_log_size': 6063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24817664, 'num_objects': 6059, 'num_object_clones': 0, 'num_object_copies': 12118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6059, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2547, 'num_write_kb': 10188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1977}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:12.686 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:22:12.686 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'6148", 'reported_seq': 6281, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:10.156351+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:22:10.156351+0000', 'last_peered': '2024-09-15T21:22:10.156351+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:22:10.156351+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 6148, 'log_dups_size': 0, 'ondisk_log_size': 6148, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25134022, 'num_objects': 6138, 'num_object_clones': 0, 'num_object_copies': 12276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6138, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2852, 'num_write_kb': 11408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1991}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:12.687 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:12.743 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 101 20 99293 99273 3.83897 1.9375 0.0982284 0.0203421 2024-09-15T21:22:13.302 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:13.302 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/195168 objects degraded (23.215%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":185,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":97584,"data_bytes":399659761,"bytes_used":1318813696,"bytes_avail":288591478784,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":195168,"degraded_ratio":0.23215383669454009,"misplaced_objects":2864,"misplaced_total":195168,"misplaced_ratio":0.014674536809313,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":84968,"recovering_keys_per_sec":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"write_bytes_sec":1895587,"read_op_per_sec":0,"write_op_per_sec":462},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:13.327 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:13.328 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:13.743 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 102 20 99752 99732 3.81891 1.79297 0.0464285 0.0204483 2024-09-15T21:22:13.909 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:13.912 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:13.932 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-15T21:22:12.705539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":399659761,"num_objects":97584,"num_object_clones":0,"num_object_copies":195168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2864,"num_objects_unfound":0,"num_objects_dirty":97584,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":57449,"num_write_kb":229790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1300,"num_bytes_recovered":5320723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97657,"ondisk_log_size":97657,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1287904,"kb_used_data":787504,"kb_used_omap":12,"kb_used_meta":500339,"kb_avail":281827616,"statfs":{"total":289910292480,"available":288591478784,"internally_reserved":0,"allocated":806404096,"data_stored":805562931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":512347139},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,3,11,2,7,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22753280,"num_objects":5555,"num_object_clones":0,"num_object_copies":11110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-249,"num_objects_unfound":0,"num_objects_dirty":5555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5555,"num_write_kb":22220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":249,"num_bytes_recovered":1019904,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003284"},"pg_stats":[{"pgid":"2.f","version":"43'6216","reported_seq":6284,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.288215+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:10.288215+0000","last_peered":"2024-09-15T21:22:10.288215+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:10.288215+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6216,"log_dups_size":0,"ondisk_log_size":6216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25460736,"num_objects":6216,"num_object_clones":0,"num_object_copies":12432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6216,"num_write_kb":24864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2084},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6073","reported_seq":6173,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:08.184557+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:08.184557+0000","last_peered":"2024-09-15T21:22:08.184557+0000","last_clean":"2024-09-15T21:22:08.184557+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:08.184557+0000","last_undegraded":"2024-09-15T21:22:08.184557+0000","last_fullsized":"2024-09-15T21:22:08.184557+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6073,"log_dups_size":0,"ondisk_log_size":6073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24851398,"num_objects":6069,"num_object_clones":0,"num_object_copies":12138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6069,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4354,"num_write_kb":17416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6220","reported_seq":6340,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:08.200443+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:08.200443+0000","last_peered":"2024-09-15T21:22:08.200443+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:08.200443+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6220,"log_dups_size":0,"ondisk_log_size":6220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25460736,"num_objects":6216,"num_object_clones":0,"num_object_copies":12432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4320,"num_write_kb":17280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":1949}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5952","reported_seq":6017,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:08.182454+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:08.182454+0000","last_peered":"2024-09-15T21:22:08.182454+0000","last_clean":"2024-09-15T21:22:08.182454+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:08.182454+0000","last_undegraded":"2024-09-15T21:22:08.182454+0000","last_fullsized":"2024-09-15T21:22:08.182454+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":5952,"log_dups_size":0,"ondisk_log_size":5952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24379392,"num_objects":5952,"num_object_clones":0,"num_object_copies":11904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5952,"num_write_kb":23808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6073","reported_seq":6199,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:10.266573+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:10.266573+0000","last_peered":"2024-09-15T21:22:10.266573+0000","last_clean":"2024-09-15T21:22:10.266573+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:10.266573+0000","last_undegraded":"2024-09-15T21:22:10.266573+0000","last_fullsized":"2024-09-15T21:22:10.266573+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6073,"log_dups_size":0,"ondisk_log_size":6073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24846819,"num_objects":6067,"num_object_clones":0,"num_object_copies":12134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2528,"num_write_kb":10112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6044","reported_seq":6175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.262513+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:10.262513+0000","last_peered":"2024-09-15T21:22:10.262513+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:10.262513+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6044,"log_dups_size":0,"ondisk_log_size":6044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24736227,"num_objects":6040,"num_object_clones":0,"num_object_copies":12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6040,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2720,"num_write_kb":10880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1964},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6180","reported_seq":6310,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.175287+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:10.175287+0000","last_peered":"2024-09-15T21:22:10.175287+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:10.175287+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6180,"log_dups_size":0,"ondisk_log_size":6180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25289670,"num_objects":6176,"num_object_clones":0,"num_object_copies":12352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2491,"num_write_kb":9964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2015},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6251","reported_seq":6374,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:10.278005+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:10.278005+0000","last_peered":"2024-09-15T21:22:10.278005+0000","last_clean":"2024-09-15T21:22:10.278005+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:10.278005+0000","last_undegraded":"2024-09-15T21:22:10.278005+0000","last_fullsized":"2024-09-15T21:22:10.278005+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6251,"log_dups_size":0,"ondisk_log_size":6251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25595904,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3023,"num_write_kb":12092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6012","reported_seq":6133,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:08.201558+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:08.201558+0000","last_peered":"2024-09-15T21:22:08.201558+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:08.201558+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6012,"log_dups_size":0,"ondisk_log_size":6012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24596963,"num_objects":6006,"num_object_clones":0,"num_object_copies":12012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6006,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3944,"num_write_kb":15776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":1919}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6028","reported_seq":6153,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.244005+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:10.244005+0000","last_peered":"2024-09-15T21:22:10.244005+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:10.244005+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6028,"log_dups_size":0,"ondisk_log_size":6028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24657920,"num_objects":6020,"num_object_clones":0,"num_object_copies":12040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2701,"num_write_kb":10804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2000},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6015","reported_seq":6128,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:08.194684+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:08.194684+0000","last_peered":"2024-09-15T21:22:08.194684+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:08.194684+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6015,"log_dups_size":0,"ondisk_log_size":6015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24617443,"num_objects":6011,"num_object_clones":0,"num_object_copies":12022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6011,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3874,"num_write_kb":15496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":1921}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6104","reported_seq":8818,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:08.196014+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:08.196014+0000","last_peered":"2024-09-15T21:22:08.196014+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:08.196014+0000","last_undegraded":"2024-09-15T21:22:08.196014+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6104,"log_dups_size":0,"ondisk_log_size":6104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24977408,"num_objects":6098,"num_object_clones":0,"num_object_copies":12196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2864,"num_objects_unfound":0,"num_objects_dirty":6098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3797,"num_write_kb":15188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1298,"num_bytes_recovered":5316608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2864},{"shards":"2","objects":3234}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6135","reported_seq":6263,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.212874+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:10.212874+0000","last_peered":"2024-09-15T21:22:10.212874+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:10.212874+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6135,"log_dups_size":0,"ondisk_log_size":6135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25120768,"num_objects":6133,"num_object_clones":0,"num_object_copies":12266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3079,"num_write_kb":12316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2039},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6140","reported_seq":6266,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.272892+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:10.272892+0000","last_peered":"2024-09-15T21:22:10.272892+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:10.272892+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6140,"log_dups_size":0,"ondisk_log_size":6140,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25116672,"num_objects":6132,"num_object_clones":0,"num_object_copies":12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3049,"num_write_kb":12196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1993},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6063","reported_seq":6191,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.264857+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:10.264857+0000","last_peered":"2024-09-15T21:22:10.264857+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:10.264857+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6063,"log_dups_size":0,"ondisk_log_size":6063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24817664,"num_objects":6059,"num_object_clones":0,"num_object_copies":12118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2547,"num_write_kb":10188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1977},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6148","reported_seq":6281,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:10.156351+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:10.156351+0000","last_peered":"2024-09-15T21:22:10.156351+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:10.156351+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6148,"log_dups_size":0,"ondisk_log_size":6148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25134022,"num_objects":6138,"num_object_clones":0,"num_object_copies":12276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2852,"num_write_kb":11408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1991},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":399659742,"num_objects":97582,"num_object_clones":0,"num_object_copies":195164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2864,"num_objects_unfound":0,"num_objects_dirty":97582,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":57447,"num_write_kb":229788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1299,"num_bytes_recovered":5320704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":807424000,"data_stored":807268641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97654,"ondisk_log_size":97654,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514008,"kb_used_data":325448,"kb_used_omap":4,"kb_used_meta":188539,"kb_avail":93857832,"statfs":{"total":96636764160,"available":96110419968,"internally_reserved":0,"allocated":333258752,"data_stored":332996738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":193064617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535520,"kb_used_data":359504,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93836320,"statfs":{"total":96636764160,"available":96088391680,"internally_reserved":0,"allocated":368132096,"data_stored":367866005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180219563},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,7,2,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":238376,"kb_used_data":102552,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94133464,"statfs":{"total":96636764160,"available":96392667136,"internally_reserved":0,"allocated":105013248,"data_stored":104700188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":139062959},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,0,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":368578560,"data_stored":368542430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":105512960,"data_stored":105429861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333332480,"data_stored":333296350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:14.743 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 103 20 100316 100296 3.80322 2.20312 0.0122866 0.0205337 2024-09-15T21:22:15.743 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 104 20 100894 100874 3.78836 2.25781 0.0524609 0.0206131 2024-09-15T21:22:16.743 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 105 20 101463 101443 3.77344 2.22266 0.045111 0.0206961 2024-09-15T21:22:16.934 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:17.488 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:17.489 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:17.523 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-15T21:22:16.706407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":410465009,"num_objects":100222,"num_object_clones":0,"num_object_copies":200444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2727,"num_objects_unfound":0,"num_objects_dirty":100222,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":60087,"num_write_kb":240342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1437,"num_bytes_recovered":5881875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100295,"ondisk_log_size":100295,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1305372,"kb_used_data":809132,"kb_used_omap":12,"kb_used_meta":496179,"kb_avail":281810148,"statfs":{"total":289910292480,"available":288573591552,"internally_reserved":0,"allocated":828551168,"data_stored":827710003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":508087299},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,7,5,7,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29208576,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-261,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":28524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":1069056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003290"},"pg_stats":[{"pgid":"2.f","version":"43'6413","reported_seq":6484,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.267790+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:15.267790+0000","last_peered":"2024-09-15T21:22:15.267790+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:15.267790+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6413,"log_dups_size":0,"ondisk_log_size":6413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26267648,"num_objects":6413,"num_object_clones":0,"num_object_copies":12826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6413,"num_write_kb":25652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2281},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6215","reported_seq":6315,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:13.151988+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:13.151988+0000","last_peered":"2024-09-15T21:22:13.151988+0000","last_clean":"2024-09-15T21:22:13.151988+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:13.151988+0000","last_undegraded":"2024-09-15T21:22:13.151988+0000","last_fullsized":"2024-09-15T21:22:13.151988+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6215,"log_dups_size":0,"ondisk_log_size":6215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25433030,"num_objects":6211,"num_object_clones":0,"num_object_copies":12422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6211,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4496,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6392","reported_seq":6512,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.139985+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:13.139985+0000","last_peered":"2024-09-15T21:22:13.139985+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:13.139985+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6392,"log_dups_size":0,"ondisk_log_size":6392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26165248,"num_objects":6388,"num_object_clones":0,"num_object_copies":12776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4492,"num_write_kb":17968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6103","reported_seq":6168,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:13.234202+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:13.234202+0000","last_peered":"2024-09-15T21:22:13.234202+0000","last_clean":"2024-09-15T21:22:13.234202+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:13.234202+0000","last_undegraded":"2024-09-15T21:22:13.234202+0000","last_fullsized":"2024-09-15T21:22:13.234202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6103,"log_dups_size":0,"ondisk_log_size":6103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24997888,"num_objects":6103,"num_object_clones":0,"num_object_copies":12206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6103,"num_write_kb":24412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6240","reported_seq":6368,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:15.300181+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:15.300181+0000","last_peered":"2024-09-15T21:22:15.300181+0000","last_clean":"2024-09-15T21:22:15.300181+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:15.300181+0000","last_undegraded":"2024-09-15T21:22:15.300181+0000","last_fullsized":"2024-09-15T21:22:15.300181+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6240,"log_dups_size":0,"ondisk_log_size":6240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25530851,"num_objects":6234,"num_object_clones":0,"num_object_copies":12468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2695,"num_write_kb":10780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6233","reported_seq":6364,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.267074+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:15.267074+0000","last_peered":"2024-09-15T21:22:15.267074+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:15.267074+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6233,"log_dups_size":0,"ondisk_log_size":6233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25510371,"num_objects":6229,"num_object_clones":0,"num_object_copies":12458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6229,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2909,"num_write_kb":11636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2153},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6360","reported_seq":6491,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.289941+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:15.289941+0000","last_peered":"2024-09-15T21:22:15.289941+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:15.289941+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6360,"log_dups_size":0,"ondisk_log_size":6360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26026950,"num_objects":6356,"num_object_clones":0,"num_object_copies":12712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2671,"num_write_kb":10684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2195},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6428","reported_seq":6552,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:15.174205+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:15.174205+0000","last_peered":"2024-09-15T21:22:15.174205+0000","last_clean":"2024-09-15T21:22:15.174205+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:15.174205+0000","last_undegraded":"2024-09-15T21:22:15.174205+0000","last_fullsized":"2024-09-15T21:22:15.174205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6428,"log_dups_size":0,"ondisk_log_size":6428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26320896,"num_objects":6426,"num_object_clones":0,"num_object_copies":12852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3200,"num_write_kb":12800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6171","reported_seq":6292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.167564+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:13.167564+0000","last_peered":"2024-09-15T21:22:13.167564+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:13.167564+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6171,"log_dups_size":0,"ondisk_log_size":6171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25248227,"num_objects":6165,"num_object_clones":0,"num_object_copies":12330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4103,"num_write_kb":16412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2078}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6177","reported_seq":6303,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.259893+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:15.259893+0000","last_peered":"2024-09-15T21:22:15.259893+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:15.259893+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6177,"log_dups_size":0,"ondisk_log_size":6177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25268224,"num_objects":6169,"num_object_clones":0,"num_object_copies":12338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2850,"num_write_kb":11400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2149},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6179","reported_seq":6291,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.224212+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:13.224212+0000","last_peered":"2024-09-15T21:22:13.224212+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:13.224212+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6179,"log_dups_size":0,"ondisk_log_size":6179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25289187,"num_objects":6175,"num_object_clones":0,"num_object_copies":12350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6175,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4038,"num_write_kb":16152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2085}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6276","reported_seq":9263,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:13.232725+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:13.232725+0000","last_peered":"2024-09-15T21:22:13.232725+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:13.232725+0000","last_undegraded":"2024-09-15T21:22:13.232725+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6276,"log_dups_size":0,"ondisk_log_size":6276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25681920,"num_objects":6270,"num_object_clones":0,"num_object_copies":12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2727,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3969,"num_write_kb":15876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1435,"num_bytes_recovered":5877760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2727},{"shards":"2","objects":3543}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6306","reported_seq":6434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.265699+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:15.265699+0000","last_peered":"2024-09-15T21:22:15.265699+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:15.265699+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6306,"log_dups_size":0,"ondisk_log_size":6306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":6304,"num_object_clones":0,"num_object_copies":12608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3250,"num_write_kb":13000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2210},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6303","reported_seq":6430,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.295721+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:15.295721+0000","last_peered":"2024-09-15T21:22:15.295721+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:15.295721+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6303,"log_dups_size":0,"ondisk_log_size":6303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25784320,"num_objects":6295,"num_object_clones":0,"num_object_copies":12590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3212,"num_write_kb":12848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2156},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6205","reported_seq":6335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.199569+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:15.199569+0000","last_peered":"2024-09-15T21:22:15.199569+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:15.199569+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6205,"log_dups_size":0,"ondisk_log_size":6205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25399296,"num_objects":6201,"num_object_clones":0,"num_object_copies":12402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2689,"num_write_kb":10756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2119},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6291","reported_seq":6424,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.244779+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:15.244779+0000","last_peered":"2024-09-15T21:22:15.244779+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:15.244779+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6291,"log_dups_size":0,"ondisk_log_size":6291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25719750,"num_objects":6281,"num_object_clones":0,"num_object_copies":12562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2995,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2134},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":410464990,"num_objects":100220,"num_object_clones":0,"num_object_copies":200440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2727,"num_objects_unfound":0,"num_objects_dirty":100220,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":60085,"num_write_kb":240340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1436,"num_bytes_recovered":5881856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":829579264,"data_stored":829423905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100292,"ondisk_log_size":100292,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506828,"kb_used_data":330748,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93865012,"statfs":{"total":96636764160,"available":96117772288,"internally_reserved":0,"allocated":338685952,"data_stored":338423938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":180285097},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547576,"kb_used_data":367400,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93824264,"statfs":{"total":96636764160,"available":96076046336,"internally_reserved":0,"allocated":376217600,"data_stored":375951509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184479403},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,2,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250968,"kb_used_data":110984,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94120872,"statfs":{"total":96636764160,"available":96379772928,"internally_reserved":0,"allocated":113647616,"data_stored":113334556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":376725504,"data_stored":376689374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114192384,"data_stored":114109285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338661376,"data_stored":338625246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:17.524 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:17.743 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 106 20 101981 101961 3.75693 2.02344 0.0798125 0.0207847 2024-09-15T21:22:18.068 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:18.185 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:18.185 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-15T21:22:16.706407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":410465009,"num_objects":100222,"num_object_clones":0,"num_object_copies":200444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2727,"num_objects_unfound":0,"num_objects_dirty":100222,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":60087,"num_write_kb":240342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1437,"num_bytes_recovered":5881875,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100295,"ondisk_log_size":100295,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1305372,"kb_used_data":809132,"kb_used_omap":12,"kb_used_meta":496179,"kb_avail":281810148,"statfs":{"total":289910292480,"available":288573591552,"internally_reserved":0,"allocated":828551168,"data_stored":827710003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":508087299},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,7,5,7,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29208576,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-261,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7131,"num_write_kb":28524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":1069056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003290"},"pg_stats":[{"pgid":"2.f","version":"43'6413","reported_seq":6484,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.267790+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:15.267790+0000","last_peered":"2024-09-15T21:22:15.267790+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:15.267790+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6413,"log_dups_size":0,"ondisk_log_size":6413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26267648,"num_objects":6413,"num_object_clones":0,"num_object_copies":12826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6413,"num_write_kb":25652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2281},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6215","reported_seq":6315,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:13.151988+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:13.151988+0000","last_peered":"2024-09-15T21:22:13.151988+0000","last_clean":"2024-09-15T21:22:13.151988+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:13.151988+0000","last_undegraded":"2024-09-15T21:22:13.151988+0000","last_fullsized":"2024-09-15T21:22:13.151988+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6215,"log_dups_size":0,"ondisk_log_size":6215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25433030,"num_objects":6211,"num_object_clones":0,"num_object_copies":12422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6211,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4496,"num_write_kb":17984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6392","reported_seq":6512,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.139985+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:13.139985+0000","last_peered":"2024-09-15T21:22:13.139985+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:13.139985+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6392,"log_dups_size":0,"ondisk_log_size":6392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26165248,"num_objects":6388,"num_object_clones":0,"num_object_copies":12776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4492,"num_write_kb":17968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6103","reported_seq":6168,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:13.234202+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:13.234202+0000","last_peered":"2024-09-15T21:22:13.234202+0000","last_clean":"2024-09-15T21:22:13.234202+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:13.234202+0000","last_undegraded":"2024-09-15T21:22:13.234202+0000","last_fullsized":"2024-09-15T21:22:13.234202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6103,"log_dups_size":0,"ondisk_log_size":6103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24997888,"num_objects":6103,"num_object_clones":0,"num_object_copies":12206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6103,"num_write_kb":24412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6240","reported_seq":6368,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:15.300181+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:15.300181+0000","last_peered":"2024-09-15T21:22:15.300181+0000","last_clean":"2024-09-15T21:22:15.300181+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:15.300181+0000","last_undegraded":"2024-09-15T21:22:15.300181+0000","last_fullsized":"2024-09-15T21:22:15.300181+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6240,"log_dups_size":0,"ondisk_log_size":6240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25530851,"num_objects":6234,"num_object_clones":0,"num_object_copies":12468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2695,"num_write_kb":10780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6233","reported_seq":6364,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.267074+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:15.267074+0000","last_peered":"2024-09-15T21:22:15.267074+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:15.267074+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6233,"log_dups_size":0,"ondisk_log_size":6233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25510371,"num_objects":6229,"num_object_clones":0,"num_object_copies":12458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6229,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2909,"num_write_kb":11636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2153},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6360","reported_seq":6491,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.289941+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:15.289941+0000","last_peered":"2024-09-15T21:22:15.289941+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:15.289941+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6360,"log_dups_size":0,"ondisk_log_size":6360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26026950,"num_objects":6356,"num_object_clones":0,"num_object_copies":12712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2671,"num_write_kb":10684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2195},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6428","reported_seq":6552,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:15.174205+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:15.174205+0000","last_peered":"2024-09-15T21:22:15.174205+0000","last_clean":"2024-09-15T21:22:15.174205+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:15.174205+0000","last_undegraded":"2024-09-15T21:22:15.174205+0000","last_fullsized":"2024-09-15T21:22:15.174205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6428,"log_dups_size":0,"ondisk_log_size":6428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26320896,"num_objects":6426,"num_object_clones":0,"num_object_copies":12852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3200,"num_write_kb":12800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6171","reported_seq":6292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.167564+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:13.167564+0000","last_peered":"2024-09-15T21:22:13.167564+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:13.167564+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6171,"log_dups_size":0,"ondisk_log_size":6171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25248227,"num_objects":6165,"num_object_clones":0,"num_object_copies":12330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4103,"num_write_kb":16412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2078}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6177","reported_seq":6303,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.259893+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:15.259893+0000","last_peered":"2024-09-15T21:22:15.259893+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:15.259893+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6177,"log_dups_size":0,"ondisk_log_size":6177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25268224,"num_objects":6169,"num_object_clones":0,"num_object_copies":12338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2850,"num_write_kb":11400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2149},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6179","reported_seq":6291,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:13.224212+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:13.224212+0000","last_peered":"2024-09-15T21:22:13.224212+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:13.224212+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6179,"log_dups_size":0,"ondisk_log_size":6179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25289187,"num_objects":6175,"num_object_clones":0,"num_object_copies":12350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6175,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4038,"num_write_kb":16152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2085}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6276","reported_seq":9263,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:13.232725+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:13.232725+0000","last_peered":"2024-09-15T21:22:13.232725+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:13.232725+0000","last_undegraded":"2024-09-15T21:22:13.232725+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6276,"log_dups_size":0,"ondisk_log_size":6276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25681920,"num_objects":6270,"num_object_clones":0,"num_object_copies":12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2727,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3969,"num_write_kb":15876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1435,"num_bytes_recovered":5877760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2727},{"shards":"2","objects":3543}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6306","reported_seq":6434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.265699+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:15.265699+0000","last_peered":"2024-09-15T21:22:15.265699+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:15.265699+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6306,"log_dups_size":0,"ondisk_log_size":6306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":6304,"num_object_clones":0,"num_object_copies":12608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3250,"num_write_kb":13000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2210},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6303","reported_seq":6430,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.295721+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:15.295721+0000","last_peered":"2024-09-15T21:22:15.295721+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:15.295721+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6303,"log_dups_size":0,"ondisk_log_size":6303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25784320,"num_objects":6295,"num_object_clones":0,"num_object_copies":12590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3212,"num_write_kb":12848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2156},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6205","reported_seq":6335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.199569+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:15.199569+0000","last_peered":"2024-09-15T21:22:15.199569+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:15.199569+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6205,"log_dups_size":0,"ondisk_log_size":6205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25399296,"num_objects":6201,"num_object_clones":0,"num_object_copies":12402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2689,"num_write_kb":10756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2119},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6291","reported_seq":6424,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.244779+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:15.244779+0000","last_peered":"2024-09-15T21:22:15.244779+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:15.244779+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6291,"log_dups_size":0,"ondisk_log_size":6291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25719750,"num_objects":6281,"num_object_clones":0,"num_object_copies":12562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2995,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2134},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":410464990,"num_objects":100220,"num_object_clones":0,"num_object_copies":200440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2727,"num_objects_unfound":0,"num_objects_dirty":100220,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":60085,"num_write_kb":240340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1436,"num_bytes_recovered":5881856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":829579264,"data_stored":829423905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100292,"ondisk_log_size":100292,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506828,"kb_used_data":330748,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93865012,"statfs":{"total":96636764160,"available":96117772288,"internally_reserved":0,"allocated":338685952,"data_stored":338423938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":180285097},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547576,"kb_used_data":367400,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93824264,"statfs":{"total":96636764160,"available":96076046336,"internally_reserved":0,"allocated":376217600,"data_stored":375951509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184479403},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,2,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250968,"kb_used_data":110984,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94120872,"statfs":{"total":96636764160,"available":96379772928,"internally_reserved":0,"allocated":113647616,"data_stored":113334556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":376725504,"data_stored":376689374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114192384,"data_stored":114109285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338661376,"data_stored":338625246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:18.186 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:18.186 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'6413", 'reported_seq': 6484, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:15.267790+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:22:15.267790+0000', 'last_peered': '2024-09-15T21:22:15.267790+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:22:15.267790+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 6413, 'log_dups_size': 0, 'ondisk_log_size': 6413, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26267648, 'num_objects': 6413, 'num_object_clones': 0, 'num_object_copies': 12826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6413, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6413, 'num_write_kb': 25652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2281}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:18.186 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:22:18.186 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'6392", 'reported_seq': 6512, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:13.139985+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:22:13.139985+0000', 'last_peered': '2024-09-15T21:22:13.139985+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:22:13.139985+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 6392, 'log_dups_size': 0, 'ondisk_log_size': 6392, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26165248, 'num_objects': 6388, 'num_object_clones': 0, 'num_object_copies': 12776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4492, 'num_write_kb': 17968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 2121}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:18.187 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:18.187 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'6233", 'reported_seq': 6364, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:15.267074+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:22:15.267074+0000', 'last_peered': '2024-09-15T21:22:15.267074+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:22:15.267074+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 6233, 'log_dups_size': 0, 'ondisk_log_size': 6233, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25510371, 'num_objects': 6229, 'num_object_clones': 0, 'num_object_copies': 12458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6229, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2909, 'num_write_kb': 11636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2153}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:18.187 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:18.187 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'6360", 'reported_seq': 6491, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:15.289941+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:22:15.289941+0000', 'last_peered': '2024-09-15T21:22:15.289941+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:22:15.289941+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 6360, 'log_dups_size': 0, 'ondisk_log_size': 6360, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26026950, 'num_objects': 6356, 'num_object_clones': 0, 'num_object_copies': 12712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6356, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2671, 'num_write_kb': 10684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2195}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:18.187 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:18.187 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'6171", 'reported_seq': 6292, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:13.167564+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:22:13.167564+0000', 'last_peered': '2024-09-15T21:22:13.167564+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:22:13.167564+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 6171, 'log_dups_size': 0, 'ondisk_log_size': 6171, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25248227, 'num_objects': 6165, 'num_object_clones': 0, 'num_object_copies': 12330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6165, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4103, 'num_write_kb': 16412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 2078}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:18.187 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:18.187 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'6177", 'reported_seq': 6303, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:15.259893+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:22:15.259893+0000', 'last_peered': '2024-09-15T21:22:15.259893+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:22:15.259893+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 6177, 'log_dups_size': 0, 'ondisk_log_size': 6177, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25268224, 'num_objects': 6169, 'num_object_clones': 0, 'num_object_copies': 12338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6169, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2850, 'num_write_kb': 11400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2149}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:18.187 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:18.187 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'6179", 'reported_seq': 6291, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:13.224212+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:22:13.224212+0000', 'last_peered': '2024-09-15T21:22:13.224212+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:22:13.224212+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 6179, 'log_dups_size': 0, 'ondisk_log_size': 6179, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25289187, 'num_objects': 6175, 'num_object_clones': 0, 'num_object_copies': 12350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6175, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4038, 'num_write_kb': 16152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 2085}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:18.188 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:18.188 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'6276", 'reported_seq': 9263, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:22:13.232725+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:22:13.232725+0000', 'last_peered': '2024-09-15T21:22:13.232725+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:22:13.232725+0000', 'last_undegraded': '2024-09-15T21:22:13.232725+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 6276, 'log_dups_size': 0, 'ondisk_log_size': 6276, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25681920, 'num_objects': 6270, 'num_object_clones': 0, 'num_object_copies': 12540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2727, 'num_objects_unfound': 0, 'num_objects_dirty': 6270, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3969, 'num_write_kb': 15876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1435, 'num_bytes_recovered': 5877760, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2727}, {'shards': '2', 'objects': 3543}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:18.188 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:18.188 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'6306", 'reported_seq': 6434, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:15.265699+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:22:15.265699+0000', 'last_peered': '2024-09-15T21:22:15.265699+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:22:15.265699+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 6306, 'log_dups_size': 0, 'ondisk_log_size': 6306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25821184, 'num_objects': 6304, 'num_object_clones': 0, 'num_object_copies': 12608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6304, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3250, 'num_write_kb': 13000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2210}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:18.188 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:18.188 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'6303", 'reported_seq': 6430, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:15.295721+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:22:15.295721+0000', 'last_peered': '2024-09-15T21:22:15.295721+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:22:15.295721+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 6303, 'log_dups_size': 0, 'ondisk_log_size': 6303, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25784320, 'num_objects': 6295, 'num_object_clones': 0, 'num_object_copies': 12590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6295, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3212, 'num_write_kb': 12848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2156}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:18.188 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:18.188 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'6205", 'reported_seq': 6335, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:15.199569+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:22:15.199569+0000', 'last_peered': '2024-09-15T21:22:15.199569+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:22:15.199569+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 6205, 'log_dups_size': 0, 'ondisk_log_size': 6205, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25399296, 'num_objects': 6201, 'num_object_clones': 0, 'num_object_copies': 12402, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6201, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2689, 'num_write_kb': 10756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2119}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:18.188 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:22:18.188 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'6291", 'reported_seq': 6424, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:15.244779+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:22:15.244779+0000', 'last_peered': '2024-09-15T21:22:15.244779+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:22:15.244779+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 6291, 'log_dups_size': 0, 'ondisk_log_size': 6291, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25719750, 'num_objects': 6281, 'num_object_clones': 0, 'num_object_copies': 12562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6281, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 2995, 'num_write_kb': 11980, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2134}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:18.189 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:18.747 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 107 20 102493 102473 3.74045 2 0.0515517 0.0208763 2024-09-15T21:22:18.895 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:18.896 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/200444 objects degraded (22.604%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":190,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":100222,"data_bytes":410465009,"bytes_used":1336700928,"bytes_avail":288573591552,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":200444,"degraded_ratio":0.22604318413122867,"misplaced_objects":2727,"misplaced_total":200444,"misplaced_ratio":0.01360479734988326,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":89063,"recovering_keys_per_sec":0,"num_objects_recovered":261,"num_bytes_recovered":1069056,"num_keys_recovered":0,"write_bytes_sec":2433380,"read_op_per_sec":0,"write_op_per_sec":594},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:18.922 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:18.922 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:19.490 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:19.492 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:19.514 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-15T21:22:18.706914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":414700273,"num_objects":101256,"num_object_clones":0,"num_object_copies":202512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2591,"num_objects_unfound":0,"num_objects_dirty":101256,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":61121,"num_write_kb":244478,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1573,"num_bytes_recovered":6438931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101329,"ondisk_log_size":101329,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1315064,"kb_used_data":814664,"kb_used_omap":12,"kb_used_meta":500339,"kb_avail":281800456,"statfs":{"total":289910292480,"available":288563666944,"internally_reserved":0,"allocated":834215936,"data_stored":833374771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":512347139},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,8,7,10,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26484736,"num_objects":6466,"num_object_clones":0,"num_object_copies":12932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-397,"num_objects_unfound":0,"num_objects_dirty":6466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6466,"num_write_kb":25864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":397,"num_bytes_recovered":1626112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003347"},"pg_stats":[{"pgid":"2.f","version":"43'6413","reported_seq":6484,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.267790+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:15.267790+0000","last_peered":"2024-09-15T21:22:15.267790+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:15.267790+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6413,"log_dups_size":0,"ondisk_log_size":6413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26267648,"num_objects":6413,"num_object_clones":0,"num_object_copies":12826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6413,"num_write_kb":25652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2281},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6400","reported_seq":6500,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.169452+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:18.169452+0000","last_peered":"2024-09-15T21:22:18.169452+0000","last_clean":"2024-09-15T21:22:18.169452+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:18.169452+0000","last_undegraded":"2024-09-15T21:22:18.169452+0000","last_fullsized":"2024-09-15T21:22:18.169452+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6400,"log_dups_size":0,"ondisk_log_size":6400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26190790,"num_objects":6396,"num_object_clones":0,"num_object_copies":12792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6396,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4681,"num_write_kb":18724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6571","reported_seq":6690,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.235043+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:18.235043+0000","last_peered":"2024-09-15T21:22:18.235043+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:18.235043+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6571,"log_dups_size":0,"ondisk_log_size":6571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26898432,"num_objects":6567,"num_object_clones":0,"num_object_copies":13134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4671,"num_write_kb":18684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2300}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6274","reported_seq":6339,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.242420+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:18.242420+0000","last_peered":"2024-09-15T21:22:18.242420+0000","last_clean":"2024-09-15T21:22:18.242420+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:18.242420+0000","last_undegraded":"2024-09-15T21:22:18.242420+0000","last_fullsized":"2024-09-15T21:22:18.242420+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6274,"log_dups_size":0,"ondisk_log_size":6274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25698304,"num_objects":6274,"num_object_clones":0,"num_object_copies":12548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6274,"num_write_kb":25096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6240","reported_seq":6368,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:15.300181+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:15.300181+0000","last_peered":"2024-09-15T21:22:15.300181+0000","last_clean":"2024-09-15T21:22:15.300181+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:15.300181+0000","last_undegraded":"2024-09-15T21:22:15.300181+0000","last_fullsized":"2024-09-15T21:22:15.300181+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6240,"log_dups_size":0,"ondisk_log_size":6240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25530851,"num_objects":6234,"num_object_clones":0,"num_object_copies":12468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2695,"num_write_kb":10780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6233","reported_seq":6364,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.267074+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:15.267074+0000","last_peered":"2024-09-15T21:22:15.267074+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:15.267074+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6233,"log_dups_size":0,"ondisk_log_size":6233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25510371,"num_objects":6229,"num_object_clones":0,"num_object_copies":12458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6229,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2909,"num_write_kb":11636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2153},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6360","reported_seq":6491,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.289941+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:15.289941+0000","last_peered":"2024-09-15T21:22:15.289941+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:15.289941+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6360,"log_dups_size":0,"ondisk_log_size":6360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26026950,"num_objects":6356,"num_object_clones":0,"num_object_copies":12712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2671,"num_write_kb":10684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2195},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6428","reported_seq":6552,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:15.174205+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:15.174205+0000","last_peered":"2024-09-15T21:22:15.174205+0000","last_clean":"2024-09-15T21:22:15.174205+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:15.174205+0000","last_undegraded":"2024-09-15T21:22:15.174205+0000","last_fullsized":"2024-09-15T21:22:15.174205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6428,"log_dups_size":0,"ondisk_log_size":6428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26320896,"num_objects":6426,"num_object_clones":0,"num_object_copies":12852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3200,"num_write_kb":12800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6339","reported_seq":6460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.181885+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:18.181885+0000","last_peered":"2024-09-15T21:22:18.181885+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:18.181885+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6339,"log_dups_size":0,"ondisk_log_size":6339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25936355,"num_objects":6333,"num_object_clones":0,"num_object_copies":12666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6333,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4271,"num_write_kb":17084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2246}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6177","reported_seq":6303,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.259893+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:15.259893+0000","last_peered":"2024-09-15T21:22:15.259893+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:15.259893+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6177,"log_dups_size":0,"ondisk_log_size":6177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25268224,"num_objects":6169,"num_object_clones":0,"num_object_copies":12338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2850,"num_write_kb":11400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2149},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6346","reported_seq":6459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.205877+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:18.205877+0000","last_peered":"2024-09-15T21:22:18.205877+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:18.205877+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6346,"log_dups_size":0,"ondisk_log_size":6346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25973219,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4205,"num_write_kb":16820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2252}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6440","reported_seq":9698,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:18.254920+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:18.254920+0000","last_peered":"2024-09-15T21:22:18.254920+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:18.254920+0000","last_undegraded":"2024-09-15T21:22:18.254920+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6440,"log_dups_size":0,"ondisk_log_size":6440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26353664,"num_objects":6434,"num_object_clones":0,"num_object_copies":12868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2591,"num_objects_unfound":0,"num_objects_dirty":6434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4133,"num_write_kb":16532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1571,"num_bytes_recovered":6434816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2591},{"shards":"2","objects":3843}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6306","reported_seq":6434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.265699+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:15.265699+0000","last_peered":"2024-09-15T21:22:15.265699+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:15.265699+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6306,"log_dups_size":0,"ondisk_log_size":6306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25821184,"num_objects":6304,"num_object_clones":0,"num_object_copies":12608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3250,"num_write_kb":13000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2210},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6303","reported_seq":6430,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.295721+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:15.295721+0000","last_peered":"2024-09-15T21:22:15.295721+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:15.295721+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6303,"log_dups_size":0,"ondisk_log_size":6303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25784320,"num_objects":6295,"num_object_clones":0,"num_object_copies":12590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3212,"num_write_kb":12848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2156},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6205","reported_seq":6335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.199569+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:15.199569+0000","last_peered":"2024-09-15T21:22:15.199569+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:15.199569+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6205,"log_dups_size":0,"ondisk_log_size":6205,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25399296,"num_objects":6201,"num_object_clones":0,"num_object_copies":12402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2689,"num_write_kb":10756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2119},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6291","reported_seq":6424,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:15.244779+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:15.244779+0000","last_peered":"2024-09-15T21:22:15.244779+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:15.244779+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6291,"log_dups_size":0,"ondisk_log_size":6291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25719750,"num_objects":6281,"num_object_clones":0,"num_object_copies":12562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6281,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2995,"num_write_kb":11980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2134},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":414700254,"num_objects":101254,"num_object_clones":0,"num_object_copies":202508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2591,"num_objects_unfound":0,"num_objects_dirty":101254,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":61119,"num_write_kb":244476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1572,"num_bytes_recovered":6438912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":835375104,"data_stored":835219745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":101326,"ondisk_log_size":101326,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516520,"kb_used_data":336280,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93855320,"statfs":{"total":96636764160,"available":96107847680,"internally_reserved":0,"allocated":344350720,"data_stored":344088706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":184544937},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547576,"kb_used_data":367400,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93824264,"statfs":{"total":96636764160,"available":96076046336,"internally_reserved":0,"allocated":376217600,"data_stored":375951509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184479403},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,2,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":250968,"kb_used_data":110984,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94120872,"statfs":{"total":96636764160,"available":96379772928,"internally_reserved":0,"allocated":113647616,"data_stored":113334556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":376725504,"data_stored":376689374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114192384,"data_stored":114109285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344457216,"data_stored":344421086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:19.745 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 108 20 103002 102982 3.72423 1.98828 0.0183063 0.0209708 2024-09-15T21:22:20.745 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 109 20 103565 103545 3.71024 2.19922 0.0987955 0.0210463 2024-09-15T21:22:21.745 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 110 20 104146 104126 3.69714 2.26953 0.0181265 0.0211234 2024-09-15T21:22:22.516 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:22.745 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 111 20 104686 104666 3.68283 2.10938 0.0876064 0.0212041 2024-09-15T21:22:23.095 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:23.098 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:23.119 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-15T21:22:22.707829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":421548785,"num_objects":102928,"num_object_clones":0,"num_object_copies":205856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2591,"num_objects_unfound":0,"num_objects_dirty":102928,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":62793,"num_write_kb":251166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1573,"num_bytes_recovered":6438931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103001,"ondisk_log_size":103001,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1366472,"kb_used_data":831384,"kb_used_omap":12,"kb_used_meta":535027,"kb_avail":281749048,"statfs":{"total":289910292480,"available":288511025152,"internally_reserved":0,"allocated":851337216,"data_stored":850496051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":547867651},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,8,12,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21889024,"num_objects":5344,"num_object_clones":0,"num_object_copies":10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-273,"num_objects_unfound":0,"num_objects_dirty":5344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5344,"num_write_kb":21376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003206"},"pg_stats":[{"pgid":"2.f","version":"43'6565","reported_seq":6636,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.302407+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:20.302407+0000","last_peered":"2024-09-15T21:22:20.302407+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:20.302407+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6565,"log_dups_size":0,"ondisk_log_size":6565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26890240,"num_objects":6565,"num_object_clones":0,"num_object_copies":13130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6565,"num_write_kb":26260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2433},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6400","reported_seq":6500,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.169452+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:18.169452+0000","last_peered":"2024-09-15T21:22:18.169452+0000","last_clean":"2024-09-15T21:22:18.169452+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:18.169452+0000","last_undegraded":"2024-09-15T21:22:18.169452+0000","last_fullsized":"2024-09-15T21:22:18.169452+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6400,"log_dups_size":0,"ondisk_log_size":6400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26190790,"num_objects":6396,"num_object_clones":0,"num_object_copies":12792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6396,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4681,"num_write_kb":18724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6571","reported_seq":6690,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.235043+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:18.235043+0000","last_peered":"2024-09-15T21:22:18.235043+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:18.235043+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6571,"log_dups_size":0,"ondisk_log_size":6571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26898432,"num_objects":6567,"num_object_clones":0,"num_object_copies":13134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4671,"num_write_kb":18684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2300}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6274","reported_seq":6339,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.242420+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:18.242420+0000","last_peered":"2024-09-15T21:22:18.242420+0000","last_clean":"2024-09-15T21:22:18.242420+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:18.242420+0000","last_undegraded":"2024-09-15T21:22:18.242420+0000","last_fullsized":"2024-09-15T21:22:18.242420+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6274,"log_dups_size":0,"ondisk_log_size":6274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25698304,"num_objects":6274,"num_object_clones":0,"num_object_copies":12548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6274,"num_write_kb":25096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6437","reported_seq":6565,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:20.335589+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:20.335589+0000","last_peered":"2024-09-15T21:22:20.335589+0000","last_clean":"2024-09-15T21:22:20.335589+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:20.335589+0000","last_undegraded":"2024-09-15T21:22:20.335589+0000","last_fullsized":"2024-09-15T21:22:20.335589+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6437,"log_dups_size":0,"ondisk_log_size":6437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26337763,"num_objects":6431,"num_object_clones":0,"num_object_copies":12862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2892,"num_write_kb":11568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6410","reported_seq":6541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.322672+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:20.322672+0000","last_peered":"2024-09-15T21:22:20.322672+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:20.322672+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6410,"log_dups_size":0,"ondisk_log_size":6410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26235363,"num_objects":6406,"num_object_clones":0,"num_object_copies":12812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3086,"num_write_kb":12344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2330},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6515","reported_seq":6646,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.208016+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:20.208016+0000","last_peered":"2024-09-15T21:22:20.208016+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:20.208016+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6515,"log_dups_size":0,"ondisk_log_size":6515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26661830,"num_objects":6511,"num_object_clones":0,"num_object_copies":13022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2826,"num_write_kb":11304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2350},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6594","reported_seq":6718,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:20.302413+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:20.302413+0000","last_peered":"2024-09-15T21:22:20.302413+0000","last_clean":"2024-09-15T21:22:20.302413+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:20.302413+0000","last_undegraded":"2024-09-15T21:22:20.302413+0000","last_fullsized":"2024-09-15T21:22:20.302413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6594,"log_dups_size":0,"ondisk_log_size":6594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27000832,"num_objects":6592,"num_object_clones":0,"num_object_copies":13184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3366,"num_write_kb":13464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6339","reported_seq":6460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.181885+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:18.181885+0000","last_peered":"2024-09-15T21:22:18.181885+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:18.181885+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6339,"log_dups_size":0,"ondisk_log_size":6339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25936355,"num_objects":6333,"num_object_clones":0,"num_object_copies":12666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6333,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4271,"num_write_kb":17084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2246}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6365","reported_seq":6491,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.314601+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:20.314601+0000","last_peered":"2024-09-15T21:22:20.314601+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:20.314601+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6365,"log_dups_size":0,"ondisk_log_size":6365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26038272,"num_objects":6357,"num_object_clones":0,"num_object_copies":12714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3038,"num_write_kb":12152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2337},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6346","reported_seq":6459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.205877+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:18.205877+0000","last_peered":"2024-09-15T21:22:18.205877+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:18.205877+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6346,"log_dups_size":0,"ondisk_log_size":6346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25973219,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4205,"num_write_kb":16820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2252}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6440","reported_seq":9698,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:18.254920+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:18.254920+0000","last_peered":"2024-09-15T21:22:18.254920+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:18.254920+0000","last_undegraded":"2024-09-15T21:22:18.254920+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6440,"log_dups_size":0,"ondisk_log_size":6440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26353664,"num_objects":6434,"num_object_clones":0,"num_object_copies":12868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2591,"num_objects_unfound":0,"num_objects_dirty":6434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4133,"num_write_kb":16532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1571,"num_bytes_recovered":6434816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2591},{"shards":"2","objects":3843}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6489","reported_seq":6615,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.304548+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:20.304548+0000","last_peered":"2024-09-15T21:22:20.304548+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:20.304548+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6489,"log_dups_size":0,"ondisk_log_size":6489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26570752,"num_objects":6487,"num_object_clones":0,"num_object_copies":12974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3433,"num_write_kb":13732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2393},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6463","reported_seq":6590,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.252863+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:20.252863+0000","last_peered":"2024-09-15T21:22:20.252863+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:20.252863+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6463,"log_dups_size":0,"ondisk_log_size":6463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26439680,"num_objects":6455,"num_object_clones":0,"num_object_copies":12910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3372,"num_write_kb":13488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2316},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6346","reported_seq":6474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.319476+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:20.319476+0000","last_peered":"2024-09-15T21:22:20.319476+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:20.319476+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6346,"log_dups_size":0,"ondisk_log_size":6346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25976832,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2830,"num_write_kb":11320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2260},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6444","reported_seq":6578,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.348371+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:20.348371+0000","last_peered":"2024-09-15T21:22:20.348371+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:20.348371+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6444,"log_dups_size":0,"ondisk_log_size":6444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26346438,"num_objects":6434,"num_object_clones":0,"num_object_copies":12868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6434,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3148,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2287},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":421548766,"num_objects":102926,"num_object_clones":0,"num_object_copies":205852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2591,"num_objects_unfound":0,"num_objects_dirty":102926,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":62791,"num_write_kb":251164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1572,"num_bytes_recovered":6438912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":852369408,"data_stored":852214049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102998,"ondisk_log_size":102998,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516520,"kb_used_data":336280,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93855320,"statfs":{"total":96636764160,"available":96107847680,"internally_reserved":0,"allocated":344350720,"data_stored":344088706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":184544937},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582136,"kb_used_data":375592,"kb_used_omap":4,"kb_used_meta":206523,"kb_avail":93789704,"statfs":{"total":96636764160,"available":96040656896,"internally_reserved":0,"allocated":384606208,"data_stored":384340117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":211480235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":267816,"kb_used_data":119512,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94104024,"statfs":{"total":96636764160,"available":96362520576,"internally_reserved":0,"allocated":122380288,"data_stored":122067228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":151842479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":385040384,"data_stored":385004254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122871808,"data_stored":122788709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344457216,"data_stored":344421086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:23.121 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:23.671 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:23.671 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:23.691 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-15T21:22:22.707829+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":421548785,"num_objects":102928,"num_object_clones":0,"num_object_copies":205856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2591,"num_objects_unfound":0,"num_objects_dirty":102928,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":62793,"num_write_kb":251166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1573,"num_bytes_recovered":6438931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":103001,"ondisk_log_size":103001,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1366472,"kb_used_data":831384,"kb_used_omap":12,"kb_used_meta":535027,"kb_avail":281749048,"statfs":{"total":289910292480,"available":288511025152,"internally_reserved":0,"allocated":851337216,"data_stored":850496051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":547867651},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,8,12,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21889024,"num_objects":5344,"num_object_clones":0,"num_object_copies":10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-273,"num_objects_unfound":0,"num_objects_dirty":5344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5344,"num_write_kb":21376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003206"},"pg_stats":[{"pgid":"2.f","version":"43'6565","reported_seq":6636,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.302407+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:20.302407+0000","last_peered":"2024-09-15T21:22:20.302407+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:20.302407+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6565,"log_dups_size":0,"ondisk_log_size":6565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26890240,"num_objects":6565,"num_object_clones":0,"num_object_copies":13130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6565,"num_write_kb":26260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2433},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6400","reported_seq":6500,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.169452+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:18.169452+0000","last_peered":"2024-09-15T21:22:18.169452+0000","last_clean":"2024-09-15T21:22:18.169452+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:18.169452+0000","last_undegraded":"2024-09-15T21:22:18.169452+0000","last_fullsized":"2024-09-15T21:22:18.169452+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6400,"log_dups_size":0,"ondisk_log_size":6400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26190790,"num_objects":6396,"num_object_clones":0,"num_object_copies":12792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6396,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4681,"num_write_kb":18724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6571","reported_seq":6690,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.235043+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:18.235043+0000","last_peered":"2024-09-15T21:22:18.235043+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:18.235043+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6571,"log_dups_size":0,"ondisk_log_size":6571,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26898432,"num_objects":6567,"num_object_clones":0,"num_object_copies":13134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4671,"num_write_kb":18684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2300}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6274","reported_seq":6339,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:18.242420+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:18.242420+0000","last_peered":"2024-09-15T21:22:18.242420+0000","last_clean":"2024-09-15T21:22:18.242420+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:18.242420+0000","last_undegraded":"2024-09-15T21:22:18.242420+0000","last_fullsized":"2024-09-15T21:22:18.242420+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6274,"log_dups_size":0,"ondisk_log_size":6274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25698304,"num_objects":6274,"num_object_clones":0,"num_object_copies":12548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6274,"num_write_kb":25096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6437","reported_seq":6565,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:20.335589+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:20.335589+0000","last_peered":"2024-09-15T21:22:20.335589+0000","last_clean":"2024-09-15T21:22:20.335589+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:20.335589+0000","last_undegraded":"2024-09-15T21:22:20.335589+0000","last_fullsized":"2024-09-15T21:22:20.335589+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6437,"log_dups_size":0,"ondisk_log_size":6437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26337763,"num_objects":6431,"num_object_clones":0,"num_object_copies":12862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2892,"num_write_kb":11568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6410","reported_seq":6541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.322672+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:20.322672+0000","last_peered":"2024-09-15T21:22:20.322672+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:20.322672+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6410,"log_dups_size":0,"ondisk_log_size":6410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26235363,"num_objects":6406,"num_object_clones":0,"num_object_copies":12812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3086,"num_write_kb":12344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2330},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6515","reported_seq":6646,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.208016+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:20.208016+0000","last_peered":"2024-09-15T21:22:20.208016+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:20.208016+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6515,"log_dups_size":0,"ondisk_log_size":6515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26661830,"num_objects":6511,"num_object_clones":0,"num_object_copies":13022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2826,"num_write_kb":11304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2350},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6594","reported_seq":6718,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:20.302413+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:20.302413+0000","last_peered":"2024-09-15T21:22:20.302413+0000","last_clean":"2024-09-15T21:22:20.302413+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:20.302413+0000","last_undegraded":"2024-09-15T21:22:20.302413+0000","last_fullsized":"2024-09-15T21:22:20.302413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6594,"log_dups_size":0,"ondisk_log_size":6594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27000832,"num_objects":6592,"num_object_clones":0,"num_object_copies":13184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3366,"num_write_kb":13464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6339","reported_seq":6460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.181885+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:18.181885+0000","last_peered":"2024-09-15T21:22:18.181885+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:18.181885+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6339,"log_dups_size":0,"ondisk_log_size":6339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25936355,"num_objects":6333,"num_object_clones":0,"num_object_copies":12666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6333,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4271,"num_write_kb":17084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2246}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352546+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352546+0000","last_peered":"2024-09-15T21:21:14.352546+0000","last_clean":"2024-09-15T21:21:14.352546+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352546+0000","last_undegraded":"2024-09-15T21:21:14.352546+0000","last_fullsized":"2024-09-15T21:21:14.352546+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6365","reported_seq":6491,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.314601+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:20.314601+0000","last_peered":"2024-09-15T21:22:20.314601+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:20.314601+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6365,"log_dups_size":0,"ondisk_log_size":6365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26038272,"num_objects":6357,"num_object_clones":0,"num_object_copies":12714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3038,"num_write_kb":12152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2337},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6346","reported_seq":6459,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:18.205877+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:18.205877+0000","last_peered":"2024-09-15T21:22:18.205877+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:18.205877+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6346,"log_dups_size":0,"ondisk_log_size":6346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25973219,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4205,"num_write_kb":16820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2252}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6440","reported_seq":9698,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:18.254920+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:18.254920+0000","last_peered":"2024-09-15T21:22:18.254920+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:18.254920+0000","last_undegraded":"2024-09-15T21:22:18.254920+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6440,"log_dups_size":0,"ondisk_log_size":6440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26353664,"num_objects":6434,"num_object_clones":0,"num_object_copies":12868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2591,"num_objects_unfound":0,"num_objects_dirty":6434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4133,"num_write_kb":16532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1571,"num_bytes_recovered":6434816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2591},{"shards":"2","objects":3843}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6489","reported_seq":6615,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.304548+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:20.304548+0000","last_peered":"2024-09-15T21:22:20.304548+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:20.304548+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6489,"log_dups_size":0,"ondisk_log_size":6489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26570752,"num_objects":6487,"num_object_clones":0,"num_object_copies":12974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3433,"num_write_kb":13732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2393},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6463","reported_seq":6590,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.252863+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:20.252863+0000","last_peered":"2024-09-15T21:22:20.252863+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:20.252863+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6463,"log_dups_size":0,"ondisk_log_size":6463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26439680,"num_objects":6455,"num_object_clones":0,"num_object_copies":12910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3372,"num_write_kb":13488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2316},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354625+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354625+0000","last_peered":"2024-09-15T21:21:14.354625+0000","last_clean":"2024-09-15T21:21:14.354625+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354625+0000","last_undegraded":"2024-09-15T21:21:14.354625+0000","last_fullsized":"2024-09-15T21:21:14.354625+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6346","reported_seq":6474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.319476+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:20.319476+0000","last_peered":"2024-09-15T21:22:20.319476+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:20.319476+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6346,"log_dups_size":0,"ondisk_log_size":6346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25976832,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2830,"num_write_kb":11320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2260},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6444","reported_seq":6578,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.348371+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:20.348371+0000","last_peered":"2024-09-15T21:22:20.348371+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:20.348371+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6444,"log_dups_size":0,"ondisk_log_size":6444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26346438,"num_objects":6434,"num_object_clones":0,"num_object_copies":12868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6434,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3148,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2287},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":421548766,"num_objects":102926,"num_object_clones":0,"num_object_copies":205852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2591,"num_objects_unfound":0,"num_objects_dirty":102926,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":62791,"num_write_kb":251164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1572,"num_bytes_recovered":6438912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":852369408,"data_stored":852214049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102998,"ondisk_log_size":102998,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516520,"kb_used_data":336280,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93855320,"statfs":{"total":96636764160,"available":96107847680,"internally_reserved":0,"allocated":344350720,"data_stored":344088706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":184544937},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582136,"kb_used_data":375592,"kb_used_omap":4,"kb_used_meta":206523,"kb_avail":93789704,"statfs":{"total":96636764160,"available":96040656896,"internally_reserved":0,"allocated":384606208,"data_stored":384340117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":211480235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":267816,"kb_used_data":119512,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94104024,"statfs":{"total":96636764160,"available":96362520576,"internally_reserved":0,"allocated":122380288,"data_stored":122067228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":151842479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":385040384,"data_stored":385004254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122871808,"data_stored":122788709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344457216,"data_stored":344421086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:23.693 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:23.693 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'6565", 'reported_seq': 6636, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:20.302407+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:22:20.302407+0000', 'last_peered': '2024-09-15T21:22:20.302407+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:22:20.302407+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 6565, 'log_dups_size': 0, 'ondisk_log_size': 6565, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26890240, 'num_objects': 6565, 'num_object_clones': 0, 'num_object_copies': 13130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6565, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6565, 'num_write_kb': 26260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2433}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:23.693 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:22:23.693 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'6571", 'reported_seq': 6690, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:18.235043+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:22:18.235043+0000', 'last_peered': '2024-09-15T21:22:18.235043+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:22:18.235043+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 6571, 'log_dups_size': 0, 'ondisk_log_size': 6571, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26898432, 'num_objects': 6567, 'num_object_clones': 0, 'num_object_copies': 13134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6567, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4671, 'num_write_kb': 18684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 2300}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:23.693 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:23.693 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'6410", 'reported_seq': 6541, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:20.322672+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:22:20.322672+0000', 'last_peered': '2024-09-15T21:22:20.322672+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:22:20.322672+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 6410, 'log_dups_size': 0, 'ondisk_log_size': 6410, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26235363, 'num_objects': 6406, 'num_object_clones': 0, 'num_object_copies': 12812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6406, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3086, 'num_write_kb': 12344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2330}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:23.693 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:23.693 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'6515", 'reported_seq': 6646, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:20.208016+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:22:20.208016+0000', 'last_peered': '2024-09-15T21:22:20.208016+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:22:20.208016+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 6515, 'log_dups_size': 0, 'ondisk_log_size': 6515, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26661830, 'num_objects': 6511, 'num_object_clones': 0, 'num_object_copies': 13022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6511, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2826, 'num_write_kb': 11304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2350}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:23.694 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:23.694 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'6339", 'reported_seq': 6460, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:18.181885+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:22:18.181885+0000', 'last_peered': '2024-09-15T21:22:18.181885+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:22:18.181885+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 6339, 'log_dups_size': 0, 'ondisk_log_size': 6339, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25936355, 'num_objects': 6333, 'num_object_clones': 0, 'num_object_copies': 12666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6333, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4271, 'num_write_kb': 17084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 2246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:23.694 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:23.694 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'6365", 'reported_seq': 6491, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:20.314601+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:22:20.314601+0000', 'last_peered': '2024-09-15T21:22:20.314601+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:22:20.314601+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 6365, 'log_dups_size': 0, 'ondisk_log_size': 6365, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26038272, 'num_objects': 6357, 'num_object_clones': 0, 'num_object_copies': 12714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6357, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3038, 'num_write_kb': 12152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2337}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:23.694 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:23.694 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'6346", 'reported_seq': 6459, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:18.205877+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:22:18.205877+0000', 'last_peered': '2024-09-15T21:22:18.205877+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:22:18.205877+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 6346, 'log_dups_size': 0, 'ondisk_log_size': 6346, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25973219, 'num_objects': 6342, 'num_object_clones': 0, 'num_object_copies': 12684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6342, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4205, 'num_write_kb': 16820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 2252}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:23.694 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:23.694 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'6440", 'reported_seq': 9698, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:22:18.254920+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:22:18.254920+0000', 'last_peered': '2024-09-15T21:22:18.254920+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:22:18.254920+0000', 'last_undegraded': '2024-09-15T21:22:18.254920+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 6440, 'log_dups_size': 0, 'ondisk_log_size': 6440, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26353664, 'num_objects': 6434, 'num_object_clones': 0, 'num_object_copies': 12868, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2591, 'num_objects_unfound': 0, 'num_objects_dirty': 6434, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4133, 'num_write_kb': 16532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1571, 'num_bytes_recovered': 6434816, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2591}, {'shards': '2', 'objects': 3843}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:23.694 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:23.694 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'6489", 'reported_seq': 6615, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:20.304548+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:22:20.304548+0000', 'last_peered': '2024-09-15T21:22:20.304548+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:22:20.304548+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 6489, 'log_dups_size': 0, 'ondisk_log_size': 6489, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26570752, 'num_objects': 6487, 'num_object_clones': 0, 'num_object_copies': 12974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6487, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3433, 'num_write_kb': 13732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2393}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:23.694 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:23.695 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'6463", 'reported_seq': 6590, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:20.252863+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:22:20.252863+0000', 'last_peered': '2024-09-15T21:22:20.252863+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:22:20.252863+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 6463, 'log_dups_size': 0, 'ondisk_log_size': 6463, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26439680, 'num_objects': 6455, 'num_object_clones': 0, 'num_object_copies': 12910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6455, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3372, 'num_write_kb': 13488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2316}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:23.695 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:23.695 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'6346", 'reported_seq': 6474, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:20.319476+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:22:20.319476+0000', 'last_peered': '2024-09-15T21:22:20.319476+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:22:20.319476+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 6346, 'log_dups_size': 0, 'ondisk_log_size': 6346, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25976832, 'num_objects': 6342, 'num_object_clones': 0, 'num_object_copies': 12684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6342, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2830, 'num_write_kb': 11320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2260}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:23.695 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:22:23.695 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'6444", 'reported_seq': 6578, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:20.348371+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:22:20.348371+0000', 'last_peered': '2024-09-15T21:22:20.348371+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:22:20.348371+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 6444, 'log_dups_size': 0, 'ondisk_log_size': 6444, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26346438, 'num_objects': 6434, 'num_object_clones': 0, 'num_object_copies': 12868, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6434, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3148, 'num_write_kb': 12592, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2287}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:23.695 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:23.745 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 112 20 105180 105160 3.66718 1.92969 0.0162996 0.0212958 2024-09-15T21:22:24.383 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:24.383 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/205856 objects degraded (22.010%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":196,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":102928,"data_bytes":421548785,"bytes_used":1399267328,"bytes_avail":288511025152,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":205856,"degraded_ratio":0.22010045857298305,"misplaced_objects":2591,"misplaced_total":205856,"misplaced_ratio":0.012586468210788121,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":93159,"recovering_keys_per_sec":0,"num_objects_recovered":273,"num_bytes_recovered":1118208,"num_keys_recovered":0,"write_bytes_sec":1823598,"read_op_per_sec":0,"write_op_per_sec":445},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:24.409 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:24.409 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:24.746 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 113 20 105661 105641 3.65135 1.87891 0.0263717 0.0213876 2024-09-15T21:22:24.942 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:24.943 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:24.977 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":134,"stamp":"2024-09-15T21:22:24.708278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":425685745,"num_objects":103938,"num_object_clones":0,"num_object_copies":207876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2456,"num_objects_unfound":0,"num_objects_dirty":103938,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":63803,"num_write_kb":255206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1708,"num_bytes_recovered":6991891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104011,"ondisk_log_size":104011,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1376184,"kb_used_data":836936,"kb_used_omap":12,"kb_used_meta":539187,"kb_avail":281739336,"statfs":{"total":289910292480,"available":288501080064,"internally_reserved":0,"allocated":857022464,"data_stored":856181299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":552127491},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,6,5,8,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26025984,"num_objects":6354,"num_object_clones":0,"num_object_copies":12708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-408,"num_objects_unfound":0,"num_objects_dirty":6354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6354,"num_write_kb":25416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":408,"num_bytes_recovered":1671168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002739"},"pg_stats":[{"pgid":"2.f","version":"43'6565","reported_seq":6636,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.302407+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:20.302407+0000","last_peered":"2024-09-15T21:22:20.302407+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:20.302407+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6565,"log_dups_size":0,"ondisk_log_size":6565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26890240,"num_objects":6565,"num_object_clones":0,"num_object_copies":13130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6565,"num_write_kb":26260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2433},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6584","reported_seq":6684,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:23.270156+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:23.270156+0000","last_peered":"2024-09-15T21:22:23.270156+0000","last_clean":"2024-09-15T21:22:23.270156+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:23.270156+0000","last_undegraded":"2024-09-15T21:22:23.270156+0000","last_fullsized":"2024-09-15T21:22:23.270156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6584,"log_dups_size":0,"ondisk_log_size":6584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26944454,"num_objects":6580,"num_object_clones":0,"num_object_copies":13160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6580,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4865,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6730","reported_seq":6850,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.140652+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:23.140652+0000","last_peered":"2024-09-15T21:22:23.140652+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:23.140652+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6730,"log_dups_size":0,"ondisk_log_size":6730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27549696,"num_objects":6726,"num_object_clones":0,"num_object_copies":13452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4830,"num_write_kb":19320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2459}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6437","reported_seq":6502,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:23.231848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:23.231848+0000","last_peered":"2024-09-15T21:22:23.231848+0000","last_clean":"2024-09-15T21:22:23.231848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:23.231848+0000","last_undegraded":"2024-09-15T21:22:23.231848+0000","last_fullsized":"2024-09-15T21:22:23.231848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6437,"log_dups_size":0,"ondisk_log_size":6437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6437,"num_write_kb":25748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6437","reported_seq":6565,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:20.335589+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:20.335589+0000","last_peered":"2024-09-15T21:22:20.335589+0000","last_clean":"2024-09-15T21:22:20.335589+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:20.335589+0000","last_undegraded":"2024-09-15T21:22:20.335589+0000","last_fullsized":"2024-09-15T21:22:20.335589+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6437,"log_dups_size":0,"ondisk_log_size":6437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26337763,"num_objects":6431,"num_object_clones":0,"num_object_copies":12862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2892,"num_write_kb":11568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6410","reported_seq":6541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.322672+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:20.322672+0000","last_peered":"2024-09-15T21:22:20.322672+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:20.322672+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6410,"log_dups_size":0,"ondisk_log_size":6410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26235363,"num_objects":6406,"num_object_clones":0,"num_object_copies":12812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6406,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3086,"num_write_kb":12344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2330},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6515","reported_seq":6646,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.208016+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:20.208016+0000","last_peered":"2024-09-15T21:22:20.208016+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:20.208016+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6515,"log_dups_size":0,"ondisk_log_size":6515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26661830,"num_objects":6511,"num_object_clones":0,"num_object_copies":13022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2826,"num_write_kb":11304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2350},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6594","reported_seq":6718,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:20.302413+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:20.302413+0000","last_peered":"2024-09-15T21:22:20.302413+0000","last_clean":"2024-09-15T21:22:20.302413+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:20.302413+0000","last_undegraded":"2024-09-15T21:22:20.302413+0000","last_fullsized":"2024-09-15T21:22:20.302413+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6594,"log_dups_size":0,"ondisk_log_size":6594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27000832,"num_objects":6592,"num_object_clones":0,"num_object_copies":13184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3366,"num_write_kb":13464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6502","reported_seq":6623,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.221867+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:23.221867+0000","last_peered":"2024-09-15T21:22:23.221867+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:23.221867+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26604003,"num_objects":6496,"num_object_clones":0,"num_object_copies":12992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6496,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4434,"num_write_kb":17736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2409}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6365","reported_seq":6491,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.314601+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:20.314601+0000","last_peered":"2024-09-15T21:22:20.314601+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:20.314601+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6365,"log_dups_size":0,"ondisk_log_size":6365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26038272,"num_objects":6357,"num_object_clones":0,"num_object_copies":12714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3038,"num_write_kb":12152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2337},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6510","reported_seq":6624,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.260321+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:23.260321+0000","last_peered":"2024-09-15T21:22:23.260321+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:23.260321+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6510,"log_dups_size":0,"ondisk_log_size":6510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26644963,"num_objects":6506,"num_object_clones":0,"num_object_copies":13012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6506,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4369,"num_write_kb":17476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2416}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6617","reported_seq":10147,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:23.275639+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:23.275639+0000","last_peered":"2024-09-15T21:22:23.275639+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:23.275639+0000","last_undegraded":"2024-09-15T21:22:23.275639+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6617,"log_dups_size":0,"ondisk_log_size":6617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27078656,"num_objects":6611,"num_object_clones":0,"num_object_copies":13222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2456,"num_objects_unfound":0,"num_objects_dirty":6611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4310,"num_write_kb":17240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1706,"num_bytes_recovered":6987776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2456},{"shards":"2","objects":4155}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6489","reported_seq":6615,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.304548+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:20.304548+0000","last_peered":"2024-09-15T21:22:20.304548+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:20.304548+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6489,"log_dups_size":0,"ondisk_log_size":6489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26570752,"num_objects":6487,"num_object_clones":0,"num_object_copies":12974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3433,"num_write_kb":13732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2393},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6463","reported_seq":6590,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.252863+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:20.252863+0000","last_peered":"2024-09-15T21:22:20.252863+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:20.252863+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6463,"log_dups_size":0,"ondisk_log_size":6463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26439680,"num_objects":6455,"num_object_clones":0,"num_object_copies":12910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3372,"num_write_kb":13488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2316},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6346","reported_seq":6474,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.319476+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:20.319476+0000","last_peered":"2024-09-15T21:22:20.319476+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:20.319476+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6346,"log_dups_size":0,"ondisk_log_size":6346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25976832,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2830,"num_write_kb":11320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2260},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6444","reported_seq":6578,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:20.348371+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:20.348371+0000","last_peered":"2024-09-15T21:22:20.348371+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:20.348371+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6444,"log_dups_size":0,"ondisk_log_size":6444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26346438,"num_objects":6434,"num_object_clones":0,"num_object_copies":12868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6434,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3148,"num_write_kb":12592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2287},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":425685726,"num_objects":103936,"num_object_clones":0,"num_object_copies":207872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2456,"num_objects_unfound":0,"num_objects_dirty":103936,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":63801,"num_write_kb":255204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1707,"num_bytes_recovered":6991872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":857960448,"data_stored":857805089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104008,"ondisk_log_size":104008,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526232,"kb_used_data":341832,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93845608,"statfs":{"total":96636764160,"available":96097902592,"internally_reserved":0,"allocated":350035968,"data_stored":349773954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":188804777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582136,"kb_used_data":375592,"kb_used_omap":4,"kb_used_meta":206523,"kb_avail":93789704,"statfs":{"total":96636764160,"available":96040656896,"internally_reserved":0,"allocated":384606208,"data_stored":384340117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":211480235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":267816,"kb_used_data":119512,"kb_used_omap":4,"kb_used_meta":148283,"kb_avail":94104024,"statfs":{"total":96636764160,"available":96362520576,"internally_reserved":0,"allocated":122380288,"data_stored":122067228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":151842479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":385040384,"data_stored":385004254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122871808,"data_stored":122788709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350048256,"data_stored":350012126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:25.745 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 114 20 106197 106177 3.63769 2.09375 0.0365451 0.0214692 2024-09-15T21:22:26.746 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 115 20 106752 106732 3.62491 2.16797 0.038799 0.0215453 2024-09-15T21:22:27.746 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 116 20 107305 107285 3.61228 2.16016 0.0207187 0.0216204 2024-09-15T21:22:27.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:28.545 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:28.546 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:28.573 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-15T21:22:26.708787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":432489201,"num_objects":105599,"num_object_clones":0,"num_object_copies":211198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2456,"num_objects_unfound":0,"num_objects_dirty":105599,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":65464,"num_write_kb":261850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1708,"num_bytes_recovered":6991891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105672,"ondisk_log_size":105672,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1367600,"kb_used_data":853312,"kb_used_omap":12,"kb_used_meta":514227,"kb_avail":281747920,"statfs":{"total":289910292480,"available":288509870080,"internally_reserved":0,"allocated":873791488,"data_stored":872950323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":526568451},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,6,6,3,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28897280,"num_objects":7055,"num_object_clones":0,"num_object_copies":14110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-271,"num_objects_unfound":0,"num_objects_dirty":7055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7055,"num_write_kb":28220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":1110016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002853"},"pg_stats":[{"pgid":"2.f","version":"43'6750","reported_seq":6821,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.376431+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:25.376431+0000","last_peered":"2024-09-15T21:22:25.376431+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:25.376431+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6750,"log_dups_size":0,"ondisk_log_size":6750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27648000,"num_objects":6750,"num_object_clones":0,"num_object_copies":13500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6750,"num_write_kb":27000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2618},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6584","reported_seq":6684,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:23.270156+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:23.270156+0000","last_peered":"2024-09-15T21:22:23.270156+0000","last_clean":"2024-09-15T21:22:23.270156+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:23.270156+0000","last_undegraded":"2024-09-15T21:22:23.270156+0000","last_fullsized":"2024-09-15T21:22:23.270156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6584,"log_dups_size":0,"ondisk_log_size":6584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26944454,"num_objects":6580,"num_object_clones":0,"num_object_copies":13160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6580,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4865,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6730","reported_seq":6850,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.140652+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:23.140652+0000","last_peered":"2024-09-15T21:22:23.140652+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:23.140652+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6730,"log_dups_size":0,"ondisk_log_size":6730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27549696,"num_objects":6726,"num_object_clones":0,"num_object_copies":13452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4830,"num_write_kb":19320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2459}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6437","reported_seq":6502,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:23.231848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:23.231848+0000","last_peered":"2024-09-15T21:22:23.231848+0000","last_clean":"2024-09-15T21:22:23.231848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:23.231848+0000","last_undegraded":"2024-09-15T21:22:23.231848+0000","last_fullsized":"2024-09-15T21:22:23.231848+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6437,"log_dups_size":0,"ondisk_log_size":6437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6437,"num_write_kb":25748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6602","reported_seq":6730,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:25.336256+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:25.336256+0000","last_peered":"2024-09-15T21:22:25.336256+0000","last_clean":"2024-09-15T21:22:25.336256+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:25.336256+0000","last_undegraded":"2024-09-15T21:22:25.336256+0000","last_fullsized":"2024-09-15T21:22:25.336256+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6602,"log_dups_size":0,"ondisk_log_size":6602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27013603,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3057,"num_write_kb":12228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6573","reported_seq":6704,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.312974+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:25.312974+0000","last_peered":"2024-09-15T21:22:25.312974+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:25.312974+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6573,"log_dups_size":0,"ondisk_log_size":6573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26903011,"num_objects":6569,"num_object_clones":0,"num_object_copies":13138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6569,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3249,"num_write_kb":12996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2493},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6679","reported_seq":6810,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.349113+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:25.349113+0000","last_peered":"2024-09-15T21:22:25.349113+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:25.349113+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6679,"log_dups_size":0,"ondisk_log_size":6679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27333574,"num_objects":6675,"num_object_clones":0,"num_object_copies":13350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2990,"num_write_kb":11960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2514},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6776","reported_seq":6900,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:25.333874+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:25.333874+0000","last_peered":"2024-09-15T21:22:25.333874+0000","last_clean":"2024-09-15T21:22:25.333874+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:25.333874+0000","last_undegraded":"2024-09-15T21:22:25.333874+0000","last_fullsized":"2024-09-15T21:22:25.333874+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6776,"log_dups_size":0,"ondisk_log_size":6776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27746304,"num_objects":6774,"num_object_clones":0,"num_object_copies":13548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3548,"num_write_kb":14192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6502","reported_seq":6623,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.221867+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:23.221867+0000","last_peered":"2024-09-15T21:22:23.221867+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:23.221867+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6502,"log_dups_size":0,"ondisk_log_size":6502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26604003,"num_objects":6496,"num_object_clones":0,"num_object_copies":12992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6496,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4434,"num_write_kb":17736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2409}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6496","reported_seq":6622,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.275136+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:25.275136+0000","last_peered":"2024-09-15T21:22:25.275136+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:25.275136+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6496,"log_dups_size":0,"ondisk_log_size":6496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26574848,"num_objects":6488,"num_object_clones":0,"num_object_copies":12976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3169,"num_write_kb":12676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2468},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6510","reported_seq":6624,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:23.260321+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:23.260321+0000","last_peered":"2024-09-15T21:22:23.260321+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:23.260321+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6510,"log_dups_size":0,"ondisk_log_size":6510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26644963,"num_objects":6506,"num_object_clones":0,"num_object_copies":13012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6506,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4369,"num_write_kb":17476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2416}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6617","reported_seq":10147,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:23.275639+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:23.275639+0000","last_peered":"2024-09-15T21:22:23.275639+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:23.275639+0000","last_undegraded":"2024-09-15T21:22:23.275639+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6617,"log_dups_size":0,"ondisk_log_size":6617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27078656,"num_objects":6611,"num_object_clones":0,"num_object_copies":13222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2456,"num_objects_unfound":0,"num_objects_dirty":6611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4310,"num_write_kb":17240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1706,"num_bytes_recovered":6987776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2456},{"shards":"2","objects":4155}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6637","reported_seq":6765,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.374487+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:25.374487+0000","last_peered":"2024-09-15T21:22:25.374487+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:25.374487+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6637,"log_dups_size":0,"ondisk_log_size":6637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27176960,"num_objects":6635,"num_object_clones":0,"num_object_copies":13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3581,"num_write_kb":14324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2541},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6657","reported_seq":6783,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.312823+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:25.312823+0000","last_peered":"2024-09-15T21:22:25.312823+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:25.312823+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6657,"log_dups_size":0,"ondisk_log_size":6657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27234304,"num_objects":6649,"num_object_clones":0,"num_object_copies":13298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3566,"num_write_kb":14264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2510},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6516","reported_seq":6646,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.265467+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:25.265467+0000","last_peered":"2024-09-15T21:22:25.265467+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:25.265467+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6516,"log_dups_size":0,"ondisk_log_size":6516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26673152,"num_objects":6512,"num_object_clones":0,"num_object_copies":13024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3000,"num_write_kb":12000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2430},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6603","reported_seq":6737,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.289147+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:25.289147+0000","last_peered":"2024-09-15T21:22:25.289147+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:25.289147+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6603,"log_dups_size":0,"ondisk_log_size":6603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26997702,"num_objects":6593,"num_object_clones":0,"num_object_copies":13186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6593,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3307,"num_write_kb":13228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2446},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":432489182,"num_objects":105597,"num_object_clones":0,"num_object_copies":211194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2456,"num_objects_unfound":0,"num_objects_dirty":105597,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":65462,"num_write_kb":261848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1707,"num_bytes_recovered":6991872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":874754048,"data_stored":874598689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105669,"ondisk_log_size":105669,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526232,"kb_used_data":341832,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93845608,"statfs":{"total":96636764160,"available":96097902592,"internally_reserved":0,"allocated":350035968,"data_stored":349773954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":188804777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577592,"kb_used_data":383528,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93794248,"statfs":{"total":96636764160,"available":96045309952,"internally_reserved":0,"allocated":392732672,"data_stored":392466581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198700715},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263776,"kb_used_data":127952,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94108064,"statfs":{"total":96636764160,"available":96366657536,"internally_reserved":0,"allocated":131022848,"data_stored":130709788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":139062959},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":393187328,"data_stored":393151198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131518464,"data_stored":131435365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350048256,"data_stored":350012126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:28.575 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:28.746 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 117 20 107804 107784 3.59807 1.94922 0.0637407 0.0217061 2024-09-15T21:22:29.113 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:29.113 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:29.145 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-15T21:22:28.709289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":436589297,"num_objects":106600,"num_object_clones":0,"num_object_copies":213200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2317,"num_objects_unfound":0,"num_objects_dirty":106600,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":66465,"num_write_kb":265854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1847,"num_bytes_recovered":7561235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106673,"ondisk_log_size":106673,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1360408,"kb_used_data":858600,"kb_used_omap":12,"kb_used_meta":501747,"kb_avail":281755112,"statfs":{"total":289910292480,"available":288517234688,"internally_reserved":0,"allocated":879206400,"data_stored":878365235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":513788931},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,6,6,12,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26124288,"num_objects":6378,"num_object_clones":0,"num_object_copies":12756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-410,"num_objects_unfound":0,"num_objects_dirty":6378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6378,"num_write_kb":25512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":1679360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002881"},"pg_stats":[{"pgid":"2.f","version":"43'6750","reported_seq":6821,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.376431+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:25.376431+0000","last_peered":"2024-09-15T21:22:25.376431+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:25.376431+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6750,"log_dups_size":0,"ondisk_log_size":6750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27648000,"num_objects":6750,"num_object_clones":0,"num_object_copies":13500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6750,"num_write_kb":27000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2618},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6742","reported_seq":6842,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.280987+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:28.280987+0000","last_peered":"2024-09-15T21:22:28.280987+0000","last_clean":"2024-09-15T21:22:28.280987+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:28.280987+0000","last_undegraded":"2024-09-15T21:22:28.280987+0000","last_fullsized":"2024-09-15T21:22:28.280987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27591622,"num_objects":6738,"num_object_clones":0,"num_object_copies":13476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6738,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5023,"num_write_kb":20092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6906","reported_seq":7026,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.208956+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:28.208956+0000","last_peered":"2024-09-15T21:22:28.208956+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:28.208956+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28270592,"num_objects":6902,"num_object_clones":0,"num_object_copies":13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5006,"num_write_kb":20024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2635}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6612","reported_seq":6677,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.283389+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:28.283389+0000","last_peered":"2024-09-15T21:22:28.283389+0000","last_clean":"2024-09-15T21:22:28.283389+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:28.283389+0000","last_undegraded":"2024-09-15T21:22:28.283389+0000","last_fullsized":"2024-09-15T21:22:28.283389+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6612,"log_dups_size":0,"ondisk_log_size":6612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27082752,"num_objects":6612,"num_object_clones":0,"num_object_copies":13224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6612,"num_write_kb":26448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6602","reported_seq":6730,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:25.336256+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:25.336256+0000","last_peered":"2024-09-15T21:22:25.336256+0000","last_clean":"2024-09-15T21:22:25.336256+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:25.336256+0000","last_undegraded":"2024-09-15T21:22:25.336256+0000","last_fullsized":"2024-09-15T21:22:25.336256+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6602,"log_dups_size":0,"ondisk_log_size":6602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27013603,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3057,"num_write_kb":12228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6573","reported_seq":6704,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.312974+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:25.312974+0000","last_peered":"2024-09-15T21:22:25.312974+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:25.312974+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6573,"log_dups_size":0,"ondisk_log_size":6573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26903011,"num_objects":6569,"num_object_clones":0,"num_object_copies":13138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6569,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3249,"num_write_kb":12996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2493},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6679","reported_seq":6810,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.349113+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:25.349113+0000","last_peered":"2024-09-15T21:22:25.349113+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:25.349113+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6679,"log_dups_size":0,"ondisk_log_size":6679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27333574,"num_objects":6675,"num_object_clones":0,"num_object_copies":13350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2990,"num_write_kb":11960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2514},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6776","reported_seq":6900,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:25.333874+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:25.333874+0000","last_peered":"2024-09-15T21:22:25.333874+0000","last_clean":"2024-09-15T21:22:25.333874+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:25.333874+0000","last_undegraded":"2024-09-15T21:22:25.333874+0000","last_fullsized":"2024-09-15T21:22:25.333874+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6776,"log_dups_size":0,"ondisk_log_size":6776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27746304,"num_objects":6774,"num_object_clones":0,"num_object_copies":13548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3548,"num_write_kb":14192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6670","reported_seq":6791,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.232320+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:28.232320+0000","last_peered":"2024-09-15T21:22:28.232320+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:28.232320+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6670,"log_dups_size":0,"ondisk_log_size":6670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27292131,"num_objects":6664,"num_object_clones":0,"num_object_copies":13328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4602,"num_write_kb":18408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2577}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6496","reported_seq":6622,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.275136+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:25.275136+0000","last_peered":"2024-09-15T21:22:25.275136+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:25.275136+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6496,"log_dups_size":0,"ondisk_log_size":6496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26574848,"num_objects":6488,"num_object_clones":0,"num_object_copies":12976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3169,"num_write_kb":12676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2468},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6678","reported_seq":6792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.253215+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:28.253215+0000","last_peered":"2024-09-15T21:22:28.253215+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:28.253215+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6678,"log_dups_size":0,"ondisk_log_size":6678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27333091,"num_objects":6674,"num_object_clones":0,"num_object_copies":13348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6674,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4537,"num_write_kb":18148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2584}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6773","reported_seq":10581,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:28.251012+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:28.251012+0000","last_peered":"2024-09-15T21:22:28.251012+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:28.251012+0000","last_undegraded":"2024-09-15T21:22:28.251012+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6773,"log_dups_size":0,"ondisk_log_size":6773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27717632,"num_objects":6767,"num_object_clones":0,"num_object_copies":13534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2317,"num_objects_unfound":0,"num_objects_dirty":6767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4466,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1845,"num_bytes_recovered":7557120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2317},{"shards":"2","objects":4450}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6637","reported_seq":6765,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.374487+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:25.374487+0000","last_peered":"2024-09-15T21:22:25.374487+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:25.374487+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6637,"log_dups_size":0,"ondisk_log_size":6637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27176960,"num_objects":6635,"num_object_clones":0,"num_object_copies":13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3581,"num_write_kb":14324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2541},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6657","reported_seq":6783,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.312823+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:25.312823+0000","last_peered":"2024-09-15T21:22:25.312823+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:25.312823+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6657,"log_dups_size":0,"ondisk_log_size":6657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27234304,"num_objects":6649,"num_object_clones":0,"num_object_copies":13298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3566,"num_write_kb":14264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2510},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6516","reported_seq":6646,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.265467+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:25.265467+0000","last_peered":"2024-09-15T21:22:25.265467+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:25.265467+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6516,"log_dups_size":0,"ondisk_log_size":6516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26673152,"num_objects":6512,"num_object_clones":0,"num_object_copies":13024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3000,"num_write_kb":12000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2430},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6603","reported_seq":6737,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.289147+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:25.289147+0000","last_peered":"2024-09-15T21:22:25.289147+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:25.289147+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6603,"log_dups_size":0,"ondisk_log_size":6603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26997702,"num_objects":6593,"num_object_clones":0,"num_object_copies":13186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6593,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3307,"num_write_kb":13228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2446},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":436589278,"num_objects":106598,"num_object_clones":0,"num_object_copies":213196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2317,"num_objects_unfound":0,"num_objects_dirty":106598,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":66463,"num_write_kb":265852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1846,"num_bytes_recovered":7561216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":880119808,"data_stored":879964449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106670,"ondisk_log_size":106670,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519040,"kb_used_data":347120,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93852800,"statfs":{"total":96636764160,"available":96105267200,"internally_reserved":0,"allocated":355450880,"data_stored":355188866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":176025257},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577592,"kb_used_data":383528,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93794248,"statfs":{"total":96636764160,"available":96045309952,"internally_reserved":0,"allocated":392732672,"data_stored":392466581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198700715},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263776,"kb_used_data":127952,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94108064,"statfs":{"total":96636764160,"available":96366657536,"internally_reserved":0,"allocated":131022848,"data_stored":130709788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":139062959},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":393187328,"data_stored":393151198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131518464,"data_stored":131435365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355414016,"data_stored":355377886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:29.147 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:29.147 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'6750", 'reported_seq': 6821, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:25.376431+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:22:25.376431+0000', 'last_peered': '2024-09-15T21:22:25.376431+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:22:25.376431+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 6750, 'log_dups_size': 0, 'ondisk_log_size': 6750, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27648000, 'num_objects': 6750, 'num_object_clones': 0, 'num_object_copies': 13500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6750, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6750, 'num_write_kb': 27000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2618}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:29.147 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:22:29.147 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'6906", 'reported_seq': 7026, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:28.208956+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:22:28.208956+0000', 'last_peered': '2024-09-15T21:22:28.208956+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:22:28.208956+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 6906, 'log_dups_size': 0, 'ondisk_log_size': 6906, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28270592, 'num_objects': 6902, 'num_object_clones': 0, 'num_object_copies': 13804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6902, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5006, 'num_write_kb': 20024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 2635}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:29.147 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:29.147 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'6573", 'reported_seq': 6704, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:25.312974+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:22:25.312974+0000', 'last_peered': '2024-09-15T21:22:25.312974+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:22:25.312974+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 6573, 'log_dups_size': 0, 'ondisk_log_size': 6573, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26903011, 'num_objects': 6569, 'num_object_clones': 0, 'num_object_copies': 13138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6569, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3249, 'num_write_kb': 12996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2493}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:29.147 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:29.147 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'6679", 'reported_seq': 6810, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:25.349113+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:22:25.349113+0000', 'last_peered': '2024-09-15T21:22:25.349113+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:22:25.349113+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 6679, 'log_dups_size': 0, 'ondisk_log_size': 6679, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27333574, 'num_objects': 6675, 'num_object_clones': 0, 'num_object_copies': 13350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6675, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2990, 'num_write_kb': 11960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2514}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:29.148 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:29.148 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'6670", 'reported_seq': 6791, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:28.232320+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:22:28.232320+0000', 'last_peered': '2024-09-15T21:22:28.232320+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:22:28.232320+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 6670, 'log_dups_size': 0, 'ondisk_log_size': 6670, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27292131, 'num_objects': 6664, 'num_object_clones': 0, 'num_object_copies': 13328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6664, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4602, 'num_write_kb': 18408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 2577}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:29.148 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:29.148 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'6496", 'reported_seq': 6622, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:25.275136+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:22:25.275136+0000', 'last_peered': '2024-09-15T21:22:25.275136+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:22:25.275136+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 6496, 'log_dups_size': 0, 'ondisk_log_size': 6496, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26574848, 'num_objects': 6488, 'num_object_clones': 0, 'num_object_copies': 12976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6488, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3169, 'num_write_kb': 12676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2468}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:29.148 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:29.148 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'6678", 'reported_seq': 6792, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:28.253215+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:22:28.253215+0000', 'last_peered': '2024-09-15T21:22:28.253215+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:22:28.253215+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 6678, 'log_dups_size': 0, 'ondisk_log_size': 6678, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27333091, 'num_objects': 6674, 'num_object_clones': 0, 'num_object_copies': 13348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6674, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4537, 'num_write_kb': 18148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 2584}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:29.148 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:29.148 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'6773", 'reported_seq': 10581, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:22:28.251012+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:22:28.251012+0000', 'last_peered': '2024-09-15T21:22:28.251012+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:22:28.251012+0000', 'last_undegraded': '2024-09-15T21:22:28.251012+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 6773, 'log_dups_size': 0, 'ondisk_log_size': 6773, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27717632, 'num_objects': 6767, 'num_object_clones': 0, 'num_object_copies': 13534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2317, 'num_objects_unfound': 0, 'num_objects_dirty': 6767, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4466, 'num_write_kb': 17864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1845, 'num_bytes_recovered': 7557120, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2317}, {'shards': '2', 'objects': 4450}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:29.148 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:29.148 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'6637", 'reported_seq': 6765, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:25.374487+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:22:25.374487+0000', 'last_peered': '2024-09-15T21:22:25.374487+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:22:25.374487+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 6637, 'log_dups_size': 0, 'ondisk_log_size': 6637, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27176960, 'num_objects': 6635, 'num_object_clones': 0, 'num_object_copies': 13270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6635, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3581, 'num_write_kb': 14324, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2541}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:29.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:29.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'6657", 'reported_seq': 6783, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:25.312823+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:22:25.312823+0000', 'last_peered': '2024-09-15T21:22:25.312823+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:22:25.312823+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 6657, 'log_dups_size': 0, 'ondisk_log_size': 6657, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27234304, 'num_objects': 6649, 'num_object_clones': 0, 'num_object_copies': 13298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6649, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3566, 'num_write_kb': 14264, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2510}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:29.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:29.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'6516", 'reported_seq': 6646, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:25.265467+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:22:25.265467+0000', 'last_peered': '2024-09-15T21:22:25.265467+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:22:25.265467+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 6516, 'log_dups_size': 0, 'ondisk_log_size': 6516, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26673152, 'num_objects': 6512, 'num_object_clones': 0, 'num_object_copies': 13024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6512, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3000, 'num_write_kb': 12000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2430}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:29.149 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:22:29.149 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'6603", 'reported_seq': 6737, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:25.289147+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:22:25.289147+0000', 'last_peered': '2024-09-15T21:22:25.289147+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:22:25.289147+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 6603, 'log_dups_size': 0, 'ondisk_log_size': 6603, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26997702, 'num_objects': 6593, 'num_object_clones': 0, 'num_object_copies': 13186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6593, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3307, 'num_write_kb': 13228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2446}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:29.149 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:29.746 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 118 20 108272 108252 3.58307 1.82812 0.0252546 0.0217952 2024-09-15T21:22:29.824 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:29.825 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/213200 objects degraded (21.252%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":201,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":106600,"data_bytes":436589297,"bytes_used":1393057792,"bytes_avail":288517234688,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":213200,"degraded_ratio":0.21251876172607878,"misplaced_objects":2317,"misplaced_total":213200,"misplaced_ratio":0.010867729831144465,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":139913,"recovering_keys_per_sec":0,"num_objects_recovered":410,"num_bytes_recovered":1679360,"num_keys_recovered":0,"write_bytes_sec":2176501,"read_op_per_sec":0,"write_op_per_sec":531},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:29.852 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:29.852 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:30.425 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:30.425 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:30.446 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-15T21:22:28.709289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":436589297,"num_objects":106600,"num_object_clones":0,"num_object_copies":213200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2317,"num_objects_unfound":0,"num_objects_dirty":106600,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":66465,"num_write_kb":265854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1847,"num_bytes_recovered":7561235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106673,"ondisk_log_size":106673,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1360408,"kb_used_data":858600,"kb_used_omap":12,"kb_used_meta":501747,"kb_avail":281755112,"statfs":{"total":289910292480,"available":288517234688,"internally_reserved":0,"allocated":879206400,"data_stored":878365235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":513788931},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,6,6,12,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26124288,"num_objects":6378,"num_object_clones":0,"num_object_copies":12756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-410,"num_objects_unfound":0,"num_objects_dirty":6378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6378,"num_write_kb":25512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":1679360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002881"},"pg_stats":[{"pgid":"2.f","version":"43'6750","reported_seq":6821,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.376431+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:25.376431+0000","last_peered":"2024-09-15T21:22:25.376431+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:25.376431+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6750,"log_dups_size":0,"ondisk_log_size":6750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27648000,"num_objects":6750,"num_object_clones":0,"num_object_copies":13500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6750,"num_write_kb":27000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2618},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6742","reported_seq":6842,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.280987+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:28.280987+0000","last_peered":"2024-09-15T21:22:28.280987+0000","last_clean":"2024-09-15T21:22:28.280987+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:28.280987+0000","last_undegraded":"2024-09-15T21:22:28.280987+0000","last_fullsized":"2024-09-15T21:22:28.280987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27591622,"num_objects":6738,"num_object_clones":0,"num_object_copies":13476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6738,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5023,"num_write_kb":20092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6906","reported_seq":7026,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.208956+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:28.208956+0000","last_peered":"2024-09-15T21:22:28.208956+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:28.208956+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28270592,"num_objects":6902,"num_object_clones":0,"num_object_copies":13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5006,"num_write_kb":20024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2635}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6612","reported_seq":6677,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.283389+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:28.283389+0000","last_peered":"2024-09-15T21:22:28.283389+0000","last_clean":"2024-09-15T21:22:28.283389+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:28.283389+0000","last_undegraded":"2024-09-15T21:22:28.283389+0000","last_fullsized":"2024-09-15T21:22:28.283389+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6612,"log_dups_size":0,"ondisk_log_size":6612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27082752,"num_objects":6612,"num_object_clones":0,"num_object_copies":13224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6612,"num_write_kb":26448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6602","reported_seq":6730,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:25.336256+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:25.336256+0000","last_peered":"2024-09-15T21:22:25.336256+0000","last_clean":"2024-09-15T21:22:25.336256+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:25.336256+0000","last_undegraded":"2024-09-15T21:22:25.336256+0000","last_fullsized":"2024-09-15T21:22:25.336256+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6602,"log_dups_size":0,"ondisk_log_size":6602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27013603,"num_objects":6596,"num_object_clones":0,"num_object_copies":13192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3057,"num_write_kb":12228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6573","reported_seq":6704,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.312974+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:25.312974+0000","last_peered":"2024-09-15T21:22:25.312974+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:25.312974+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6573,"log_dups_size":0,"ondisk_log_size":6573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26903011,"num_objects":6569,"num_object_clones":0,"num_object_copies":13138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6569,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3249,"num_write_kb":12996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2493},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6679","reported_seq":6810,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.349113+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:25.349113+0000","last_peered":"2024-09-15T21:22:25.349113+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:25.349113+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6679,"log_dups_size":0,"ondisk_log_size":6679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27333574,"num_objects":6675,"num_object_clones":0,"num_object_copies":13350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2990,"num_write_kb":11960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2514},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6776","reported_seq":6900,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:25.333874+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:25.333874+0000","last_peered":"2024-09-15T21:22:25.333874+0000","last_clean":"2024-09-15T21:22:25.333874+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:25.333874+0000","last_undegraded":"2024-09-15T21:22:25.333874+0000","last_fullsized":"2024-09-15T21:22:25.333874+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6776,"log_dups_size":0,"ondisk_log_size":6776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27746304,"num_objects":6774,"num_object_clones":0,"num_object_copies":13548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3548,"num_write_kb":14192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6670","reported_seq":6791,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.232320+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:28.232320+0000","last_peered":"2024-09-15T21:22:28.232320+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:28.232320+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6670,"log_dups_size":0,"ondisk_log_size":6670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27292131,"num_objects":6664,"num_object_clones":0,"num_object_copies":13328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4602,"num_write_kb":18408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2577}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6496","reported_seq":6622,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.275136+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:25.275136+0000","last_peered":"2024-09-15T21:22:25.275136+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:25.275136+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6496,"log_dups_size":0,"ondisk_log_size":6496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26574848,"num_objects":6488,"num_object_clones":0,"num_object_copies":12976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3169,"num_write_kb":12676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2468},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6678","reported_seq":6792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.253215+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:28.253215+0000","last_peered":"2024-09-15T21:22:28.253215+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:28.253215+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6678,"log_dups_size":0,"ondisk_log_size":6678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27333091,"num_objects":6674,"num_object_clones":0,"num_object_copies":13348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6674,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4537,"num_write_kb":18148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2584}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6773","reported_seq":10581,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:28.251012+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:28.251012+0000","last_peered":"2024-09-15T21:22:28.251012+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:28.251012+0000","last_undegraded":"2024-09-15T21:22:28.251012+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6773,"log_dups_size":0,"ondisk_log_size":6773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27717632,"num_objects":6767,"num_object_clones":0,"num_object_copies":13534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2317,"num_objects_unfound":0,"num_objects_dirty":6767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4466,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1845,"num_bytes_recovered":7557120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2317},{"shards":"2","objects":4450}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6637","reported_seq":6765,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.374487+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:25.374487+0000","last_peered":"2024-09-15T21:22:25.374487+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:25.374487+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6637,"log_dups_size":0,"ondisk_log_size":6637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27176960,"num_objects":6635,"num_object_clones":0,"num_object_copies":13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3581,"num_write_kb":14324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2541},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6657","reported_seq":6783,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.312823+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:25.312823+0000","last_peered":"2024-09-15T21:22:25.312823+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:25.312823+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6657,"log_dups_size":0,"ondisk_log_size":6657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27234304,"num_objects":6649,"num_object_clones":0,"num_object_copies":13298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3566,"num_write_kb":14264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2510},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6516","reported_seq":6646,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.265467+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:25.265467+0000","last_peered":"2024-09-15T21:22:25.265467+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:25.265467+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6516,"log_dups_size":0,"ondisk_log_size":6516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26673152,"num_objects":6512,"num_object_clones":0,"num_object_copies":13024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3000,"num_write_kb":12000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2430},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6603","reported_seq":6737,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:25.289147+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:25.289147+0000","last_peered":"2024-09-15T21:22:25.289147+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:25.289147+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6603,"log_dups_size":0,"ondisk_log_size":6603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26997702,"num_objects":6593,"num_object_clones":0,"num_object_copies":13186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6593,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3307,"num_write_kb":13228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2446},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":436589278,"num_objects":106598,"num_object_clones":0,"num_object_copies":213196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2317,"num_objects_unfound":0,"num_objects_dirty":106598,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":66463,"num_write_kb":265852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1846,"num_bytes_recovered":7561216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":880119808,"data_stored":879964449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":106670,"ondisk_log_size":106670,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519040,"kb_used_data":347120,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93852800,"statfs":{"total":96636764160,"available":96105267200,"internally_reserved":0,"allocated":355450880,"data_stored":355188866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":176025257},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577592,"kb_used_data":383528,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93794248,"statfs":{"total":96636764160,"available":96045309952,"internally_reserved":0,"allocated":392732672,"data_stored":392466581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198700715},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,2,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263776,"kb_used_data":127952,"kb_used_omap":4,"kb_used_meta":135803,"kb_avail":94108064,"statfs":{"total":96636764160,"available":96366657536,"internally_reserved":0,"allocated":131022848,"data_stored":130709788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":139062959},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,3,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":393187328,"data_stored":393151198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131518464,"data_stored":131435365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355414016,"data_stored":355377886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:30.746 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 119 20 108768 108748 3.56924 1.9375 0.0569005 0.0218816 2024-09-15T21:22:31.746 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:2024-09-15T21:22:31.744183+0000 min lat: 0.00235499 max lat: 2.60418 avg lat: 0.0219557 2024-09-15T21:22:31.746 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:22:31.746 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 120 20 109306 109286 3.55701 2.10156 0.0540178 0.0219557 2024-09-15T21:22:32.746 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 121 20 109846 109826 3.54504 2.10938 0.0263745 0.0220306 2024-09-15T21:22:33.449 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:33.746 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 122 20 110376 110356 3.53295 2.07031 0.0441735 0.0221054 2024-09-15T21:22:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:34.015 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:34.043 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-15T21:22:32.710214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":443106033,"num_objects":108191,"num_object_clones":0,"num_object_copies":216382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2317,"num_objects_unfound":0,"num_objects_dirty":108191,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":68056,"num_write_kb":272218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1847,"num_bytes_recovered":7561235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108264,"ondisk_log_size":108264,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1439024,"kb_used_data":874752,"kb_used_omap":12,"kb_used_meta":564211,"kb_avail":281676496,"statfs":{"total":289910292480,"available":288436731904,"internally_reserved":0,"allocated":895746048,"data_stored":894904883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":577752067},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3,11,14,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21557248,"num_objects":5263,"num_object_clones":0,"num_object_copies":10526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":5263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5263,"num_write_kb":21052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002850"},"pg_stats":[{"pgid":"2.f","version":"43'6927","reported_seq":6998,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.338782+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:30.338782+0000","last_peered":"2024-09-15T21:22:30.338782+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:30.338782+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6927,"log_dups_size":0,"ondisk_log_size":6927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28372992,"num_objects":6927,"num_object_clones":0,"num_object_copies":13854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6927,"num_write_kb":27708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2795},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6742","reported_seq":6842,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.280987+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:28.280987+0000","last_peered":"2024-09-15T21:22:28.280987+0000","last_clean":"2024-09-15T21:22:28.280987+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:28.280987+0000","last_undegraded":"2024-09-15T21:22:28.280987+0000","last_fullsized":"2024-09-15T21:22:28.280987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27591622,"num_objects":6738,"num_object_clones":0,"num_object_copies":13476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6738,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5023,"num_write_kb":20092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6906","reported_seq":7026,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.208956+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:28.208956+0000","last_peered":"2024-09-15T21:22:28.208956+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:28.208956+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28270592,"num_objects":6902,"num_object_clones":0,"num_object_copies":13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5006,"num_write_kb":20024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2635}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6612","reported_seq":6677,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.283389+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:28.283389+0000","last_peered":"2024-09-15T21:22:28.283389+0000","last_clean":"2024-09-15T21:22:28.283389+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:28.283389+0000","last_undegraded":"2024-09-15T21:22:28.283389+0000","last_fullsized":"2024-09-15T21:22:28.283389+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6612,"log_dups_size":0,"ondisk_log_size":6612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27082752,"num_objects":6612,"num_object_clones":0,"num_object_copies":13224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6612,"num_write_kb":26448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6763","reported_seq":6891,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:30.360772+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:30.360772+0000","last_peered":"2024-09-15T21:22:30.360772+0000","last_clean":"2024-09-15T21:22:30.360772+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:30.360772+0000","last_undegraded":"2024-09-15T21:22:30.360772+0000","last_fullsized":"2024-09-15T21:22:30.360772+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6763,"log_dups_size":0,"ondisk_log_size":6763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27673059,"num_objects":6757,"num_object_clones":0,"num_object_copies":13514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3218,"num_write_kb":12872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6742","reported_seq":6872,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.408240+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:30.408240+0000","last_peered":"2024-09-15T21:22:30.408240+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:30.408240+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27595235,"num_objects":6738,"num_object_clones":0,"num_object_copies":13476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3418,"num_write_kb":13672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2662},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6852","reported_seq":6983,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.259516+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:30.259516+0000","last_peered":"2024-09-15T21:22:30.259516+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:30.259516+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6852,"log_dups_size":0,"ondisk_log_size":6852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28042182,"num_objects":6848,"num_object_clones":0,"num_object_copies":13696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3163,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2687},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6919","reported_seq":7043,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:30.319548+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:30.319548+0000","last_peered":"2024-09-15T21:22:30.319548+0000","last_clean":"2024-09-15T21:22:30.319548+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:30.319548+0000","last_undegraded":"2024-09-15T21:22:30.319548+0000","last_fullsized":"2024-09-15T21:22:30.319548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6919,"log_dups_size":0,"ondisk_log_size":6919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28332032,"num_objects":6917,"num_object_clones":0,"num_object_copies":13834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3691,"num_write_kb":14764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6670","reported_seq":6791,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.232320+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:28.232320+0000","last_peered":"2024-09-15T21:22:28.232320+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:28.232320+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6670,"log_dups_size":0,"ondisk_log_size":6670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27292131,"num_objects":6664,"num_object_clones":0,"num_object_copies":13328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4602,"num_write_kb":18408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2577}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6652","reported_seq":6778,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.413117+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:30.413117+0000","last_peered":"2024-09-15T21:22:30.413117+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:30.413117+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6652,"log_dups_size":0,"ondisk_log_size":6652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27213824,"num_objects":6644,"num_object_clones":0,"num_object_copies":13288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3325,"num_write_kb":13300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2624},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6678","reported_seq":6792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.253215+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:28.253215+0000","last_peered":"2024-09-15T21:22:28.253215+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:28.253215+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6678,"log_dups_size":0,"ondisk_log_size":6678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27333091,"num_objects":6674,"num_object_clones":0,"num_object_copies":13348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6674,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4537,"num_write_kb":18148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2584}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6773","reported_seq":10581,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:28.251012+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:28.251012+0000","last_peered":"2024-09-15T21:22:28.251012+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:28.251012+0000","last_undegraded":"2024-09-15T21:22:28.251012+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6773,"log_dups_size":0,"ondisk_log_size":6773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27717632,"num_objects":6767,"num_object_clones":0,"num_object_copies":13534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2317,"num_objects_unfound":0,"num_objects_dirty":6767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4466,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1845,"num_bytes_recovered":7557120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2317},{"shards":"2","objects":4450}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6793","reported_seq":6922,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.375639+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:30.375639+0000","last_peered":"2024-09-15T21:22:30.375639+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:30.375639+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6793,"log_dups_size":0,"ondisk_log_size":6793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27815936,"num_objects":6791,"num_object_clones":0,"num_object_copies":13582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3737,"num_write_kb":14948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2697},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6813","reported_seq":6941,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.343129+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:30.343129+0000","last_peered":"2024-09-15T21:22:30.343129+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:30.343129+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6813,"log_dups_size":0,"ondisk_log_size":6813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27873280,"num_objects":6805,"num_object_clones":0,"num_object_copies":13610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3722,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2666},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6661","reported_seq":6792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.355991+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:30.355991+0000","last_peered":"2024-09-15T21:22:30.355991+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:30.355991+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6661,"log_dups_size":0,"ondisk_log_size":6661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27267072,"num_objects":6657,"num_object_clones":0,"num_object_copies":13314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3145,"num_write_kb":12580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2575},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6758","reported_seq":6892,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.339837+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:30.339837+0000","last_peered":"2024-09-15T21:22:30.339837+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:30.339837+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6758,"log_dups_size":0,"ondisk_log_size":6758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27632582,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3462,"num_write_kb":13848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2601},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":443106014,"num_objects":108189,"num_object_clones":0,"num_object_copies":216378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2317,"num_objects_unfound":0,"num_objects_dirty":108189,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":68054,"num_write_kb":272216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1846,"num_bytes_recovered":7561216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":896724992,"data_stored":896569633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108261,"ondisk_log_size":108261,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519040,"kb_used_data":347120,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93852800,"statfs":{"total":96636764160,"available":96105267200,"internally_reserved":0,"allocated":355450880,"data_stored":355188866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":176025257},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643760,"kb_used_data":391392,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93728080,"statfs":{"total":96636764160,"available":95977553920,"internally_reserved":0,"allocated":400785408,"data_stored":400519317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258404011},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":276224,"kb_used_data":136240,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94095616,"statfs":{"total":96636764160,"available":96353910784,"internally_reserved":0,"allocated":139509760,"data_stored":139196700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,6,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401113088,"data_stored":401076958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":140197888,"data_stored":140114789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355414016,"data_stored":355377886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:34.044 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:34.583 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:34.583 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:34.607 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-15T21:22:32.710214+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":443106033,"num_objects":108191,"num_object_clones":0,"num_object_copies":216382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2317,"num_objects_unfound":0,"num_objects_dirty":108191,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":68056,"num_write_kb":272218,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1847,"num_bytes_recovered":7561235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108264,"ondisk_log_size":108264,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1439024,"kb_used_data":874752,"kb_used_omap":12,"kb_used_meta":564211,"kb_avail":281676496,"statfs":{"total":289910292480,"available":288436731904,"internally_reserved":0,"allocated":895746048,"data_stored":894904883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":577752067},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3,11,14,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21557248,"num_objects":5263,"num_object_clones":0,"num_object_copies":10526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":5263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5263,"num_write_kb":21052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002850"},"pg_stats":[{"pgid":"2.f","version":"43'6927","reported_seq":6998,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.338782+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:30.338782+0000","last_peered":"2024-09-15T21:22:30.338782+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:30.338782+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6927,"log_dups_size":0,"ondisk_log_size":6927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28372992,"num_objects":6927,"num_object_clones":0,"num_object_copies":13854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6927,"num_write_kb":27708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2795},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6742","reported_seq":6842,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.280987+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:28.280987+0000","last_peered":"2024-09-15T21:22:28.280987+0000","last_clean":"2024-09-15T21:22:28.280987+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:28.280987+0000","last_undegraded":"2024-09-15T21:22:28.280987+0000","last_fullsized":"2024-09-15T21:22:28.280987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27591622,"num_objects":6738,"num_object_clones":0,"num_object_copies":13476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6738,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5023,"num_write_kb":20092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6906","reported_seq":7026,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.208956+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:28.208956+0000","last_peered":"2024-09-15T21:22:28.208956+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:28.208956+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":6906,"log_dups_size":0,"ondisk_log_size":6906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28270592,"num_objects":6902,"num_object_clones":0,"num_object_copies":13804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5006,"num_write_kb":20024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2635}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6612","reported_seq":6677,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:28.283389+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:28.283389+0000","last_peered":"2024-09-15T21:22:28.283389+0000","last_clean":"2024-09-15T21:22:28.283389+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:28.283389+0000","last_undegraded":"2024-09-15T21:22:28.283389+0000","last_fullsized":"2024-09-15T21:22:28.283389+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6612,"log_dups_size":0,"ondisk_log_size":6612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27082752,"num_objects":6612,"num_object_clones":0,"num_object_copies":13224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6612,"num_write_kb":26448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6763","reported_seq":6891,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:30.360772+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:30.360772+0000","last_peered":"2024-09-15T21:22:30.360772+0000","last_clean":"2024-09-15T21:22:30.360772+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:30.360772+0000","last_undegraded":"2024-09-15T21:22:30.360772+0000","last_fullsized":"2024-09-15T21:22:30.360772+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6763,"log_dups_size":0,"ondisk_log_size":6763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27673059,"num_objects":6757,"num_object_clones":0,"num_object_copies":13514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3218,"num_write_kb":12872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6742","reported_seq":6872,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.408240+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:30.408240+0000","last_peered":"2024-09-15T21:22:30.408240+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:30.408240+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27595235,"num_objects":6738,"num_object_clones":0,"num_object_copies":13476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3418,"num_write_kb":13672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2662},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6852","reported_seq":6983,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.259516+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:30.259516+0000","last_peered":"2024-09-15T21:22:30.259516+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:30.259516+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6852,"log_dups_size":0,"ondisk_log_size":6852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28042182,"num_objects":6848,"num_object_clones":0,"num_object_copies":13696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3163,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2687},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6919","reported_seq":7043,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:30.319548+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:30.319548+0000","last_peered":"2024-09-15T21:22:30.319548+0000","last_clean":"2024-09-15T21:22:30.319548+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:30.319548+0000","last_undegraded":"2024-09-15T21:22:30.319548+0000","last_fullsized":"2024-09-15T21:22:30.319548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6919,"log_dups_size":0,"ondisk_log_size":6919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28332032,"num_objects":6917,"num_object_clones":0,"num_object_copies":13834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3691,"num_write_kb":14764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6670","reported_seq":6791,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.232320+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:28.232320+0000","last_peered":"2024-09-15T21:22:28.232320+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:28.232320+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6670,"log_dups_size":0,"ondisk_log_size":6670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27292131,"num_objects":6664,"num_object_clones":0,"num_object_copies":13328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6664,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4602,"num_write_kb":18408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2577}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6652","reported_seq":6778,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.413117+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:30.413117+0000","last_peered":"2024-09-15T21:22:30.413117+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:30.413117+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6652,"log_dups_size":0,"ondisk_log_size":6652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27213824,"num_objects":6644,"num_object_clones":0,"num_object_copies":13288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3325,"num_write_kb":13300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2624},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6678","reported_seq":6792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:28.253215+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:28.253215+0000","last_peered":"2024-09-15T21:22:28.253215+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:28.253215+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6678,"log_dups_size":0,"ondisk_log_size":6678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27333091,"num_objects":6674,"num_object_clones":0,"num_object_copies":13348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6674,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4537,"num_write_kb":18148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2584}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6773","reported_seq":10581,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:28.251012+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:28.251012+0000","last_peered":"2024-09-15T21:22:28.251012+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:28.251012+0000","last_undegraded":"2024-09-15T21:22:28.251012+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6773,"log_dups_size":0,"ondisk_log_size":6773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27717632,"num_objects":6767,"num_object_clones":0,"num_object_copies":13534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2317,"num_objects_unfound":0,"num_objects_dirty":6767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4466,"num_write_kb":17864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1845,"num_bytes_recovered":7557120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2317},{"shards":"2","objects":4450}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6793","reported_seq":6922,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.375639+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:30.375639+0000","last_peered":"2024-09-15T21:22:30.375639+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:30.375639+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6793,"log_dups_size":0,"ondisk_log_size":6793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27815936,"num_objects":6791,"num_object_clones":0,"num_object_copies":13582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3737,"num_write_kb":14948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2697},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6813","reported_seq":6941,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.343129+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:30.343129+0000","last_peered":"2024-09-15T21:22:30.343129+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:30.343129+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6813,"log_dups_size":0,"ondisk_log_size":6813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27873280,"num_objects":6805,"num_object_clones":0,"num_object_copies":13610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3722,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2666},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6661","reported_seq":6792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.355991+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:30.355991+0000","last_peered":"2024-09-15T21:22:30.355991+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:30.355991+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6661,"log_dups_size":0,"ondisk_log_size":6661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27267072,"num_objects":6657,"num_object_clones":0,"num_object_copies":13314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3145,"num_write_kb":12580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2575},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6758","reported_seq":6892,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.339837+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:30.339837+0000","last_peered":"2024-09-15T21:22:30.339837+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:30.339837+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6758,"log_dups_size":0,"ondisk_log_size":6758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27632582,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3462,"num_write_kb":13848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2601},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":443106014,"num_objects":108189,"num_object_clones":0,"num_object_copies":216378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2317,"num_objects_unfound":0,"num_objects_dirty":108189,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":68054,"num_write_kb":272216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1846,"num_bytes_recovered":7561216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":896724992,"data_stored":896569633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":108261,"ondisk_log_size":108261,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519040,"kb_used_data":347120,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93852800,"statfs":{"total":96636764160,"available":96105267200,"internally_reserved":0,"allocated":355450880,"data_stored":355188866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":176025257},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643760,"kb_used_data":391392,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93728080,"statfs":{"total":96636764160,"available":95977553920,"internally_reserved":0,"allocated":400785408,"data_stored":400519317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258404011},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":276224,"kb_used_data":136240,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94095616,"statfs":{"total":96636764160,"available":96353910784,"internally_reserved":0,"allocated":139509760,"data_stored":139196700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,6,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401113088,"data_stored":401076958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":140197888,"data_stored":140114789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355414016,"data_stored":355377886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:34.609 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:34.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'6927", 'reported_seq': 6998, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:30.338782+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:22:30.338782+0000', 'last_peered': '2024-09-15T21:22:30.338782+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:22:30.338782+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 6927, 'log_dups_size': 0, 'ondisk_log_size': 6927, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28372992, 'num_objects': 6927, 'num_object_clones': 0, 'num_object_copies': 13854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6927, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6927, 'num_write_kb': 27708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2795}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:34.610 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:22:34.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'6906", 'reported_seq': 7026, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:28.208956+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:22:28.208956+0000', 'last_peered': '2024-09-15T21:22:28.208956+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:22:28.208956+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 6906, 'log_dups_size': 0, 'ondisk_log_size': 6906, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28270592, 'num_objects': 6902, 'num_object_clones': 0, 'num_object_copies': 13804, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6902, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5006, 'num_write_kb': 20024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 2635}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:34.610 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:34.610 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'6742", 'reported_seq': 6872, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:30.408240+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:22:30.408240+0000', 'last_peered': '2024-09-15T21:22:30.408240+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:22:30.408240+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 6742, 'log_dups_size': 0, 'ondisk_log_size': 6742, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27595235, 'num_objects': 6738, 'num_object_clones': 0, 'num_object_copies': 13476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6738, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3418, 'num_write_kb': 13672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2662}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:34.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:34.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'6852", 'reported_seq': 6983, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:30.259516+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:22:30.259516+0000', 'last_peered': '2024-09-15T21:22:30.259516+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:22:30.259516+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 6852, 'log_dups_size': 0, 'ondisk_log_size': 6852, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28042182, 'num_objects': 6848, 'num_object_clones': 0, 'num_object_copies': 13696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6848, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3163, 'num_write_kb': 12652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2687}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:34.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:34.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'6670", 'reported_seq': 6791, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:28.232320+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:22:28.232320+0000', 'last_peered': '2024-09-15T21:22:28.232320+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:22:28.232320+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 6670, 'log_dups_size': 0, 'ondisk_log_size': 6670, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27292131, 'num_objects': 6664, 'num_object_clones': 0, 'num_object_copies': 13328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6664, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4602, 'num_write_kb': 18408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 2577}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:34.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:34.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'6652", 'reported_seq': 6778, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:30.413117+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:22:30.413117+0000', 'last_peered': '2024-09-15T21:22:30.413117+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:22:30.413117+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 6652, 'log_dups_size': 0, 'ondisk_log_size': 6652, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27213824, 'num_objects': 6644, 'num_object_clones': 0, 'num_object_copies': 13288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6644, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3325, 'num_write_kb': 13300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2624}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:34.612 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:34.612 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'6678", 'reported_seq': 6792, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:28.253215+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:22:28.253215+0000', 'last_peered': '2024-09-15T21:22:28.253215+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:22:28.253215+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 6678, 'log_dups_size': 0, 'ondisk_log_size': 6678, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27333091, 'num_objects': 6674, 'num_object_clones': 0, 'num_object_copies': 13348, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6674, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4537, 'num_write_kb': 18148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 2584}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:34.612 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:34.612 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'6773", 'reported_seq': 10581, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:22:28.251012+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:22:28.251012+0000', 'last_peered': '2024-09-15T21:22:28.251012+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:22:28.251012+0000', 'last_undegraded': '2024-09-15T21:22:28.251012+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 6773, 'log_dups_size': 0, 'ondisk_log_size': 6773, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27717632, 'num_objects': 6767, 'num_object_clones': 0, 'num_object_copies': 13534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2317, 'num_objects_unfound': 0, 'num_objects_dirty': 6767, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4466, 'num_write_kb': 17864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1845, 'num_bytes_recovered': 7557120, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2317}, {'shards': '2', 'objects': 4450}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:34.612 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:34.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'6793", 'reported_seq': 6922, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:30.375639+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:22:30.375639+0000', 'last_peered': '2024-09-15T21:22:30.375639+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:22:30.375639+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 6793, 'log_dups_size': 0, 'ondisk_log_size': 6793, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27815936, 'num_objects': 6791, 'num_object_clones': 0, 'num_object_copies': 13582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6791, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3737, 'num_write_kb': 14948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2697}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:34.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:34.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'6813", 'reported_seq': 6941, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:30.343129+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:22:30.343129+0000', 'last_peered': '2024-09-15T21:22:30.343129+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:22:30.343129+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 6813, 'log_dups_size': 0, 'ondisk_log_size': 6813, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27873280, 'num_objects': 6805, 'num_object_clones': 0, 'num_object_copies': 13610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6805, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3722, 'num_write_kb': 14888, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2666}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:34.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:34.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'6661", 'reported_seq': 6792, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:30.355991+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:22:30.355991+0000', 'last_peered': '2024-09-15T21:22:30.355991+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:22:30.355991+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 6661, 'log_dups_size': 0, 'ondisk_log_size': 6661, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27267072, 'num_objects': 6657, 'num_object_clones': 0, 'num_object_copies': 13314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3145, 'num_write_kb': 12580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2575}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:34.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:22:34.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'6758", 'reported_seq': 6892, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:30.339837+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:22:30.339837+0000', 'last_peered': '2024-09-15T21:22:30.339837+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:22:30.339837+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 6758, 'log_dups_size': 0, 'ondisk_log_size': 6758, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27632582, 'num_objects': 6748, 'num_object_clones': 0, 'num_object_copies': 13496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6748, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3462, 'num_write_kb': 13848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2601}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:34.616 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:34.746 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 123 20 110820 110800 3.51833 1.73438 0.0486691 0.0221962 2024-09-15T21:22:35.308 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:35.308 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/218350 objects degraded (20.751%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":207,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":109175,"data_bytes":447136497,"bytes_used":1483055104,"bytes_avail":288427237376,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":218350,"degraded_ratio":0.20750629722921915,"misplaced_objects":2173,"misplaced_total":218350,"misplaced_ratio":0.0099519120677810848,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":142604,"recovering_keys_per_sec":0,"num_objects_recovered":418,"num_bytes_recovered":1712128,"num_keys_recovered":0,"write_bytes_sec":2131219,"read_op_per_sec":0,"write_op_per_sec":520},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:35.333 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:35.333 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:35.746 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 124 20 111301 111281 3.5051 1.87891 0.117081 0.0222793 2024-09-15T21:22:35.884 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:35.885 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:35.910 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-15T21:22:34.713967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":447136497,"num_objects":109175,"num_object_clones":0,"num_object_copies":218350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2173,"num_objects_unfound":0,"num_objects_dirty":109175,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":69040,"num_write_kb":276154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1991,"num_bytes_recovered":8151059,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109248,"ondisk_log_size":109248,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1448296,"kb_used_data":879864,"kb_used_omap":12,"kb_used_meta":568371,"kb_avail":281667224,"statfs":{"total":289910292480,"available":288427237376,"internally_reserved":0,"allocated":900980736,"data_stored":900139571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":582011907},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,1,2,9,11,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25587712,"num_objects":6247,"num_object_clones":0,"num_object_copies":12494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-418,"num_objects_unfound":0,"num_objects_dirty":6247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6247,"num_write_kb":24988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":418,"num_bytes_recovered":1712128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006137"},"pg_stats":[{"pgid":"2.f","version":"43'6927","reported_seq":6998,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.338782+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:30.338782+0000","last_peered":"2024-09-15T21:22:30.338782+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:30.338782+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6927,"log_dups_size":0,"ondisk_log_size":6927,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28372992,"num_objects":6927,"num_object_clones":0,"num_object_copies":13854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6927,"num_write_kb":27708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2795},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6895","reported_seq":6994,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:33.234120+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:33.234120+0000","last_peered":"2024-09-15T21:22:33.234120+0000","last_clean":"2024-09-15T21:22:33.234120+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:33.234120+0000","last_undegraded":"2024-09-15T21:22:33.234120+0000","last_fullsized":"2024-09-15T21:22:33.234120+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":6895,"log_dups_size":0,"ondisk_log_size":6895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28218310,"num_objects":6891,"num_object_clones":0,"num_object_copies":13782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6891,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5176,"num_write_kb":20704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7073","reported_seq":7193,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:33.267490+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:33.267490+0000","last_peered":"2024-09-15T21:22:33.267490+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:33.267490+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7073,"log_dups_size":0,"ondisk_log_size":7073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5173,"num_write_kb":20692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2802}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6775","reported_seq":6841,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:33.238788+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:33.238788+0000","last_peered":"2024-09-15T21:22:33.238788+0000","last_clean":"2024-09-15T21:22:33.238788+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:33.238788+0000","last_undegraded":"2024-09-15T21:22:33.238788+0000","last_fullsized":"2024-09-15T21:22:33.238788+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6775,"log_dups_size":0,"ondisk_log_size":6775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6775,"num_write_kb":27100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6763","reported_seq":6891,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:30.360772+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:30.360772+0000","last_peered":"2024-09-15T21:22:30.360772+0000","last_clean":"2024-09-15T21:22:30.360772+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:30.360772+0000","last_undegraded":"2024-09-15T21:22:30.360772+0000","last_fullsized":"2024-09-15T21:22:30.360772+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6763,"log_dups_size":0,"ondisk_log_size":6763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27673059,"num_objects":6757,"num_object_clones":0,"num_object_copies":13514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3218,"num_write_kb":12872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6742","reported_seq":6872,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.408240+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:30.408240+0000","last_peered":"2024-09-15T21:22:30.408240+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:30.408240+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6742,"log_dups_size":0,"ondisk_log_size":6742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27595235,"num_objects":6738,"num_object_clones":0,"num_object_copies":13476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3418,"num_write_kb":13672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2662},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6852","reported_seq":6983,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.259516+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:30.259516+0000","last_peered":"2024-09-15T21:22:30.259516+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:30.259516+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":6852,"log_dups_size":0,"ondisk_log_size":6852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28042182,"num_objects":6848,"num_object_clones":0,"num_object_copies":13696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3163,"num_write_kb":12652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2687},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6919","reported_seq":7043,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:30.319548+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:30.319548+0000","last_peered":"2024-09-15T21:22:30.319548+0000","last_clean":"2024-09-15T21:22:30.319548+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:30.319548+0000","last_undegraded":"2024-09-15T21:22:30.319548+0000","last_fullsized":"2024-09-15T21:22:30.319548+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":6919,"log_dups_size":0,"ondisk_log_size":6919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28332032,"num_objects":6917,"num_object_clones":0,"num_object_copies":13834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3691,"num_write_kb":14764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6829","reported_seq":6950,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:33.269812+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:33.269812+0000","last_peered":"2024-09-15T21:22:33.269812+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:33.269812+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":6829,"log_dups_size":0,"ondisk_log_size":6829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27943395,"num_objects":6823,"num_object_clones":0,"num_object_copies":13646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4761,"num_write_kb":19044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2736}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6652","reported_seq":6778,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.413117+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:30.413117+0000","last_peered":"2024-09-15T21:22:30.413117+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:30.413117+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6652,"log_dups_size":0,"ondisk_log_size":6652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27213824,"num_objects":6644,"num_object_clones":0,"num_object_copies":13288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3325,"num_write_kb":13300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2624},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6859","reported_seq":6973,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:33.238871+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:33.238871+0000","last_peered":"2024-09-15T21:22:33.238871+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:33.238871+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":6859,"log_dups_size":0,"ondisk_log_size":6859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28074467,"num_objects":6855,"num_object_clones":0,"num_object_copies":13710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6855,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4718,"num_write_kb":18872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2765}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6934","reported_seq":11030,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:33.291951+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:33.291951+0000","last_peered":"2024-09-15T21:22:33.291951+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:33.291951+0000","last_undegraded":"2024-09-15T21:22:33.291951+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":6934,"log_dups_size":0,"ondisk_log_size":6934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28377088,"num_objects":6928,"num_object_clones":0,"num_object_copies":13856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2173,"num_objects_unfound":0,"num_objects_dirty":6928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4627,"num_write_kb":18508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1989,"num_bytes_recovered":8146944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2173},{"shards":"2","objects":4755}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6793","reported_seq":6922,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.375639+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:30.375639+0000","last_peered":"2024-09-15T21:22:30.375639+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:30.375639+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6793,"log_dups_size":0,"ondisk_log_size":6793,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27815936,"num_objects":6791,"num_object_clones":0,"num_object_copies":13582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3737,"num_write_kb":14948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2697},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6813","reported_seq":6941,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.343129+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:30.343129+0000","last_peered":"2024-09-15T21:22:30.343129+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:30.343129+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6813,"log_dups_size":0,"ondisk_log_size":6813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27873280,"num_objects":6805,"num_object_clones":0,"num_object_copies":13610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3722,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2666},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6661","reported_seq":6792,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.355991+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:30.355991+0000","last_peered":"2024-09-15T21:22:30.355991+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:30.355991+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6661,"log_dups_size":0,"ondisk_log_size":6661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27267072,"num_objects":6657,"num_object_clones":0,"num_object_copies":13314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3145,"num_write_kb":12580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2575},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6758","reported_seq":6892,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:30.339837+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:30.339837+0000","last_peered":"2024-09-15T21:22:30.339837+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:30.339837+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6758,"log_dups_size":0,"ondisk_log_size":6758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27632582,"num_objects":6748,"num_object_clones":0,"num_object_copies":13496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3462,"num_write_kb":13848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2601},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":447136478,"num_objects":109173,"num_object_clones":0,"num_object_copies":218346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2173,"num_objects_unfound":0,"num_objects_dirty":109173,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":69038,"num_write_kb":276152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1990,"num_bytes_recovered":8151040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":901955584,"data_stored":901800225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":109245,"ondisk_log_size":109245,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528312,"kb_used_data":352232,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93843528,"statfs":{"total":96636764160,"available":96095772672,"internally_reserved":0,"allocated":360685568,"data_stored":360423554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":180285097},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,1,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643760,"kb_used_data":391392,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93728080,"statfs":{"total":96636764160,"available":95977553920,"internally_reserved":0,"allocated":400785408,"data_stored":400519317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":258404011},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":276224,"kb_used_data":136240,"kb_used_omap":4,"kb_used_meta":139963,"kb_avail":94095616,"statfs":{"total":96636764160,"available":96353910784,"internally_reserved":0,"allocated":139509760,"data_stored":139196700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":143322799},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,6,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401113088,"data_stored":401076958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":140197888,"data_stored":140114789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360644608,"data_stored":360608478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:36.747 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 125 20 111830 111810 3.49358 2.06641 0.0173809 0.0223536 2024-09-15T21:22:37.747 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 126 20 112382 112362 3.48297 2.15625 0.0252127 0.0224235 2024-09-15T21:22:38.747 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 127 20 112899 112879 3.47144 2.01953 0.0935036 0.0224914 2024-09-15T21:22:38.913 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:39.473 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:39.474 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:39.495 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-15T21:22:38.714930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":457720561,"num_objects":111759,"num_object_clones":0,"num_object_copies":223518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2034,"num_objects_unfound":0,"num_objects_dirty":111759,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":71624,"num_write_kb":286490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2130,"num_bytes_recovered":8720403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111832,"ondisk_log_size":111832,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521828,"kb_used_data":900660,"kb_used_omap":12,"kb_used_meta":621107,"kb_avail":281593692,"statfs":{"total":289910292480,"available":288351940608,"internally_reserved":0,"allocated":922275840,"data_stored":921434675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":636013571},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,3,9,13,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25231360,"num_objects":6160,"num_object_clones":0,"num_object_copies":12320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-422,"num_objects_unfound":0,"num_objects_dirty":6160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6160,"num_write_kb":24640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":1728512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006142"},"pg_stats":[{"pgid":"2.f","version":"43'7118","reported_seq":7189,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.299238+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:35.299238+0000","last_peered":"2024-09-15T21:22:35.299238+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:35.299238+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7118,"log_dups_size":0,"ondisk_log_size":7118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29155328,"num_objects":7118,"num_object_clones":0,"num_object_copies":14236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7118,"num_write_kb":28472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2986},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7053","reported_seq":7153,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.179472+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:38.179472+0000","last_peered":"2024-09-15T21:22:38.179472+0000","last_clean":"2024-09-15T21:22:38.179472+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:38.179472+0000","last_undegraded":"2024-09-15T21:22:38.179472+0000","last_fullsized":"2024-09-15T21:22:38.179472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28865478,"num_objects":7049,"num_object_clones":0,"num_object_copies":14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5334,"num_write_kb":21336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7233","reported_seq":7353,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.261786+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:38.261786+0000","last_peered":"2024-09-15T21:22:38.261786+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:38.261786+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7233,"log_dups_size":0,"ondisk_log_size":7233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29609984,"num_objects":7229,"num_object_clones":0,"num_object_copies":14458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5333,"num_write_kb":21332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2962}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6919","reported_seq":6984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.274247+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:38.274247+0000","last_peered":"2024-09-15T21:22:38.274247+0000","last_clean":"2024-09-15T21:22:38.274247+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:38.274247+0000","last_undegraded":"2024-09-15T21:22:38.274247+0000","last_fullsized":"2024-09-15T21:22:38.274247+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6919,"log_dups_size":0,"ondisk_log_size":6919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28340224,"num_objects":6919,"num_object_clones":0,"num_object_copies":13838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6919,"num_write_kb":27676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6922","reported_seq":7050,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:35.398076+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:35.398076+0000","last_peered":"2024-09-15T21:22:35.398076+0000","last_clean":"2024-09-15T21:22:35.398076+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:35.398076+0000","last_undegraded":"2024-09-15T21:22:35.398076+0000","last_fullsized":"2024-09-15T21:22:35.398076+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6922,"log_dups_size":0,"ondisk_log_size":6922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28324323,"num_objects":6916,"num_object_clones":0,"num_object_copies":13832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3377,"num_write_kb":13508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6902","reported_seq":7033,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.346059+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:35.346059+0000","last_peered":"2024-09-15T21:22:35.346059+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:35.346059+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6902,"log_dups_size":0,"ondisk_log_size":6902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28250595,"num_objects":6898,"num_object_clones":0,"num_object_copies":13796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3578,"num_write_kb":14312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2822},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7021","reported_seq":7151,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.428996+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:35.428996+0000","last_peered":"2024-09-15T21:22:35.428996+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:35.428996+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7021,"log_dups_size":0,"ondisk_log_size":7021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28734406,"num_objects":7017,"num_object_clones":0,"num_object_copies":14034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3332,"num_write_kb":13328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2856},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7067","reported_seq":7191,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:35.420385+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:35.420385+0000","last_peered":"2024-09-15T21:22:35.420385+0000","last_clean":"2024-09-15T21:22:35.420385+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:35.420385+0000","last_undegraded":"2024-09-15T21:22:35.420385+0000","last_fullsized":"2024-09-15T21:22:35.420385+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7067,"log_dups_size":0,"ondisk_log_size":7067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28938240,"num_objects":7065,"num_object_clones":0,"num_object_copies":14130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3839,"num_write_kb":15356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7005","reported_seq":7127,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.286128+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:38.286128+0000","last_peered":"2024-09-15T21:22:38.286128+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:38.286128+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7005,"log_dups_size":0,"ondisk_log_size":7005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28664291,"num_objects":6999,"num_object_clones":0,"num_object_copies":13998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6999,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4937,"num_write_kb":19748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2912}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6813","reported_seq":6939,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.344184+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:35.344184+0000","last_peered":"2024-09-15T21:22:35.344184+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:35.344184+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6813,"log_dups_size":0,"ondisk_log_size":6813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27873280,"num_objects":6805,"num_object_clones":0,"num_object_copies":13610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3486,"num_write_kb":13944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2785},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7027","reported_seq":7141,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.222860+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:38.222860+0000","last_peered":"2024-09-15T21:22:38.222860+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:38.222860+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7027,"log_dups_size":0,"ondisk_log_size":7027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762595,"num_objects":7023,"num_object_clones":0,"num_object_copies":14046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7023,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4886,"num_write_kb":19544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2933}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7102","reported_seq":11475,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:38.302006+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:38.302006+0000","last_peered":"2024-09-15T21:22:38.302006+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:38.302006+0000","last_undegraded":"2024-09-15T21:22:38.302006+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7102,"log_dups_size":0,"ondisk_log_size":7102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29065216,"num_objects":7096,"num_object_clones":0,"num_object_copies":14192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2034,"num_objects_unfound":0,"num_objects_dirty":7096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4795,"num_write_kb":19180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2128,"num_bytes_recovered":8716288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2034},{"shards":"2","objects":5062}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6951","reported_seq":7080,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.413064+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:35.413064+0000","last_peered":"2024-09-15T21:22:35.413064+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:35.413064+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6951,"log_dups_size":0,"ondisk_log_size":6951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28463104,"num_objects":6949,"num_object_clones":0,"num_object_copies":13898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3895,"num_write_kb":15580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2855},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6957","reported_seq":7085,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.341280+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:35.341280+0000","last_peered":"2024-09-15T21:22:35.341280+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:35.341280+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6957,"log_dups_size":0,"ondisk_log_size":6957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28463104,"num_objects":6949,"num_object_clones":0,"num_object_copies":13898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3866,"num_write_kb":15464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2810},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6806","reported_seq":6936,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.278757+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:35.278757+0000","last_peered":"2024-09-15T21:22:35.278757+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:35.278757+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6806,"log_dups_size":0,"ondisk_log_size":6806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27860992,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3290,"num_write_kb":13160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2720},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6933","reported_seq":7067,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.361273+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:35.361273+0000","last_peered":"2024-09-15T21:22:35.361273+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:35.361273+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6933,"log_dups_size":0,"ondisk_log_size":6933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28349382,"num_objects":6923,"num_object_clones":0,"num_object_copies":13846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6923,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3637,"num_write_kb":14548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2776},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":457720542,"num_objects":111757,"num_object_clones":0,"num_object_copies":223514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2034,"num_objects_unfound":0,"num_objects_dirty":111757,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":71622,"num_write_kb":286488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2129,"num_bytes_recovered":8720384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":923582464,"data_stored":923427105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111829,"ondisk_log_size":111829,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537624,"kb_used_data":357384,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93834216,"statfs":{"total":96636764160,"available":96086237184,"internally_reserved":0,"allocated":365961216,"data_stored":365699202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":184544937},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673428,"kb_used_data":398852,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93698412,"statfs":{"total":96636764160,"available":95947173888,"internally_reserved":0,"allocated":408424448,"data_stored":408158357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,6,7,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310776,"kb_used_data":144424,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94061064,"statfs":{"total":96636764160,"available":96318529536,"internally_reserved":0,"allocated":147890176,"data_stored":147577116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":170323631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":408924160,"data_stored":408888030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":148701184,"data_stored":148618085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365957120,"data_stored":365920990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:39.496 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:39.747 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 128 20 113377 113357 3.45891 1.86719 0.0405976 0.0225773 2024-09-15T21:22:40.079 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:40.081 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:40.105 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-09-15T21:22:38.714930+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":457720561,"num_objects":111759,"num_object_clones":0,"num_object_copies":223518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2034,"num_objects_unfound":0,"num_objects_dirty":111759,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":71624,"num_write_kb":286490,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2130,"num_bytes_recovered":8720403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111832,"ondisk_log_size":111832,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521828,"kb_used_data":900660,"kb_used_omap":12,"kb_used_meta":621107,"kb_avail":281593692,"statfs":{"total":289910292480,"available":288351940608,"internally_reserved":0,"allocated":922275840,"data_stored":921434675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":636013571},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,3,9,13,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25231360,"num_objects":6160,"num_object_clones":0,"num_object_copies":12320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-422,"num_objects_unfound":0,"num_objects_dirty":6160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6160,"num_write_kb":24640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":422,"num_bytes_recovered":1728512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006142"},"pg_stats":[{"pgid":"2.f","version":"43'7118","reported_seq":7189,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.299238+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:35.299238+0000","last_peered":"2024-09-15T21:22:35.299238+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:35.299238+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7118,"log_dups_size":0,"ondisk_log_size":7118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29155328,"num_objects":7118,"num_object_clones":0,"num_object_copies":14236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7118,"num_write_kb":28472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2986},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7053","reported_seq":7153,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.179472+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:38.179472+0000","last_peered":"2024-09-15T21:22:38.179472+0000","last_clean":"2024-09-15T21:22:38.179472+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:38.179472+0000","last_undegraded":"2024-09-15T21:22:38.179472+0000","last_fullsized":"2024-09-15T21:22:38.179472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28865478,"num_objects":7049,"num_object_clones":0,"num_object_copies":14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5334,"num_write_kb":21336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7233","reported_seq":7353,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.261786+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:38.261786+0000","last_peered":"2024-09-15T21:22:38.261786+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:38.261786+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7233,"log_dups_size":0,"ondisk_log_size":7233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29609984,"num_objects":7229,"num_object_clones":0,"num_object_copies":14458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5333,"num_write_kb":21332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2962}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6919","reported_seq":6984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.274247+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:38.274247+0000","last_peered":"2024-09-15T21:22:38.274247+0000","last_clean":"2024-09-15T21:22:38.274247+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:38.274247+0000","last_undegraded":"2024-09-15T21:22:38.274247+0000","last_fullsized":"2024-09-15T21:22:38.274247+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6919,"log_dups_size":0,"ondisk_log_size":6919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28340224,"num_objects":6919,"num_object_clones":0,"num_object_copies":13838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6919,"num_write_kb":27676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6922","reported_seq":7050,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:35.398076+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:35.398076+0000","last_peered":"2024-09-15T21:22:35.398076+0000","last_clean":"2024-09-15T21:22:35.398076+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:35.398076+0000","last_undegraded":"2024-09-15T21:22:35.398076+0000","last_fullsized":"2024-09-15T21:22:35.398076+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":6922,"log_dups_size":0,"ondisk_log_size":6922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28324323,"num_objects":6916,"num_object_clones":0,"num_object_copies":13832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3377,"num_write_kb":13508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6902","reported_seq":7033,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.346059+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:35.346059+0000","last_peered":"2024-09-15T21:22:35.346059+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:35.346059+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":6902,"log_dups_size":0,"ondisk_log_size":6902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28250595,"num_objects":6898,"num_object_clones":0,"num_object_copies":13796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3578,"num_write_kb":14312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2822},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7021","reported_seq":7151,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.428996+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:35.428996+0000","last_peered":"2024-09-15T21:22:35.428996+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:35.428996+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7021,"log_dups_size":0,"ondisk_log_size":7021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28734406,"num_objects":7017,"num_object_clones":0,"num_object_copies":14034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3332,"num_write_kb":13328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2856},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7067","reported_seq":7191,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:35.420385+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:35.420385+0000","last_peered":"2024-09-15T21:22:35.420385+0000","last_clean":"2024-09-15T21:22:35.420385+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:35.420385+0000","last_undegraded":"2024-09-15T21:22:35.420385+0000","last_fullsized":"2024-09-15T21:22:35.420385+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7067,"log_dups_size":0,"ondisk_log_size":7067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28938240,"num_objects":7065,"num_object_clones":0,"num_object_copies":14130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3839,"num_write_kb":15356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7005","reported_seq":7127,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.286128+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:38.286128+0000","last_peered":"2024-09-15T21:22:38.286128+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:38.286128+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7005,"log_dups_size":0,"ondisk_log_size":7005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28664291,"num_objects":6999,"num_object_clones":0,"num_object_copies":13998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6999,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4937,"num_write_kb":19748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2912}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6813","reported_seq":6939,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.344184+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:35.344184+0000","last_peered":"2024-09-15T21:22:35.344184+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:35.344184+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6813,"log_dups_size":0,"ondisk_log_size":6813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27873280,"num_objects":6805,"num_object_clones":0,"num_object_copies":13610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3486,"num_write_kb":13944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2785},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7027","reported_seq":7141,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.222860+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:38.222860+0000","last_peered":"2024-09-15T21:22:38.222860+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:38.222860+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7027,"log_dups_size":0,"ondisk_log_size":7027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762595,"num_objects":7023,"num_object_clones":0,"num_object_copies":14046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7023,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4886,"num_write_kb":19544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2933}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7102","reported_seq":11475,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:38.302006+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:38.302006+0000","last_peered":"2024-09-15T21:22:38.302006+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:38.302006+0000","last_undegraded":"2024-09-15T21:22:38.302006+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7102,"log_dups_size":0,"ondisk_log_size":7102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29065216,"num_objects":7096,"num_object_clones":0,"num_object_copies":14192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2034,"num_objects_unfound":0,"num_objects_dirty":7096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4795,"num_write_kb":19180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2128,"num_bytes_recovered":8716288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2034},{"shards":"2","objects":5062}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6951","reported_seq":7080,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.413064+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:35.413064+0000","last_peered":"2024-09-15T21:22:35.413064+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:35.413064+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":6951,"log_dups_size":0,"ondisk_log_size":6951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28463104,"num_objects":6949,"num_object_clones":0,"num_object_copies":13898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3895,"num_write_kb":15580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2855},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6957","reported_seq":7085,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.341280+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:35.341280+0000","last_peered":"2024-09-15T21:22:35.341280+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:35.341280+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":6957,"log_dups_size":0,"ondisk_log_size":6957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28463104,"num_objects":6949,"num_object_clones":0,"num_object_copies":13898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3866,"num_write_kb":15464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2810},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6806","reported_seq":6936,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.278757+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:35.278757+0000","last_peered":"2024-09-15T21:22:35.278757+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:35.278757+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6806,"log_dups_size":0,"ondisk_log_size":6806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27860992,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3290,"num_write_kb":13160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2720},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6933","reported_seq":7067,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:35.361273+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:35.361273+0000","last_peered":"2024-09-15T21:22:35.361273+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:35.361273+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":6933,"log_dups_size":0,"ondisk_log_size":6933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28349382,"num_objects":6923,"num_object_clones":0,"num_object_copies":13846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6923,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3637,"num_write_kb":14548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2776},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":457720542,"num_objects":111757,"num_object_clones":0,"num_object_copies":223514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2034,"num_objects_unfound":0,"num_objects_dirty":111757,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":71622,"num_write_kb":286488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2129,"num_bytes_recovered":8720384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":923582464,"data_stored":923427105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111829,"ondisk_log_size":111829,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537624,"kb_used_data":357384,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93834216,"statfs":{"total":96636764160,"available":96086237184,"internally_reserved":0,"allocated":365961216,"data_stored":365699202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":184544937},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673428,"kb_used_data":398852,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93698412,"statfs":{"total":96636764160,"available":95947173888,"internally_reserved":0,"allocated":408424448,"data_stored":408158357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,6,7,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310776,"kb_used_data":144424,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94061064,"statfs":{"total":96636764160,"available":96318529536,"internally_reserved":0,"allocated":147890176,"data_stored":147577116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":170323631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":408924160,"data_stored":408888030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":148701184,"data_stored":148618085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365957120,"data_stored":365920990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:40.107 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:40.107 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'7118", 'reported_seq': 7189, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:35.299238+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:22:35.299238+0000', 'last_peered': '2024-09-15T21:22:35.299238+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:22:35.299238+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 7118, 'log_dups_size': 0, 'ondisk_log_size': 7118, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29155328, 'num_objects': 7118, 'num_object_clones': 0, 'num_object_copies': 14236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7118, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7118, 'num_write_kb': 28472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2986}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:40.107 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:22:40.107 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'7233", 'reported_seq': 7353, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:38.261786+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:22:38.261786+0000', 'last_peered': '2024-09-15T21:22:38.261786+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:22:38.261786+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 7233, 'log_dups_size': 0, 'ondisk_log_size': 7233, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29609984, 'num_objects': 7229, 'num_object_clones': 0, 'num_object_copies': 14458, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7229, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5333, 'num_write_kb': 21332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 2962}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:40.107 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:40.107 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'6902", 'reported_seq': 7033, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:35.346059+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:22:35.346059+0000', 'last_peered': '2024-09-15T21:22:35.346059+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:22:35.346059+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 6902, 'log_dups_size': 0, 'ondisk_log_size': 6902, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28250595, 'num_objects': 6898, 'num_object_clones': 0, 'num_object_copies': 13796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6898, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3578, 'num_write_kb': 14312, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2822}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:40.107 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:40.107 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'7021", 'reported_seq': 7151, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:35.428996+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:22:35.428996+0000', 'last_peered': '2024-09-15T21:22:35.428996+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:22:35.428996+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 7021, 'log_dups_size': 0, 'ondisk_log_size': 7021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28734406, 'num_objects': 7017, 'num_object_clones': 0, 'num_object_copies': 14034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7017, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3332, 'num_write_kb': 13328, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2856}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:40.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:40.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7005", 'reported_seq': 7127, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:38.286128+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:22:38.286128+0000', 'last_peered': '2024-09-15T21:22:38.286128+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:22:38.286128+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 7005, 'log_dups_size': 0, 'ondisk_log_size': 7005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28664291, 'num_objects': 6999, 'num_object_clones': 0, 'num_object_copies': 13998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6999, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4937, 'num_write_kb': 19748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 2912}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:40.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:40.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'6813", 'reported_seq': 6939, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:35.344184+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:22:35.344184+0000', 'last_peered': '2024-09-15T21:22:35.344184+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:22:35.344184+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 6813, 'log_dups_size': 0, 'ondisk_log_size': 6813, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27873280, 'num_objects': 6805, 'num_object_clones': 0, 'num_object_copies': 13610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6805, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3486, 'num_write_kb': 13944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2785}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:40.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:40.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7027", 'reported_seq': 7141, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:38.222860+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:22:38.222860+0000', 'last_peered': '2024-09-15T21:22:38.222860+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:22:38.222860+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 7027, 'log_dups_size': 0, 'ondisk_log_size': 7027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28762595, 'num_objects': 7023, 'num_object_clones': 0, 'num_object_copies': 14046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7023, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4886, 'num_write_kb': 19544, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 2933}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:40.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:40.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'7102", 'reported_seq': 11475, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:22:38.302006+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:22:38.302006+0000', 'last_peered': '2024-09-15T21:22:38.302006+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:22:38.302006+0000', 'last_undegraded': '2024-09-15T21:22:38.302006+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 7102, 'log_dups_size': 0, 'ondisk_log_size': 7102, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29065216, 'num_objects': 7096, 'num_object_clones': 0, 'num_object_copies': 14192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2034, 'num_objects_unfound': 0, 'num_objects_dirty': 7096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4795, 'num_write_kb': 19180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2128, 'num_bytes_recovered': 8716288, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2034}, {'shards': '2', 'objects': 5062}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:40.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:40.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'6951", 'reported_seq': 7080, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:35.413064+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:22:35.413064+0000', 'last_peered': '2024-09-15T21:22:35.413064+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:22:35.413064+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 6951, 'log_dups_size': 0, 'ondisk_log_size': 6951, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28463104, 'num_objects': 6949, 'num_object_clones': 0, 'num_object_copies': 13898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6949, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3895, 'num_write_kb': 15580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2855}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:40.109 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:40.109 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'6957", 'reported_seq': 7085, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:35.341280+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:22:35.341280+0000', 'last_peered': '2024-09-15T21:22:35.341280+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:22:35.341280+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 6957, 'log_dups_size': 0, 'ondisk_log_size': 6957, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28463104, 'num_objects': 6949, 'num_object_clones': 0, 'num_object_copies': 13898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6949, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3866, 'num_write_kb': 15464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2810}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:40.109 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:40.109 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'6806", 'reported_seq': 6936, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:35.278757+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:22:35.278757+0000', 'last_peered': '2024-09-15T21:22:35.278757+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:22:35.278757+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 6806, 'log_dups_size': 0, 'ondisk_log_size': 6806, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27860992, 'num_objects': 6802, 'num_object_clones': 0, 'num_object_copies': 13604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6802, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3290, 'num_write_kb': 13160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2720}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:40.109 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:22:40.109 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'6933", 'reported_seq': 7067, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:35.361273+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:22:35.361273+0000', 'last_peered': '2024-09-15T21:22:35.361273+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:22:35.361273+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 6933, 'log_dups_size': 0, 'ondisk_log_size': 6933, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28349382, 'num_objects': 6923, 'num_object_clones': 0, 'num_object_copies': 13846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6923, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3637, 'num_write_kb': 14548, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2776}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:40.109 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:40.747 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 129 20 113845 113825 3.44627 1.82812 0.0214817 0.0226617 2024-09-15T21:22:40.800 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:40.801 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/223518 objects degraded (20.271%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":212,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":111759,"data_bytes":457720561,"bytes_used":1558351872,"bytes_avail":288351940608,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":223518,"degraded_ratio":0.20270850669744717,"misplaced_objects":2034,"misplaced_total":223518,"misplaced_ratio":0.0090999382600059058,"recovering_objects_per_sec":35,"recovering_bytes_per_sec":143968,"recovering_keys_per_sec":0,"num_objects_recovered":422,"num_bytes_recovered":1728512,"num_keys_recovered":0,"write_bytes_sec":2101537,"read_op_per_sec":0,"write_op_per_sec":513},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:40.831 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:40.831 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:41.382 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:41.385 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:41.404 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-15T21:22:40.715290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":464122609,"num_objects":113322,"num_object_clones":0,"num_object_copies":226644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2034,"num_objects_unfound":0,"num_objects_dirty":113322,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":73187,"num_write_kb":292742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2130,"num_bytes_recovered":8720403,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113395,"ondisk_log_size":113395,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521008,"kb_used_data":908160,"kb_used_omap":12,"kb_used_meta":612787,"kb_avail":281594512,"statfs":{"total":289910292480,"available":288352780288,"internally_reserved":0,"allocated":929955840,"data_stored":929114675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":627493891},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,7,6,11,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27533312,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6722,"num_write_kb":26888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":1159168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006001"},"pg_stats":[{"pgid":"2.f","version":"43'7269","reported_seq":7340,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.418292+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:40.418292+0000","last_peered":"2024-09-15T21:22:40.418292+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:40.418292+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7269,"log_dups_size":0,"ondisk_log_size":7269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29773824,"num_objects":7269,"num_object_clones":0,"num_object_copies":14538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7269,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3137},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7053","reported_seq":7153,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.179472+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:38.179472+0000","last_peered":"2024-09-15T21:22:38.179472+0000","last_clean":"2024-09-15T21:22:38.179472+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:38.179472+0000","last_undegraded":"2024-09-15T21:22:38.179472+0000","last_fullsized":"2024-09-15T21:22:38.179472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28865478,"num_objects":7049,"num_object_clones":0,"num_object_copies":14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7049,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5334,"num_write_kb":21336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7233","reported_seq":7353,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.261786+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:38.261786+0000","last_peered":"2024-09-15T21:22:38.261786+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:38.261786+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7233,"log_dups_size":0,"ondisk_log_size":7233,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29609984,"num_objects":7229,"num_object_clones":0,"num_object_copies":14458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5333,"num_write_kb":21332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":2962}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6919","reported_seq":6984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:38.274247+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:38.274247+0000","last_peered":"2024-09-15T21:22:38.274247+0000","last_clean":"2024-09-15T21:22:38.274247+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:38.274247+0000","last_undegraded":"2024-09-15T21:22:38.274247+0000","last_fullsized":"2024-09-15T21:22:38.274247+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":6919,"log_dups_size":0,"ondisk_log_size":6919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28340224,"num_objects":6919,"num_object_clones":0,"num_object_copies":13838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6919,"num_write_kb":27676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7063","reported_seq":7191,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:40.364349+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:40.364349+0000","last_peered":"2024-09-15T21:22:40.364349+0000","last_clean":"2024-09-15T21:22:40.364349+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:40.364349+0000","last_undegraded":"2024-09-15T21:22:40.364349+0000","last_fullsized":"2024-09-15T21:22:40.364349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7063,"log_dups_size":0,"ondisk_log_size":7063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28901859,"num_objects":7057,"num_object_clones":0,"num_object_copies":14114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3518,"num_write_kb":14072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7082","reported_seq":7213,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.385051+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:40.385051+0000","last_peered":"2024-09-15T21:22:40.385051+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:40.385051+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28987875,"num_objects":7078,"num_object_clones":0,"num_object_copies":14156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7078,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3758,"num_write_kb":15032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3002},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7169","reported_seq":7298,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.350338+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:40.350338+0000","last_peered":"2024-09-15T21:22:40.350338+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:40.350338+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7169,"log_dups_size":0,"ondisk_log_size":7169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29340614,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3480,"num_write_kb":13920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3004},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7247","reported_seq":7371,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:40.374291+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:40.374291+0000","last_peered":"2024-09-15T21:22:40.374291+0000","last_clean":"2024-09-15T21:22:40.374291+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:40.374291+0000","last_undegraded":"2024-09-15T21:22:40.374291+0000","last_fullsized":"2024-09-15T21:22:40.374291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7247,"log_dups_size":0,"ondisk_log_size":7247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29675520,"num_objects":7245,"num_object_clones":0,"num_object_copies":14490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4019,"num_write_kb":16076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7005","reported_seq":7127,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.286128+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:38.286128+0000","last_peered":"2024-09-15T21:22:38.286128+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:38.286128+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7005,"log_dups_size":0,"ondisk_log_size":7005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28664291,"num_objects":6999,"num_object_clones":0,"num_object_copies":13998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6999,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4937,"num_write_kb":19748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":2912}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6973","reported_seq":7099,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.387660+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:40.387660+0000","last_peered":"2024-09-15T21:22:40.387660+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:40.387660+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6973,"log_dups_size":0,"ondisk_log_size":6973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28528640,"num_objects":6965,"num_object_clones":0,"num_object_copies":13930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3646,"num_write_kb":14584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2945},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7027","reported_seq":7141,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:38.222860+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:38.222860+0000","last_peered":"2024-09-15T21:22:38.222860+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:38.222860+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7027,"log_dups_size":0,"ondisk_log_size":7027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28762595,"num_objects":7023,"num_object_clones":0,"num_object_copies":14046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7023,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4886,"num_write_kb":19544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":2933}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7102","reported_seq":11475,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:38.302006+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:38.302006+0000","last_peered":"2024-09-15T21:22:38.302006+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:38.302006+0000","last_undegraded":"2024-09-15T21:22:38.302006+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7102,"log_dups_size":0,"ondisk_log_size":7102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29065216,"num_objects":7096,"num_object_clones":0,"num_object_copies":14192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2034,"num_objects_unfound":0,"num_objects_dirty":7096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4795,"num_write_kb":19180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2128,"num_bytes_recovered":8716288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2034},{"shards":"2","objects":5062}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7100","reported_seq":7229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.419202+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:40.419202+0000","last_peered":"2024-09-15T21:22:40.419202+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:40.419202+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7100,"log_dups_size":0,"ondisk_log_size":7100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29073408,"num_objects":7098,"num_object_clones":0,"num_object_copies":14196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4044,"num_write_kb":16176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3004},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7116","reported_seq":7244,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.395746+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:40.395746+0000","last_peered":"2024-09-15T21:22:40.395746+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:40.395746+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7116,"log_dups_size":0,"ondisk_log_size":7116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29114368,"num_objects":7108,"num_object_clones":0,"num_object_copies":14216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4025,"num_write_kb":16100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2969},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6952","reported_seq":7083,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.395860+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:40.395860+0000","last_peered":"2024-09-15T21:22:40.395860+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:40.395860+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6952,"log_dups_size":0,"ondisk_log_size":6952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28459008,"num_objects":6948,"num_object_clones":0,"num_object_copies":13896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3436,"num_write_kb":13744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2866},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7082","reported_seq":7216,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.419869+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:40.419869+0000","last_peered":"2024-09-15T21:22:40.419869+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:40.419869+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28959686,"num_objects":7072,"num_object_clones":0,"num_object_copies":14144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3786,"num_write_kb":15144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2925},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":464122590,"num_objects":113320,"num_object_clones":0,"num_object_copies":226640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":2034,"num_objects_unfound":0,"num_objects_dirty":113320,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":73185,"num_write_kb":292740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2129,"num_bytes_recovered":8720384,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":931262464,"data_stored":931107105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113392,"ondisk_log_size":113392,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537624,"kb_used_data":357384,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93834216,"statfs":{"total":96636764160,"available":96086237184,"internally_reserved":0,"allocated":365961216,"data_stored":365699202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":184544937},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672608,"kb_used_data":406352,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93699232,"statfs":{"total":96636764160,"available":95948013568,"internally_reserved":0,"allocated":416104448,"data_stored":415838357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,3,5,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310776,"kb_used_data":144424,"kb_used_omap":4,"kb_used_meta":166331,"kb_avail":94061064,"statfs":{"total":96636764160,"available":96318529536,"internally_reserved":0,"allocated":147890176,"data_stored":147577116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":170323631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":416604160,"data_stored":416568030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":148701184,"data_stored":148618085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365957120,"data_stored":365920990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:41.748 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 130 20 114349 114329 3.4349 1.96875 0.0932727 0.0227353 2024-09-15T21:22:42.747 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 131 20 114901 114881 3.42514 2.15625 0.0143113 0.0228018 2024-09-15T21:22:43.748 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 132 20 115434 115414 3.41496 2.08203 0.0282544 0.0228696 2024-09-15T21:22:44.406 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:44.748 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 133 20 115944 115924 3.40426 1.99219 0.02645 0.0229399 2024-09-15T21:22:44.971 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:44.971 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:45.005 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-15T21:22:44.717772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":468030193,"num_objects":114276,"num_object_clones":0,"num_object_copies":228552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1892,"num_objects_unfound":0,"num_objects_dirty":114276,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":74141,"num_write_kb":296558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2274,"num_bytes_recovered":9302035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114349,"ondisk_log_size":114349,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1526044,"kb_used_data":921516,"kb_used_omap":12,"kb_used_meta":604467,"kb_avail":281589476,"statfs":{"total":289910292480,"available":288347623424,"internally_reserved":0,"allocated":943632384,"data_stored":942798445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":618974211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,4,8,9,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24924160,"num_objects":6085,"num_object_clones":0,"num_object_copies":12170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-425,"num_objects_unfound":0,"num_objects_dirty":6085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6085,"num_write_kb":24340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":1740800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007558"},"pg_stats":[{"pgid":"2.f","version":"43'7269","reported_seq":7340,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.418292+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:40.418292+0000","last_peered":"2024-09-15T21:22:40.418292+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:40.418292+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7269,"log_dups_size":0,"ondisk_log_size":7269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29773824,"num_objects":7269,"num_object_clones":0,"num_object_copies":14538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7269,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3137},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7217","reported_seq":7317,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.279839+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:43.279839+0000","last_peered":"2024-09-15T21:22:43.279839+0000","last_clean":"2024-09-15T21:22:43.279839+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:43.279839+0000","last_undegraded":"2024-09-15T21:22:43.279839+0000","last_fullsized":"2024-09-15T21:22:43.279839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7217,"log_dups_size":0,"ondisk_log_size":7217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29537222,"num_objects":7213,"num_object_clones":0,"num_object_copies":14426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7213,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5498,"num_write_kb":21992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7377","reported_seq":7496,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.316930+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:43.316930+0000","last_peered":"2024-09-15T21:22:43.316930+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:43.316930+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7377,"log_dups_size":0,"ondisk_log_size":7377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30199808,"num_objects":7373,"num_object_clones":0,"num_object_copies":14746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5477,"num_write_kb":21908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3106}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7090","reported_seq":7156,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.273555+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:43.273555+0000","last_peered":"2024-09-15T21:22:43.273555+0000","last_clean":"2024-09-15T21:22:43.273555+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:43.273555+0000","last_undegraded":"2024-09-15T21:22:43.273555+0000","last_fullsized":"2024-09-15T21:22:43.273555+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7090,"log_dups_size":0,"ondisk_log_size":7090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29040640,"num_objects":7090,"num_object_clones":0,"num_object_copies":14180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7090,"num_write_kb":28360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7063","reported_seq":7191,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:40.364349+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:40.364349+0000","last_peered":"2024-09-15T21:22:40.364349+0000","last_clean":"2024-09-15T21:22:40.364349+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:40.364349+0000","last_undegraded":"2024-09-15T21:22:40.364349+0000","last_fullsized":"2024-09-15T21:22:40.364349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7063,"log_dups_size":0,"ondisk_log_size":7063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28901859,"num_objects":7057,"num_object_clones":0,"num_object_copies":14114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3518,"num_write_kb":14072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7082","reported_seq":7213,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.385051+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:40.385051+0000","last_peered":"2024-09-15T21:22:40.385051+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:40.385051+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28987875,"num_objects":7078,"num_object_clones":0,"num_object_copies":14156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7078,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3758,"num_write_kb":15032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3002},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7169","reported_seq":7298,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.350338+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:40.350338+0000","last_peered":"2024-09-15T21:22:40.350338+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:40.350338+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7169,"log_dups_size":0,"ondisk_log_size":7169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29340614,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3480,"num_write_kb":13920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3004},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7247","reported_seq":7371,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:40.374291+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:40.374291+0000","last_peered":"2024-09-15T21:22:40.374291+0000","last_clean":"2024-09-15T21:22:40.374291+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:40.374291+0000","last_undegraded":"2024-09-15T21:22:40.374291+0000","last_fullsized":"2024-09-15T21:22:40.374291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7247,"log_dups_size":0,"ondisk_log_size":7247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29675520,"num_objects":7245,"num_object_clones":0,"num_object_copies":14490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4019,"num_write_kb":16076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7166","reported_seq":7287,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.299800+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:43.299800+0000","last_peered":"2024-09-15T21:22:43.299800+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:43.299800+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7166,"log_dups_size":0,"ondisk_log_size":7166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29323747,"num_objects":7160,"num_object_clones":0,"num_object_copies":14320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5098,"num_write_kb":20392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6973","reported_seq":7099,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.387660+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:40.387660+0000","last_peered":"2024-09-15T21:22:40.387660+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:40.387660+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6973,"log_dups_size":0,"ondisk_log_size":6973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28528640,"num_objects":6965,"num_object_clones":0,"num_object_copies":13930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3646,"num_write_kb":14584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2945},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7189","reported_seq":7303,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.211290+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:43.211290+0000","last_peered":"2024-09-15T21:22:43.211290+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:43.211290+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7189,"log_dups_size":0,"ondisk_log_size":7189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29426147,"num_objects":7185,"num_object_clones":0,"num_object_copies":14370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5048,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7254","reported_seq":11916,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:43.312333+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:43.312333+0000","last_peered":"2024-09-15T21:22:43.312333+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:43.312333+0000","last_undegraded":"2024-09-15T21:22:43.312333+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":7248,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1892,"num_objects_unfound":0,"num_objects_dirty":7248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4947,"num_write_kb":19788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2272,"num_bytes_recovered":9297920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1892},{"shards":"2","objects":5356}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7100","reported_seq":7229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.419202+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:40.419202+0000","last_peered":"2024-09-15T21:22:40.419202+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:40.419202+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7100,"log_dups_size":0,"ondisk_log_size":7100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29073408,"num_objects":7098,"num_object_clones":0,"num_object_copies":14196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4044,"num_write_kb":16176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3004},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7116","reported_seq":7244,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.395746+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:40.395746+0000","last_peered":"2024-09-15T21:22:40.395746+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:40.395746+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7116,"log_dups_size":0,"ondisk_log_size":7116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29114368,"num_objects":7108,"num_object_clones":0,"num_object_copies":14216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4025,"num_write_kb":16100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2969},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6952","reported_seq":7083,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.395860+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:40.395860+0000","last_peered":"2024-09-15T21:22:40.395860+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:40.395860+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6952,"log_dups_size":0,"ondisk_log_size":6952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28459008,"num_objects":6948,"num_object_clones":0,"num_object_copies":13896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3436,"num_write_kb":13744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2866},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7082","reported_seq":7216,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.419869+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:40.419869+0000","last_peered":"2024-09-15T21:22:40.419869+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:40.419869+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28959686,"num_objects":7072,"num_object_clones":0,"num_object_copies":14144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3786,"num_write_kb":15144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2925},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":468030174,"num_objects":114274,"num_object_clones":0,"num_object_copies":228548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1892,"num_objects_unfound":0,"num_objects_dirty":114274,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":74139,"num_write_kb":296556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2273,"num_bytes_recovered":9302016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":944869376,"data_stored":944721243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114346,"ondisk_log_size":114346,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546696,"kb_used_data":362296,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93825144,"statfs":{"total":96636764160,"available":96076947456,"internally_reserved":0,"allocated":370991104,"data_stored":370729090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":188804777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672608,"kb_used_data":406352,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93699232,"statfs":{"total":96636764160,"available":95948013568,"internally_reserved":0,"allocated":416104448,"data_stored":415838357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,3,5,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306740,"kb_used_data":152868,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94065100,"statfs":{"total":96636764160,"available":96322662400,"internally_reserved":0,"allocated":156536832,"data_stored":156230998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":157544111},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":416604160,"data_stored":416568030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":157118464,"data_stored":157042591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":371146752,"data_stored":371110622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:45.007 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:45.586 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:45.587 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:45.609 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2024-09-15T21:22:44.717772+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":468030193,"num_objects":114276,"num_object_clones":0,"num_object_copies":228552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1892,"num_objects_unfound":0,"num_objects_dirty":114276,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":74141,"num_write_kb":296558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2274,"num_bytes_recovered":9302035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114349,"ondisk_log_size":114349,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1526044,"kb_used_data":921516,"kb_used_omap":12,"kb_used_meta":604467,"kb_avail":281589476,"statfs":{"total":289910292480,"available":288347623424,"internally_reserved":0,"allocated":943632384,"data_stored":942798445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":618974211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,4,8,9,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24924160,"num_objects":6085,"num_object_clones":0,"num_object_copies":12170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-425,"num_objects_unfound":0,"num_objects_dirty":6085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6085,"num_write_kb":24340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":427,"num_bytes_recovered":1740800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007558"},"pg_stats":[{"pgid":"2.f","version":"43'7269","reported_seq":7340,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.418292+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:40.418292+0000","last_peered":"2024-09-15T21:22:40.418292+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:40.418292+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7269,"log_dups_size":0,"ondisk_log_size":7269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29773824,"num_objects":7269,"num_object_clones":0,"num_object_copies":14538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7269,"num_write_kb":29076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3137},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7217","reported_seq":7317,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.279839+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:43.279839+0000","last_peered":"2024-09-15T21:22:43.279839+0000","last_clean":"2024-09-15T21:22:43.279839+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:43.279839+0000","last_undegraded":"2024-09-15T21:22:43.279839+0000","last_fullsized":"2024-09-15T21:22:43.279839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7217,"log_dups_size":0,"ondisk_log_size":7217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29537222,"num_objects":7213,"num_object_clones":0,"num_object_copies":14426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7213,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5498,"num_write_kb":21992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7377","reported_seq":7496,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.316930+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:43.316930+0000","last_peered":"2024-09-15T21:22:43.316930+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:43.316930+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7377,"log_dups_size":0,"ondisk_log_size":7377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30199808,"num_objects":7373,"num_object_clones":0,"num_object_copies":14746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5477,"num_write_kb":21908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3106}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7090","reported_seq":7156,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.273555+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:43.273555+0000","last_peered":"2024-09-15T21:22:43.273555+0000","last_clean":"2024-09-15T21:22:43.273555+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:43.273555+0000","last_undegraded":"2024-09-15T21:22:43.273555+0000","last_fullsized":"2024-09-15T21:22:43.273555+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7090,"log_dups_size":0,"ondisk_log_size":7090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29040640,"num_objects":7090,"num_object_clones":0,"num_object_copies":14180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7090,"num_write_kb":28360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7063","reported_seq":7191,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:40.364349+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:40.364349+0000","last_peered":"2024-09-15T21:22:40.364349+0000","last_clean":"2024-09-15T21:22:40.364349+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:40.364349+0000","last_undegraded":"2024-09-15T21:22:40.364349+0000","last_fullsized":"2024-09-15T21:22:40.364349+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7063,"log_dups_size":0,"ondisk_log_size":7063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28901859,"num_objects":7057,"num_object_clones":0,"num_object_copies":14114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3518,"num_write_kb":14072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7082","reported_seq":7213,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.385051+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:40.385051+0000","last_peered":"2024-09-15T21:22:40.385051+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:40.385051+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28987875,"num_objects":7078,"num_object_clones":0,"num_object_copies":14156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7078,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3758,"num_write_kb":15032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3002},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7169","reported_seq":7298,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.350338+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:40.350338+0000","last_peered":"2024-09-15T21:22:40.350338+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:40.350338+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7169,"log_dups_size":0,"ondisk_log_size":7169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29340614,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3480,"num_write_kb":13920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3004},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7247","reported_seq":7371,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:40.374291+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:40.374291+0000","last_peered":"2024-09-15T21:22:40.374291+0000","last_clean":"2024-09-15T21:22:40.374291+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:40.374291+0000","last_undegraded":"2024-09-15T21:22:40.374291+0000","last_fullsized":"2024-09-15T21:22:40.374291+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7247,"log_dups_size":0,"ondisk_log_size":7247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29675520,"num_objects":7245,"num_object_clones":0,"num_object_copies":14490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4019,"num_write_kb":16076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7166","reported_seq":7287,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.299800+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:43.299800+0000","last_peered":"2024-09-15T21:22:43.299800+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:43.299800+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7166,"log_dups_size":0,"ondisk_log_size":7166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29323747,"num_objects":7160,"num_object_clones":0,"num_object_copies":14320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5098,"num_write_kb":20392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6973","reported_seq":7099,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.387660+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:40.387660+0000","last_peered":"2024-09-15T21:22:40.387660+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:40.387660+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":6973,"log_dups_size":0,"ondisk_log_size":6973,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28528640,"num_objects":6965,"num_object_clones":0,"num_object_copies":13930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3646,"num_write_kb":14584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2945},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7189","reported_seq":7303,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.211290+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:43.211290+0000","last_peered":"2024-09-15T21:22:43.211290+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:43.211290+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7189,"log_dups_size":0,"ondisk_log_size":7189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29426147,"num_objects":7185,"num_object_clones":0,"num_object_copies":14370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5048,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7254","reported_seq":11916,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:43.312333+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:43.312333+0000","last_peered":"2024-09-15T21:22:43.312333+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:43.312333+0000","last_undegraded":"2024-09-15T21:22:43.312333+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":7248,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1892,"num_objects_unfound":0,"num_objects_dirty":7248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4947,"num_write_kb":19788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2272,"num_bytes_recovered":9297920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1892},{"shards":"2","objects":5356}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7100","reported_seq":7229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.419202+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:40.419202+0000","last_peered":"2024-09-15T21:22:40.419202+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:40.419202+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7100,"log_dups_size":0,"ondisk_log_size":7100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29073408,"num_objects":7098,"num_object_clones":0,"num_object_copies":14196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7098,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4044,"num_write_kb":16176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3004},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7116","reported_seq":7244,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.395746+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:40.395746+0000","last_peered":"2024-09-15T21:22:40.395746+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:40.395746+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7116,"log_dups_size":0,"ondisk_log_size":7116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29114368,"num_objects":7108,"num_object_clones":0,"num_object_copies":14216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4025,"num_write_kb":16100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2969},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6952","reported_seq":7083,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.395860+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:40.395860+0000","last_peered":"2024-09-15T21:22:40.395860+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:40.395860+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":6952,"log_dups_size":0,"ondisk_log_size":6952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28459008,"num_objects":6948,"num_object_clones":0,"num_object_copies":13896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3436,"num_write_kb":13744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2866},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7082","reported_seq":7216,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:40.419869+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:40.419869+0000","last_peered":"2024-09-15T21:22:40.419869+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:40.419869+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28959686,"num_objects":7072,"num_object_clones":0,"num_object_copies":14144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3786,"num_write_kb":15144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2925},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":468030174,"num_objects":114274,"num_object_clones":0,"num_object_copies":228548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1892,"num_objects_unfound":0,"num_objects_dirty":114274,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":74139,"num_write_kb":296556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2273,"num_bytes_recovered":9302016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":944869376,"data_stored":944721243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114346,"ondisk_log_size":114346,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546696,"kb_used_data":362296,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93825144,"statfs":{"total":96636764160,"available":96076947456,"internally_reserved":0,"allocated":370991104,"data_stored":370729090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":188804777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672608,"kb_used_data":406352,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93699232,"statfs":{"total":96636764160,"available":95948013568,"internally_reserved":0,"allocated":416104448,"data_stored":415838357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,3,5,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306740,"kb_used_data":152868,"kb_used_omap":4,"kb_used_meta":153851,"kb_avail":94065100,"statfs":{"total":96636764160,"available":96322662400,"internally_reserved":0,"allocated":156536832,"data_stored":156230998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":157544111},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":416604160,"data_stored":416568030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":157118464,"data_stored":157042591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":371146752,"data_stored":371110622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:45.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:45.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'7269", 'reported_seq': 7340, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:40.418292+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:22:40.418292+0000', 'last_peered': '2024-09-15T21:22:40.418292+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:22:40.418292+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 7269, 'log_dups_size': 0, 'ondisk_log_size': 7269, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29773824, 'num_objects': 7269, 'num_object_clones': 0, 'num_object_copies': 14538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7269, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7269, 'num_write_kb': 29076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3137}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:45.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:22:45.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'7377", 'reported_seq': 7496, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:43.316930+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:22:43.316930+0000', 'last_peered': '2024-09-15T21:22:43.316930+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:22:43.316930+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 7377, 'log_dups_size': 0, 'ondisk_log_size': 7377, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30199808, 'num_objects': 7373, 'num_object_clones': 0, 'num_object_copies': 14746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7373, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5477, 'num_write_kb': 21908, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 3106}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:45.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:45.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'7082", 'reported_seq': 7213, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:40.385051+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:22:40.385051+0000', 'last_peered': '2024-09-15T21:22:40.385051+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:22:40.385051+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 7082, 'log_dups_size': 0, 'ondisk_log_size': 7082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28987875, 'num_objects': 7078, 'num_object_clones': 0, 'num_object_copies': 14156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7078, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3758, 'num_write_kb': 15032, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3002}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:45.611 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:45.611 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'7169", 'reported_seq': 7298, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:40.350338+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:22:40.350338+0000', 'last_peered': '2024-09-15T21:22:40.350338+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:22:40.350338+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 7169, 'log_dups_size': 0, 'ondisk_log_size': 7169, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29340614, 'num_objects': 7165, 'num_object_clones': 0, 'num_object_copies': 14330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3480, 'num_write_kb': 13920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3004}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:45.612 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:45.612 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7166", 'reported_seq': 7287, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:43.299800+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:22:43.299800+0000', 'last_peered': '2024-09-15T21:22:43.299800+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:22:43.299800+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 7166, 'log_dups_size': 0, 'ondisk_log_size': 7166, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29323747, 'num_objects': 7160, 'num_object_clones': 0, 'num_object_copies': 14320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7160, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5098, 'num_write_kb': 20392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 3073}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:45.612 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:45.612 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'6973", 'reported_seq': 7099, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:40.387660+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:22:40.387660+0000', 'last_peered': '2024-09-15T21:22:40.387660+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:22:40.387660+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 6973, 'log_dups_size': 0, 'ondisk_log_size': 6973, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28528640, 'num_objects': 6965, 'num_object_clones': 0, 'num_object_copies': 13930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6965, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3646, 'num_write_kb': 14584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2945}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:45.612 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:45.612 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7189", 'reported_seq': 7303, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:43.211290+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:22:43.211290+0000', 'last_peered': '2024-09-15T21:22:43.211290+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:22:43.211290+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 7189, 'log_dups_size': 0, 'ondisk_log_size': 7189, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29426147, 'num_objects': 7185, 'num_object_clones': 0, 'num_object_copies': 14370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7185, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5048, 'num_write_kb': 20192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 3095}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:45.612 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:45.612 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'7254", 'reported_seq': 11916, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:22:43.312333+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:22:43.312333+0000', 'last_peered': '2024-09-15T21:22:43.312333+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:22:43.312333+0000', 'last_undegraded': '2024-09-15T21:22:43.312333+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 7254, 'log_dups_size': 0, 'ondisk_log_size': 7254, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29687808, 'num_objects': 7248, 'num_object_clones': 0, 'num_object_copies': 14496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1892, 'num_objects_unfound': 0, 'num_objects_dirty': 7248, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4947, 'num_write_kb': 19788, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2272, 'num_bytes_recovered': 9297920, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1892}, {'shards': '2', 'objects': 5356}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:45.612 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:45.612 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'7100", 'reported_seq': 7229, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:40.419202+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:22:40.419202+0000', 'last_peered': '2024-09-15T21:22:40.419202+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:22:40.419202+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 7100, 'log_dups_size': 0, 'ondisk_log_size': 7100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29073408, 'num_objects': 7098, 'num_object_clones': 0, 'num_object_copies': 14196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7098, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4044, 'num_write_kb': 16176, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3004}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:45.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:45.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'7116", 'reported_seq': 7244, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:40.395746+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:22:40.395746+0000', 'last_peered': '2024-09-15T21:22:40.395746+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:22:40.395746+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 7116, 'log_dups_size': 0, 'ondisk_log_size': 7116, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29114368, 'num_objects': 7108, 'num_object_clones': 0, 'num_object_copies': 14216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7108, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4025, 'num_write_kb': 16100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2969}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:45.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:45.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'6952", 'reported_seq': 7083, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:40.395860+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:22:40.395860+0000', 'last_peered': '2024-09-15T21:22:40.395860+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:22:40.395860+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 6952, 'log_dups_size': 0, 'ondisk_log_size': 6952, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28459008, 'num_objects': 6948, 'num_object_clones': 0, 'num_object_copies': 13896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6948, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3436, 'num_write_kb': 13744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2866}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:45.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:22:45.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'7082", 'reported_seq': 7216, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:40.419869+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:22:40.419869+0000', 'last_peered': '2024-09-15T21:22:40.419869+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:22:40.419869+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 7082, 'log_dups_size': 0, 'ondisk_log_size': 7082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28959686, 'num_objects': 7072, 'num_object_clones': 0, 'num_object_copies': 14144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7072, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 3786, 'num_write_kb': 15144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2925}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:45.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:45.748 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 134 20 116414 116394 3.39256 1.83594 0.0572384 0.023021 2024-09-15T21:22:46.288 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:46.288 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/228552 objects degraded (19.824%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":218,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":114276,"data_bytes":468030193,"bytes_used":1562669056,"bytes_avail":288347623424,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":228552,"degraded_ratio":0.19824372571668591,"misplaced_objects":1892,"misplaced_total":228552,"misplaced_ratio":0.0082782036473100212,"recovering_objects_per_sec":35,"recovering_bytes_per_sec":144975,"recovering_keys_per_sec":0,"num_objects_recovered":427,"num_bytes_recovered":1740800,"num_keys_recovered":0,"write_bytes_sec":2075705,"read_op_per_sec":0,"write_op_per_sec":506},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:46.312 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:46.312 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:46.748 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 135 20 116878 116858 3.38086 1.8125 0.0396128 0.023101 2024-09-15T21:22:46.847 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:46.848 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:46.869 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-15T21:22:46.718298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":474624753,"num_objects":115886,"num_object_clones":0,"num_object_copies":231772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1892,"num_objects_unfound":0,"num_objects_dirty":115886,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":75751,"num_write_kb":302998,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2274,"num_bytes_recovered":9302035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115959,"ondisk_log_size":115959,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1568608,"kb_used_data":937712,"kb_used_omap":12,"kb_used_meta":630835,"kb_avail":281546912,"statfs":{"total":289910292480,"available":288304037888,"internally_reserved":0,"allocated":960217088,"data_stored":959383149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":645975043},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,3,13,14,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27488256,"num_objects":6711,"num_object_clones":0,"num_object_copies":13422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-281,"num_objects_unfound":0,"num_objects_dirty":6711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6711,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":1150976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004330"},"pg_stats":[{"pgid":"2.f","version":"43'7432","reported_seq":7500,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.379320+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:45.379320+0000","last_peered":"2024-09-15T21:22:45.379320+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:45.379320+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7432,"log_dups_size":0,"ondisk_log_size":7432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30441472,"num_objects":7432,"num_object_clones":0,"num_object_copies":14864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7432,"num_write_kb":29728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3300},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7217","reported_seq":7317,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.279839+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:43.279839+0000","last_peered":"2024-09-15T21:22:43.279839+0000","last_clean":"2024-09-15T21:22:43.279839+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:43.279839+0000","last_undegraded":"2024-09-15T21:22:43.279839+0000","last_fullsized":"2024-09-15T21:22:43.279839+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7217,"log_dups_size":0,"ondisk_log_size":7217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29537222,"num_objects":7213,"num_object_clones":0,"num_object_copies":14426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7213,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5498,"num_write_kb":21992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7377","reported_seq":7496,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.316930+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:43.316930+0000","last_peered":"2024-09-15T21:22:43.316930+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:43.316930+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7377,"log_dups_size":0,"ondisk_log_size":7377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30199808,"num_objects":7373,"num_object_clones":0,"num_object_copies":14746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5477,"num_write_kb":21908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3106}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7090","reported_seq":7156,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:43.273555+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:43.273555+0000","last_peered":"2024-09-15T21:22:43.273555+0000","last_clean":"2024-09-15T21:22:43.273555+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:43.273555+0000","last_undegraded":"2024-09-15T21:22:43.273555+0000","last_fullsized":"2024-09-15T21:22:43.273555+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7090,"log_dups_size":0,"ondisk_log_size":7090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29040640,"num_objects":7090,"num_object_clones":0,"num_object_copies":14180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7090,"num_write_kb":28360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7238","reported_seq":7366,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:45.399167+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:45.399167+0000","last_peered":"2024-09-15T21:22:45.399167+0000","last_clean":"2024-09-15T21:22:45.399167+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:45.399167+0000","last_undegraded":"2024-09-15T21:22:45.399167+0000","last_fullsized":"2024-09-15T21:22:45.399167+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7238,"log_dups_size":0,"ondisk_log_size":7238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29618659,"num_objects":7232,"num_object_clones":0,"num_object_copies":14464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3693,"num_write_kb":14772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7254","reported_seq":7385,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.395378+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:45.395378+0000","last_peered":"2024-09-15T21:22:45.395378+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:45.395378+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29692387,"num_objects":7250,"num_object_clones":0,"num_object_copies":14500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3930,"num_write_kb":15720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3174},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7308","reported_seq":7439,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.452040+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:45.452040+0000","last_peered":"2024-09-15T21:22:45.452040+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:45.452040+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7308,"log_dups_size":0,"ondisk_log_size":7308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29909958,"num_objects":7304,"num_object_clones":0,"num_object_copies":14608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3619,"num_write_kb":14476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3143},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7398","reported_seq":7522,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:45.395439+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:45.395439+0000","last_peered":"2024-09-15T21:22:45.395439+0000","last_clean":"2024-09-15T21:22:45.395439+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:45.395439+0000","last_undegraded":"2024-09-15T21:22:45.395439+0000","last_fullsized":"2024-09-15T21:22:45.395439+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7398,"log_dups_size":0,"ondisk_log_size":7398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294016,"num_objects":7396,"num_object_clones":0,"num_object_copies":14792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4170,"num_write_kb":16680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7166","reported_seq":7287,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.299800+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:43.299800+0000","last_peered":"2024-09-15T21:22:43.299800+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:43.299800+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7166,"log_dups_size":0,"ondisk_log_size":7166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29323747,"num_objects":7160,"num_object_clones":0,"num_object_copies":14320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5098,"num_write_kb":20392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7122","reported_seq":7248,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.452383+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:45.452383+0000","last_peered":"2024-09-15T21:22:45.452383+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:45.452383+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7122,"log_dups_size":0,"ondisk_log_size":7122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29138944,"num_objects":7114,"num_object_clones":0,"num_object_copies":14228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3795,"num_write_kb":15180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3094},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7189","reported_seq":7303,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:43.211290+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:43.211290+0000","last_peered":"2024-09-15T21:22:43.211290+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:43.211290+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7189,"log_dups_size":0,"ondisk_log_size":7189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29426147,"num_objects":7185,"num_object_clones":0,"num_object_copies":14370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7185,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5048,"num_write_kb":20192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3095}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7254","reported_seq":11916,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:43.312333+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:43.312333+0000","last_peered":"2024-09-15T21:22:43.312333+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:43.312333+0000","last_undegraded":"2024-09-15T21:22:43.312333+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":7248,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1892,"num_objects_unfound":0,"num_objects_dirty":7248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4947,"num_write_kb":19788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2272,"num_bytes_recovered":9297920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1892},{"shards":"2","objects":5356}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7258","reported_seq":7387,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.448418+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:45.448418+0000","last_peered":"2024-09-15T21:22:45.448418+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:45.448418+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7258,"log_dups_size":0,"ondisk_log_size":7258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29720576,"num_objects":7256,"num_object_clones":0,"num_object_copies":14512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4202,"num_write_kb":16808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3162},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7291","reported_seq":7419,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.442989+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:45.442989+0000","last_peered":"2024-09-15T21:22:45.442989+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:45.442989+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7291,"log_dups_size":0,"ondisk_log_size":7291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29831168,"num_objects":7283,"num_object_clones":0,"num_object_copies":14566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4200,"num_write_kb":16800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3144},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7114","reported_seq":7244,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.430410+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:45.430410+0000","last_peered":"2024-09-15T21:22:45.430410+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:45.430410+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7114,"log_dups_size":0,"ondisk_log_size":7114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29122560,"num_objects":7110,"num_object_clones":0,"num_object_copies":14220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3598,"num_write_kb":14392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3028},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7248","reported_seq":7382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.384453+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:45.384453+0000","last_peered":"2024-09-15T21:22:45.384453+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:45.384453+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7248,"log_dups_size":0,"ondisk_log_size":7248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29639622,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3952,"num_write_kb":15808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3091},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":474624734,"num_objects":115884,"num_object_clones":0,"num_object_copies":231768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1892,"num_objects_unfound":0,"num_objects_dirty":115884,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":75749,"num_write_kb":302996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2273,"num_bytes_recovered":9302016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":961351680,"data_stored":961203547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115956,"ondisk_log_size":115956,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546696,"kb_used_data":362296,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93825144,"statfs":{"total":96636764160,"available":96076947456,"internally_reserved":0,"allocated":370991104,"data_stored":370729090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":188804777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684692,"kb_used_data":414276,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93687148,"statfs":{"total":96636764160,"available":95935639552,"internally_reserved":0,"allocated":424218624,"data_stored":423952533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276885163},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,7,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":337220,"kb_used_data":161140,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":94034620,"statfs":{"total":96636764160,"available":96291450880,"internally_reserved":0,"allocated":165007360,"data_stored":164701526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,6,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":424620032,"data_stored":424583902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165584896,"data_stored":165509023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":371146752,"data_stored":371110622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:47.748 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 136 20 117397 117377 3.3709 2.02734 0.0299523 0.0231683 2024-09-15T21:22:48.748 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 137 20 117937 117917 3.36169 2.10938 0.0369807 0.0232318 2024-09-15T21:22:49.748 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 138 20 118462 118442 3.35219 2.05078 0.029055 0.0232981 2024-09-15T21:22:49.871 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:50.410 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:50.411 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:50.431 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-15T21:22:48.721258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":478507761,"num_objects":116834,"num_object_clones":0,"num_object_copies":233668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":116834,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":76699,"num_write_kb":306790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2430,"num_bytes_recovered":9941011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116907,"ondisk_log_size":116907,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1561204,"kb_used_data":942788,"kb_used_omap":12,"kb_used_meta":618355,"kb_avail":281554316,"statfs":{"total":289910292480,"available":288311619584,"internally_reserved":0,"allocated":965414912,"data_stored":964580973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":633195523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,5,17,13,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24776704,"num_objects":6049,"num_object_clones":0,"num_object_copies":12098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-437,"num_objects_unfound":0,"num_objects_dirty":6049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6049,"num_write_kb":24196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":439,"num_bytes_recovered":1789952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006816"},"pg_stats":[{"pgid":"2.f","version":"43'7432","reported_seq":7500,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.379320+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:45.379320+0000","last_peered":"2024-09-15T21:22:45.379320+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:45.379320+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7432,"log_dups_size":0,"ondisk_log_size":7432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30441472,"num_objects":7432,"num_object_clones":0,"num_object_copies":14864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7432,"num_write_kb":29728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3300},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7365","reported_seq":7465,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.338846+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:48.338846+0000","last_peered":"2024-09-15T21:22:48.338846+0000","last_clean":"2024-09-15T21:22:48.338846+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:48.338846+0000","last_undegraded":"2024-09-15T21:22:48.338846+0000","last_fullsized":"2024-09-15T21:22:48.338846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7365,"log_dups_size":0,"ondisk_log_size":7365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30143430,"num_objects":7361,"num_object_clones":0,"num_object_copies":14722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7361,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5646,"num_write_kb":22584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7539","reported_seq":7659,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.339349+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:48.339349+0000","last_peered":"2024-09-15T21:22:48.339349+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:48.339349+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7539,"log_dups_size":0,"ondisk_log_size":7539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30863360,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5639,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3268}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7235","reported_seq":7301,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.282931+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:48.282931+0000","last_peered":"2024-09-15T21:22:48.282931+0000","last_clean":"2024-09-15T21:22:48.282931+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:48.282931+0000","last_undegraded":"2024-09-15T21:22:48.282931+0000","last_fullsized":"2024-09-15T21:22:48.282931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7235,"log_dups_size":0,"ondisk_log_size":7235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29634560,"num_objects":7235,"num_object_clones":0,"num_object_copies":14470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7235,"num_write_kb":28940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7238","reported_seq":7366,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:45.399167+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:45.399167+0000","last_peered":"2024-09-15T21:22:45.399167+0000","last_clean":"2024-09-15T21:22:45.399167+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:45.399167+0000","last_undegraded":"2024-09-15T21:22:45.399167+0000","last_fullsized":"2024-09-15T21:22:45.399167+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7238,"log_dups_size":0,"ondisk_log_size":7238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29618659,"num_objects":7232,"num_object_clones":0,"num_object_copies":14464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3693,"num_write_kb":14772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7254","reported_seq":7385,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.395378+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:45.395378+0000","last_peered":"2024-09-15T21:22:45.395378+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:45.395378+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7254,"log_dups_size":0,"ondisk_log_size":7254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29692387,"num_objects":7250,"num_object_clones":0,"num_object_copies":14500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3930,"num_write_kb":15720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3174},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7308","reported_seq":7439,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.452040+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:45.452040+0000","last_peered":"2024-09-15T21:22:45.452040+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:45.452040+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7308,"log_dups_size":0,"ondisk_log_size":7308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29909958,"num_objects":7304,"num_object_clones":0,"num_object_copies":14608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3619,"num_write_kb":14476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3143},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7398","reported_seq":7522,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:45.395439+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:45.395439+0000","last_peered":"2024-09-15T21:22:45.395439+0000","last_clean":"2024-09-15T21:22:45.395439+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:45.395439+0000","last_undegraded":"2024-09-15T21:22:45.395439+0000","last_fullsized":"2024-09-15T21:22:45.395439+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7398,"log_dups_size":0,"ondisk_log_size":7398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294016,"num_objects":7396,"num_object_clones":0,"num_object_copies":14792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4170,"num_write_kb":16680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7334","reported_seq":7456,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.319254+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:48.319254+0000","last_peered":"2024-09-15T21:22:48.319254+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:48.319254+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7334,"log_dups_size":0,"ondisk_log_size":7334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30011875,"num_objects":7328,"num_object_clones":0,"num_object_copies":14656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7328,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5266,"num_write_kb":21064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3241}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7122","reported_seq":7248,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.452383+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:45.452383+0000","last_peered":"2024-09-15T21:22:45.452383+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:45.452383+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7122,"log_dups_size":0,"ondisk_log_size":7122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29138944,"num_objects":7114,"num_object_clones":0,"num_object_copies":14228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3795,"num_write_kb":15180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3094},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7355","reported_seq":7469,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.312608+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:48.312608+0000","last_peered":"2024-09-15T21:22:48.312608+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:48.312608+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7355,"log_dups_size":0,"ondisk_log_size":7355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30106083,"num_objects":7351,"num_object_clones":0,"num_object_copies":14702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5214,"num_write_kb":20856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3261}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7413","reported_seq":12387,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:48.322329+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:48.322329+0000","last_peered":"2024-09-15T21:22:48.322329+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:48.322329+0000","last_undegraded":"2024-09-15T21:22:48.322329+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7413,"log_dups_size":0,"ondisk_log_size":7413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30339072,"num_objects":7407,"num_object_clones":0,"num_object_copies":14814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":7407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5106,"num_write_kb":20424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2428,"num_bytes_recovered":9936896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1736},{"shards":"2","objects":5671}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7258","reported_seq":7387,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.448418+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:45.448418+0000","last_peered":"2024-09-15T21:22:45.448418+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:45.448418+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7258,"log_dups_size":0,"ondisk_log_size":7258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29720576,"num_objects":7256,"num_object_clones":0,"num_object_copies":14512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4202,"num_write_kb":16808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3162},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7291","reported_seq":7419,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.442989+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:45.442989+0000","last_peered":"2024-09-15T21:22:45.442989+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:45.442989+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7291,"log_dups_size":0,"ondisk_log_size":7291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29831168,"num_objects":7283,"num_object_clones":0,"num_object_copies":14566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4200,"num_write_kb":16800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3144},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7114","reported_seq":7244,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.430410+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:45.430410+0000","last_peered":"2024-09-15T21:22:45.430410+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:45.430410+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7114,"log_dups_size":0,"ondisk_log_size":7114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29122560,"num_objects":7110,"num_object_clones":0,"num_object_copies":14220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3598,"num_write_kb":14392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3028},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7248","reported_seq":7382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:45.384453+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:45.384453+0000","last_peered":"2024-09-15T21:22:45.384453+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:45.384453+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7248,"log_dups_size":0,"ondisk_log_size":7248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29639622,"num_objects":7238,"num_object_clones":0,"num_object_copies":14476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7238,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3952,"num_write_kb":15808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3091},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":478507742,"num_objects":116832,"num_object_clones":0,"num_object_copies":233664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":116832,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":76697,"num_write_kb":306788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2429,"num_bytes_recovered":9940992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":966606848,"data_stored":966458715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":116904,"ondisk_log_size":116904,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539292,"kb_used_data":367372,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93832548,"statfs":{"total":96636764160,"available":96084529152,"internally_reserved":0,"allocated":376188928,"data_stored":375926914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":176025257},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":684692,"kb_used_data":414276,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93687148,"statfs":{"total":96636764160,"available":95935639552,"internally_reserved":0,"allocated":424218624,"data_stored":423952533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276885163},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,7,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":337220,"kb_used_data":161140,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":94034620,"statfs":{"total":96636764160,"available":96291450880,"internally_reserved":0,"allocated":165007360,"data_stored":164701526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,6,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":424620032,"data_stored":424583902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165584896,"data_stored":165509023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376401920,"data_stored":376365790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:50.432 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:50.748 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 139 20 118939 118919 3.34148 1.86328 0.0206934 0.0233723 2024-09-15T21:22:50.999 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:50.999 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:51.022 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-15T21:22:50.721746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":484942577,"num_objects":118405,"num_object_clones":0,"num_object_copies":236810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":118405,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":78270,"num_write_kb":313074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2430,"num_bytes_recovered":9941011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118478,"ondisk_log_size":118478,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1572676,"kb_used_data":950100,"kb_used_omap":12,"kb_used_meta":622515,"kb_avail":281542844,"statfs":{"total":289910292480,"available":288299872256,"internally_reserved":0,"allocated":972902400,"data_stored":972068461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":637455363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,16,9,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27222016,"num_objects":6646,"num_object_clones":0,"num_object_copies":13292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-298,"num_objects_unfound":0,"num_objects_dirty":6646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6646,"num_write_kb":26584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1220608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006815"},"pg_stats":[{"pgid":"2.f","version":"43'7592","reported_seq":7661,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.507151+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:50.507151+0000","last_peered":"2024-09-15T21:22:50.507151+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:50.507151+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7592,"log_dups_size":0,"ondisk_log_size":7592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31096832,"num_objects":7592,"num_object_clones":0,"num_object_copies":15184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7592,"num_write_kb":30368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3460},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7365","reported_seq":7465,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.338846+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:48.338846+0000","last_peered":"2024-09-15T21:22:48.338846+0000","last_clean":"2024-09-15T21:22:48.338846+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:48.338846+0000","last_undegraded":"2024-09-15T21:22:48.338846+0000","last_fullsized":"2024-09-15T21:22:48.338846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7365,"log_dups_size":0,"ondisk_log_size":7365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30143430,"num_objects":7361,"num_object_clones":0,"num_object_copies":14722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7361,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5646,"num_write_kb":22584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7539","reported_seq":7659,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.339349+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:48.339349+0000","last_peered":"2024-09-15T21:22:48.339349+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:48.339349+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7539,"log_dups_size":0,"ondisk_log_size":7539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30863360,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5639,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3268}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7235","reported_seq":7301,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.282931+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:48.282931+0000","last_peered":"2024-09-15T21:22:48.282931+0000","last_clean":"2024-09-15T21:22:48.282931+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:48.282931+0000","last_undegraded":"2024-09-15T21:22:48.282931+0000","last_fullsized":"2024-09-15T21:22:48.282931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7235,"log_dups_size":0,"ondisk_log_size":7235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29634560,"num_objects":7235,"num_object_clones":0,"num_object_copies":14470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7235,"num_write_kb":28940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7399","reported_seq":7527,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:50.400568+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:50.400568+0000","last_peered":"2024-09-15T21:22:50.400568+0000","last_clean":"2024-09-15T21:22:50.400568+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:50.400568+0000","last_undegraded":"2024-09-15T21:22:50.400568+0000","last_fullsized":"2024-09-15T21:22:50.400568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30278115,"num_objects":7393,"num_object_clones":0,"num_object_copies":14786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3854,"num_write_kb":15416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7385","reported_seq":7516,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.436929+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:50.436929+0000","last_peered":"2024-09-15T21:22:50.436929+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:50.436929+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7385,"log_dups_size":0,"ondisk_log_size":7385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30228963,"num_objects":7381,"num_object_clones":0,"num_object_copies":14762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4061,"num_write_kb":16244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3305},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7466","reported_seq":7597,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.460813+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:50.460813+0000","last_peered":"2024-09-15T21:22:50.460813+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:50.460813+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7466,"log_dups_size":0,"ondisk_log_size":7466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30557126,"num_objects":7462,"num_object_clones":0,"num_object_copies":14924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3777,"num_write_kb":15108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3301},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7567","reported_seq":7691,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:50.455389+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:50.455389+0000","last_peered":"2024-09-15T21:22:50.455389+0000","last_clean":"2024-09-15T21:22:50.455389+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:50.455389+0000","last_undegraded":"2024-09-15T21:22:50.455389+0000","last_fullsized":"2024-09-15T21:22:50.455389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7567,"log_dups_size":0,"ondisk_log_size":7567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30986240,"num_objects":7565,"num_object_clones":0,"num_object_copies":15130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4339,"num_write_kb":17356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7334","reported_seq":7456,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.319254+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:48.319254+0000","last_peered":"2024-09-15T21:22:48.319254+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:48.319254+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7334,"log_dups_size":0,"ondisk_log_size":7334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30011875,"num_objects":7328,"num_object_clones":0,"num_object_copies":14656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7328,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5266,"num_write_kb":21064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3241}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7256","reported_seq":7382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.441826+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:50.441826+0000","last_peered":"2024-09-15T21:22:50.441826+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:50.441826+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7256,"log_dups_size":0,"ondisk_log_size":7256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":7248,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3929,"num_write_kb":15716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3228},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7355","reported_seq":7469,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.312608+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:48.312608+0000","last_peered":"2024-09-15T21:22:48.312608+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:48.312608+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7355,"log_dups_size":0,"ondisk_log_size":7355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30106083,"num_objects":7351,"num_object_clones":0,"num_object_copies":14702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5214,"num_write_kb":20856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3261}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7413","reported_seq":12387,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:48.322329+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:48.322329+0000","last_peered":"2024-09-15T21:22:48.322329+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:48.322329+0000","last_undegraded":"2024-09-15T21:22:48.322329+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7413,"log_dups_size":0,"ondisk_log_size":7413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30339072,"num_objects":7407,"num_object_clones":0,"num_object_copies":14814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":7407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5106,"num_write_kb":20424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2428,"num_bytes_recovered":9936896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1736},{"shards":"2","objects":5671}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7427","reported_seq":7555,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.436311+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:50.436311+0000","last_peered":"2024-09-15T21:22:50.436311+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:50.436311+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7427,"log_dups_size":0,"ondisk_log_size":7427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30412800,"num_objects":7425,"num_object_clones":0,"num_object_copies":14850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4371,"num_write_kb":17484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3331},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7429","reported_seq":7556,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.488423+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:50.488423+0000","last_peered":"2024-09-15T21:22:50.488423+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:50.488423+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7429,"log_dups_size":0,"ondisk_log_size":7429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30396416,"num_objects":7421,"num_object_clones":0,"num_object_copies":14842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4338,"num_write_kb":17352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3282},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7290","reported_seq":7421,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.432080+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:50.432080+0000","last_peered":"2024-09-15T21:22:50.432080+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:50.432080+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7290,"log_dups_size":0,"ondisk_log_size":7290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29843456,"num_objects":7286,"num_object_clones":0,"num_object_copies":14572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3774,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3204},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7423","reported_seq":7557,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.477306+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:50.477306+0000","last_peered":"2024-09-15T21:22:50.477306+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:50.477306+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7423,"log_dups_size":0,"ondisk_log_size":7423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30356422,"num_objects":7413,"num_object_clones":0,"num_object_copies":14826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7413,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4127,"num_write_kb":16508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3266},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":484942558,"num_objects":118403,"num_object_clones":0,"num_object_copies":236806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":118403,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":78268,"num_write_kb":313072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2429,"num_bytes_recovered":9940992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":974274560,"data_stored":974126427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118475,"ondisk_log_size":118475,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539292,"kb_used_data":367372,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93832548,"statfs":{"total":96636764160,"available":96084529152,"internally_reserved":0,"allocated":376188928,"data_stored":375926914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":176025257},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696164,"kb_used_data":421588,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93675676,"statfs":{"total":96636764160,"available":95923892224,"internally_reserved":0,"allocated":431706112,"data_stored":431440021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":337220,"kb_used_data":161140,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":94034620,"statfs":{"total":96636764160,"available":96291450880,"internally_reserved":0,"allocated":165007360,"data_stored":164701526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,6,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":432287744,"data_stored":432251614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165584896,"data_stored":165509023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376401920,"data_stored":376365790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:51.023 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:51.023 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'7592", 'reported_seq': 7661, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.507151+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:22:50.507151+0000', 'last_peered': '2024-09-15T21:22:50.507151+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:22:50.507151+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 7592, 'log_dups_size': 0, 'ondisk_log_size': 7592, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31096832, 'num_objects': 7592, 'num_object_clones': 0, 'num_object_copies': 15184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7592, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7592, 'num_write_kb': 30368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3460}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.023 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:22:51.023 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'7539", 'reported_seq': 7659, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:48.339349+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:22:48.339349+0000', 'last_peered': '2024-09-15T21:22:48.339349+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:22:48.339349+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 7539, 'log_dups_size': 0, 'ondisk_log_size': 7539, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30863360, 'num_objects': 7535, 'num_object_clones': 0, 'num_object_copies': 15070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7535, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5639, 'num_write_kb': 22556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 3268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:51.024 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:51.024 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'7385", 'reported_seq': 7516, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.436929+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:22:50.436929+0000', 'last_peered': '2024-09-15T21:22:50.436929+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:22:50.436929+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 7385, 'log_dups_size': 0, 'ondisk_log_size': 7385, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30228963, 'num_objects': 7381, 'num_object_clones': 0, 'num_object_copies': 14762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7381, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4061, 'num_write_kb': 16244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3305}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.024 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:51.024 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'7466", 'reported_seq': 7597, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.460813+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:22:50.460813+0000', 'last_peered': '2024-09-15T21:22:50.460813+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:22:50.460813+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 7466, 'log_dups_size': 0, 'ondisk_log_size': 7466, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30557126, 'num_objects': 7462, 'num_object_clones': 0, 'num_object_copies': 14924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7462, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3777, 'num_write_kb': 15108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3301}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.024 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:51.024 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7334", 'reported_seq': 7456, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:48.319254+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:22:48.319254+0000', 'last_peered': '2024-09-15T21:22:48.319254+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:22:48.319254+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 7334, 'log_dups_size': 0, 'ondisk_log_size': 7334, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30011875, 'num_objects': 7328, 'num_object_clones': 0, 'num_object_copies': 14656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7328, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5266, 'num_write_kb': 21064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 3241}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:51.024 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:51.024 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'7256", 'reported_seq': 7382, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.441826+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:22:50.441826+0000', 'last_peered': '2024-09-15T21:22:50.441826+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:22:50.441826+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 7256, 'log_dups_size': 0, 'ondisk_log_size': 7256, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29687808, 'num_objects': 7248, 'num_object_clones': 0, 'num_object_copies': 14496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7248, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3929, 'num_write_kb': 15716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3228}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.024 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:51.024 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7355", 'reported_seq': 7469, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:48.312608+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:22:48.312608+0000', 'last_peered': '2024-09-15T21:22:48.312608+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:22:48.312608+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 7355, 'log_dups_size': 0, 'ondisk_log_size': 7355, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30106083, 'num_objects': 7351, 'num_object_clones': 0, 'num_object_copies': 14702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7351, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5214, 'num_write_kb': 20856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 3261}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:51.025 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:51.025 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'7413", 'reported_seq': 12387, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:22:48.322329+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:22:48.322329+0000', 'last_peered': '2024-09-15T21:22:48.322329+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:22:48.322329+0000', 'last_undegraded': '2024-09-15T21:22:48.322329+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 7413, 'log_dups_size': 0, 'ondisk_log_size': 7413, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30339072, 'num_objects': 7407, 'num_object_clones': 0, 'num_object_copies': 14814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1736, 'num_objects_unfound': 0, 'num_objects_dirty': 7407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5106, 'num_write_kb': 20424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2428, 'num_bytes_recovered': 9936896, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1736}, {'shards': '2', 'objects': 5671}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:51.025 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:51.025 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'7427", 'reported_seq': 7555, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.436311+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:22:50.436311+0000', 'last_peered': '2024-09-15T21:22:50.436311+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:22:50.436311+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 7427, 'log_dups_size': 0, 'ondisk_log_size': 7427, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30412800, 'num_objects': 7425, 'num_object_clones': 0, 'num_object_copies': 14850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7425, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4371, 'num_write_kb': 17484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3331}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.025 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:51.025 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'7429", 'reported_seq': 7556, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.488423+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:22:50.488423+0000', 'last_peered': '2024-09-15T21:22:50.488423+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:22:50.488423+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 7429, 'log_dups_size': 0, 'ondisk_log_size': 7429, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30396416, 'num_objects': 7421, 'num_object_clones': 0, 'num_object_copies': 14842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7421, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4338, 'num_write_kb': 17352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3282}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.025 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:51.025 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'7290", 'reported_seq': 7421, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.432080+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:22:50.432080+0000', 'last_peered': '2024-09-15T21:22:50.432080+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:22:50.432080+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 7290, 'log_dups_size': 0, 'ondisk_log_size': 7290, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29843456, 'num_objects': 7286, 'num_object_clones': 0, 'num_object_copies': 14572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7286, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3774, 'num_write_kb': 15096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3204}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.026 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:22:51.026 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'7423", 'reported_seq': 7557, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.477306+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:22:50.477306+0000', 'last_peered': '2024-09-15T21:22:50.477306+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:22:50.477306+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 7423, 'log_dups_size': 0, 'ondisk_log_size': 7423, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30356422, 'num_objects': 7413, 'num_object_clones': 0, 'num_object_copies': 14826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7413, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4127, 'num_write_kb': 16508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3266}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:51.026 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:51.719 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:51.719 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/236810 objects degraded (19.133%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":223,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":118405,"data_bytes":484942577,"bytes_used":1610420224,"bytes_avail":288299872256,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":236810,"degraded_ratio":0.19133060259279591,"misplaced_objects":1736,"misplaced_total":236810,"misplaced_ratio":0.0073307715045817321,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":101659,"recovering_keys_per_sec":0,"num_objects_recovered":300,"num_bytes_recovered":1220608,"num_keys_recovered":0,"write_bytes_sec":2267213,"read_op_per_sec":0,"write_op_per_sec":553},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:51.744 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:51.744 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:51.749 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:2024-09-15T21:22:51.746471+0000 min lat: 0.00235499 max lat: 2.60418 avg lat: 0.0234484 2024-09-15T21:22:51.749 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:22:51.749 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 140 20 119401 119381 3.3305 1.80469 0.0184885 0.0234484 2024-09-15T21:22:52.301 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:52.304 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:52.327 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-15T21:22:50.721746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":484942577,"num_objects":118405,"num_object_clones":0,"num_object_copies":236810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":118405,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":78270,"num_write_kb":313074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2430,"num_bytes_recovered":9941011,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118478,"ondisk_log_size":118478,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1572676,"kb_used_data":950100,"kb_used_omap":12,"kb_used_meta":622515,"kb_avail":281542844,"statfs":{"total":289910292480,"available":288299872256,"internally_reserved":0,"allocated":972902400,"data_stored":972068461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":637455363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,16,9,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27222016,"num_objects":6646,"num_object_clones":0,"num_object_copies":13292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-298,"num_objects_unfound":0,"num_objects_dirty":6646,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6646,"num_write_kb":26584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":1220608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006815"},"pg_stats":[{"pgid":"2.f","version":"43'7592","reported_seq":7661,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.507151+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:50.507151+0000","last_peered":"2024-09-15T21:22:50.507151+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:50.507151+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7592,"log_dups_size":0,"ondisk_log_size":7592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31096832,"num_objects":7592,"num_object_clones":0,"num_object_copies":15184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7592,"num_write_kb":30368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3460},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7365","reported_seq":7465,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.338846+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:48.338846+0000","last_peered":"2024-09-15T21:22:48.338846+0000","last_clean":"2024-09-15T21:22:48.338846+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:48.338846+0000","last_undegraded":"2024-09-15T21:22:48.338846+0000","last_fullsized":"2024-09-15T21:22:48.338846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7365,"log_dups_size":0,"ondisk_log_size":7365,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30143430,"num_objects":7361,"num_object_clones":0,"num_object_copies":14722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7361,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5646,"num_write_kb":22584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7539","reported_seq":7659,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.339349+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:48.339349+0000","last_peered":"2024-09-15T21:22:48.339349+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:48.339349+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7539,"log_dups_size":0,"ondisk_log_size":7539,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30863360,"num_objects":7535,"num_object_clones":0,"num_object_copies":15070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5639,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3268}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7235","reported_seq":7301,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:48.282931+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:48.282931+0000","last_peered":"2024-09-15T21:22:48.282931+0000","last_clean":"2024-09-15T21:22:48.282931+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:48.282931+0000","last_undegraded":"2024-09-15T21:22:48.282931+0000","last_fullsized":"2024-09-15T21:22:48.282931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7235,"log_dups_size":0,"ondisk_log_size":7235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29634560,"num_objects":7235,"num_object_clones":0,"num_object_copies":14470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7235,"num_write_kb":28940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7399","reported_seq":7527,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:50.400568+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:50.400568+0000","last_peered":"2024-09-15T21:22:50.400568+0000","last_clean":"2024-09-15T21:22:50.400568+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:50.400568+0000","last_undegraded":"2024-09-15T21:22:50.400568+0000","last_fullsized":"2024-09-15T21:22:50.400568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30278115,"num_objects":7393,"num_object_clones":0,"num_object_copies":14786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3854,"num_write_kb":15416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7385","reported_seq":7516,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.436929+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:50.436929+0000","last_peered":"2024-09-15T21:22:50.436929+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:50.436929+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7385,"log_dups_size":0,"ondisk_log_size":7385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30228963,"num_objects":7381,"num_object_clones":0,"num_object_copies":14762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4061,"num_write_kb":16244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3305},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7466","reported_seq":7597,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.460813+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:50.460813+0000","last_peered":"2024-09-15T21:22:50.460813+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:50.460813+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7466,"log_dups_size":0,"ondisk_log_size":7466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30557126,"num_objects":7462,"num_object_clones":0,"num_object_copies":14924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3777,"num_write_kb":15108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3301},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7567","reported_seq":7691,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:50.455389+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:50.455389+0000","last_peered":"2024-09-15T21:22:50.455389+0000","last_clean":"2024-09-15T21:22:50.455389+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:50.455389+0000","last_undegraded":"2024-09-15T21:22:50.455389+0000","last_fullsized":"2024-09-15T21:22:50.455389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7567,"log_dups_size":0,"ondisk_log_size":7567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30986240,"num_objects":7565,"num_object_clones":0,"num_object_copies":15130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4339,"num_write_kb":17356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7334","reported_seq":7456,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.319254+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:48.319254+0000","last_peered":"2024-09-15T21:22:48.319254+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:48.319254+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7334,"log_dups_size":0,"ondisk_log_size":7334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30011875,"num_objects":7328,"num_object_clones":0,"num_object_copies":14656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7328,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5266,"num_write_kb":21064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3241}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7256","reported_seq":7382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.441826+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:50.441826+0000","last_peered":"2024-09-15T21:22:50.441826+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:50.441826+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7256,"log_dups_size":0,"ondisk_log_size":7256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":7248,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3929,"num_write_kb":15716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3228},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7355","reported_seq":7469,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:48.312608+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:48.312608+0000","last_peered":"2024-09-15T21:22:48.312608+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:48.312608+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7355,"log_dups_size":0,"ondisk_log_size":7355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30106083,"num_objects":7351,"num_object_clones":0,"num_object_copies":14702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5214,"num_write_kb":20856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3261}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7413","reported_seq":12387,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:48.322329+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:48.322329+0000","last_peered":"2024-09-15T21:22:48.322329+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:48.322329+0000","last_undegraded":"2024-09-15T21:22:48.322329+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7413,"log_dups_size":0,"ondisk_log_size":7413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30339072,"num_objects":7407,"num_object_clones":0,"num_object_copies":14814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":7407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5106,"num_write_kb":20424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2428,"num_bytes_recovered":9936896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1736},{"shards":"2","objects":5671}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7427","reported_seq":7555,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.436311+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:50.436311+0000","last_peered":"2024-09-15T21:22:50.436311+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:50.436311+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7427,"log_dups_size":0,"ondisk_log_size":7427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30412800,"num_objects":7425,"num_object_clones":0,"num_object_copies":14850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4371,"num_write_kb":17484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3331},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7429","reported_seq":7556,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.488423+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:50.488423+0000","last_peered":"2024-09-15T21:22:50.488423+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:50.488423+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7429,"log_dups_size":0,"ondisk_log_size":7429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30396416,"num_objects":7421,"num_object_clones":0,"num_object_copies":14842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4338,"num_write_kb":17352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3282},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7290","reported_seq":7421,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.432080+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:50.432080+0000","last_peered":"2024-09-15T21:22:50.432080+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:50.432080+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7290,"log_dups_size":0,"ondisk_log_size":7290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29843456,"num_objects":7286,"num_object_clones":0,"num_object_copies":14572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3774,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3204},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7423","reported_seq":7557,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.477306+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:50.477306+0000","last_peered":"2024-09-15T21:22:50.477306+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:50.477306+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7423,"log_dups_size":0,"ondisk_log_size":7423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30356422,"num_objects":7413,"num_object_clones":0,"num_object_copies":14826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7413,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4127,"num_write_kb":16508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3266},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":484942558,"num_objects":118403,"num_object_clones":0,"num_object_copies":236806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1736,"num_objects_unfound":0,"num_objects_dirty":118403,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":78268,"num_write_kb":313072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2429,"num_bytes_recovered":9940992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":974274560,"data_stored":974126427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118475,"ondisk_log_size":118475,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539292,"kb_used_data":367372,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93832548,"statfs":{"total":96636764160,"available":96084529152,"internally_reserved":0,"allocated":376188928,"data_stored":375926914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":176025257},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,6,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696164,"kb_used_data":421588,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93675676,"statfs":{"total":96636764160,"available":95923892224,"internally_reserved":0,"allocated":431706112,"data_stored":431440021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":337220,"kb_used_data":161140,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":94034620,"statfs":{"total":96636764160,"available":96291450880,"internally_reserved":0,"allocated":165007360,"data_stored":164701526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,6,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":432287744,"data_stored":432251614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":165584896,"data_stored":165509023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":376401920,"data_stored":376365790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:52.748 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 141 20 119884 119864 3.32026 1.88672 0.0324109 0.0235216 2024-09-15T21:22:53.750 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 142 20 120413 120393 3.3114 2.06641 0.0228442 0.0235869 2024-09-15T21:22:54.750 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 143 20 120927 120907 3.30228 2.00781 0.0256988 0.0236487 2024-09-15T21:22:55.330 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:55.750 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 144 20 121425 121405 3.29286 1.94531 0.0385216 0.0237164 2024-09-15T21:22:55.890 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:55.890 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:55.917 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-15T21:22:54.723051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":488735473,"num_objects":119331,"num_object_clones":0,"num_object_copies":238662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1574,"num_objects_unfound":0,"num_objects_dirty":119331,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":79196,"num_write_kb":316778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2592,"num_bytes_recovered":10604563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119404,"ondisk_log_size":119404,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1598476,"kb_used_data":963420,"kb_used_omap":12,"kb_used_meta":634995,"kb_avail":281517044,"statfs":{"total":289910292480,"available":288273453056,"internally_reserved":0,"allocated":986542080,"data_stored":985708141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":650234883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,4,7,11,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24612864,"num_objects":6009,"num_object_clones":0,"num_object_copies":12018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-460,"num_objects_unfound":0,"num_objects_dirty":6009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6009,"num_write_kb":24036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":462,"num_bytes_recovered":1884160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007262"},"pg_stats":[{"pgid":"2.f","version":"43'7592","reported_seq":7661,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.507151+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:50.507151+0000","last_peered":"2024-09-15T21:22:50.507151+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:50.507151+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7592,"log_dups_size":0,"ondisk_log_size":7592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31096832,"num_objects":7592,"num_object_clones":0,"num_object_copies":15184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7592,"num_write_kb":30368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3460},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7509","reported_seq":7609,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.281077+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:53.281077+0000","last_peered":"2024-09-15T21:22:53.281077+0000","last_clean":"2024-09-15T21:22:53.281077+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:53.281077+0000","last_undegraded":"2024-09-15T21:22:53.281077+0000","last_fullsized":"2024-09-15T21:22:53.281077+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7509,"log_dups_size":0,"ondisk_log_size":7509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30733254,"num_objects":7505,"num_object_clones":0,"num_object_copies":15010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7505,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5790,"num_write_kb":23160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7691","reported_seq":7811,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.336992+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:53.336992+0000","last_peered":"2024-09-15T21:22:53.336992+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:53.336992+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7691,"log_dups_size":0,"ondisk_log_size":7691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31485952,"num_objects":7687,"num_object_clones":0,"num_object_copies":15374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5791,"num_write_kb":23164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3420}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7388","reported_seq":7454,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.354940+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:53.354940+0000","last_peered":"2024-09-15T21:22:53.354940+0000","last_clean":"2024-09-15T21:22:53.354940+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:53.354940+0000","last_undegraded":"2024-09-15T21:22:53.354940+0000","last_fullsized":"2024-09-15T21:22:53.354940+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7388,"log_dups_size":0,"ondisk_log_size":7388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261248,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7388,"num_write_kb":29552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7399","reported_seq":7527,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:50.400568+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:50.400568+0000","last_peered":"2024-09-15T21:22:50.400568+0000","last_clean":"2024-09-15T21:22:50.400568+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:50.400568+0000","last_undegraded":"2024-09-15T21:22:50.400568+0000","last_fullsized":"2024-09-15T21:22:50.400568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30278115,"num_objects":7393,"num_object_clones":0,"num_object_copies":14786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3854,"num_write_kb":15416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7385","reported_seq":7516,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.436929+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:50.436929+0000","last_peered":"2024-09-15T21:22:50.436929+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:50.436929+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7385,"log_dups_size":0,"ondisk_log_size":7385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30228963,"num_objects":7381,"num_object_clones":0,"num_object_copies":14762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4061,"num_write_kb":16244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3305},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7466","reported_seq":7597,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.460813+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:50.460813+0000","last_peered":"2024-09-15T21:22:50.460813+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:50.460813+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7466,"log_dups_size":0,"ondisk_log_size":7466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30557126,"num_objects":7462,"num_object_clones":0,"num_object_copies":14924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3777,"num_write_kb":15108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3301},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7567","reported_seq":7691,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:50.455389+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:50.455389+0000","last_peered":"2024-09-15T21:22:50.455389+0000","last_clean":"2024-09-15T21:22:50.455389+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:50.455389+0000","last_undegraded":"2024-09-15T21:22:50.455389+0000","last_fullsized":"2024-09-15T21:22:50.455389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7567,"log_dups_size":0,"ondisk_log_size":7567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30986240,"num_objects":7565,"num_object_clones":0,"num_object_copies":15130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4339,"num_write_kb":17356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7499","reported_seq":7621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.306673+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:53.306673+0000","last_peered":"2024-09-15T21:22:53.306673+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:53.306673+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7499,"log_dups_size":0,"ondisk_log_size":7499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687715,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5431,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7256","reported_seq":7382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.441826+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:50.441826+0000","last_peered":"2024-09-15T21:22:50.441826+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:50.441826+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7256,"log_dups_size":0,"ondisk_log_size":7256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":7248,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3929,"num_write_kb":15716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3228},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7514","reported_seq":7628,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.305245+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:53.305245+0000","last_peered":"2024-09-15T21:22:53.305245+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:53.305245+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7514,"log_dups_size":0,"ondisk_log_size":7514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30757347,"num_objects":7510,"num_object_clones":0,"num_object_copies":15020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5373,"num_write_kb":21492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3420}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7566","reported_seq":12865,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:53.347588+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:53.347588+0000","last_peered":"2024-09-15T21:22:53.347588+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:53.347588+0000","last_undegraded":"2024-09-15T21:22:53.347588+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7566,"log_dups_size":0,"ondisk_log_size":7566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30965760,"num_objects":7560,"num_object_clones":0,"num_object_copies":15120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1574,"num_objects_unfound":0,"num_objects_dirty":7560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5259,"num_write_kb":21036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2590,"num_bytes_recovered":10600448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1574},{"shards":"2","objects":5986}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7427","reported_seq":7555,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.436311+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:50.436311+0000","last_peered":"2024-09-15T21:22:50.436311+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:50.436311+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7427,"log_dups_size":0,"ondisk_log_size":7427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30412800,"num_objects":7425,"num_object_clones":0,"num_object_copies":14850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4371,"num_write_kb":17484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3331},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7429","reported_seq":7556,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.488423+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:50.488423+0000","last_peered":"2024-09-15T21:22:50.488423+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:50.488423+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7429,"log_dups_size":0,"ondisk_log_size":7429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30396416,"num_objects":7421,"num_object_clones":0,"num_object_copies":14842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4338,"num_write_kb":17352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3282},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7290","reported_seq":7421,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.432080+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:50.432080+0000","last_peered":"2024-09-15T21:22:50.432080+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:50.432080+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7290,"log_dups_size":0,"ondisk_log_size":7290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29843456,"num_objects":7286,"num_object_clones":0,"num_object_copies":14572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3774,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3204},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7423","reported_seq":7557,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.477306+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:50.477306+0000","last_peered":"2024-09-15T21:22:50.477306+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:50.477306+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7423,"log_dups_size":0,"ondisk_log_size":7423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30356422,"num_objects":7413,"num_object_clones":0,"num_object_copies":14826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7413,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4127,"num_write_kb":16508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3266},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":488735454,"num_objects":119329,"num_object_clones":0,"num_object_copies":238658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1574,"num_objects_unfound":0,"num_objects_dirty":119329,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":79194,"num_write_kb":316776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2591,"num_bytes_recovered":10604544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":987873280,"data_stored":987725147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119401,"ondisk_log_size":119401,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548524,"kb_used_data":372444,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93823316,"statfs":{"total":96636764160,"available":96075075584,"internally_reserved":0,"allocated":381382656,"data_stored":381120642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":180285097},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696164,"kb_used_data":421588,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93675676,"statfs":{"total":96636764160,"available":95923892224,"internally_reserved":0,"allocated":431706112,"data_stored":431440021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":353788,"kb_used_data":169388,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":94018052,"statfs":{"total":96636764160,"available":96274485248,"internally_reserved":0,"allocated":173453312,"data_stored":173147478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":188804783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":432287744,"data_stored":432251614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":174125056,"data_stored":174049183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381460480,"data_stored":381424350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:55.919 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:56.465 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:56.465 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:56.494 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-15T21:22:54.723051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":488735473,"num_objects":119331,"num_object_clones":0,"num_object_copies":238662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1574,"num_objects_unfound":0,"num_objects_dirty":119331,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":79196,"num_write_kb":316778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2592,"num_bytes_recovered":10604563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119404,"ondisk_log_size":119404,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1598476,"kb_used_data":963420,"kb_used_omap":12,"kb_used_meta":634995,"kb_avail":281517044,"statfs":{"total":289910292480,"available":288273453056,"internally_reserved":0,"allocated":986542080,"data_stored":985708141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":650234883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,4,7,11,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24612864,"num_objects":6009,"num_object_clones":0,"num_object_copies":12018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-460,"num_objects_unfound":0,"num_objects_dirty":6009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6009,"num_write_kb":24036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":462,"num_bytes_recovered":1884160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007262"},"pg_stats":[{"pgid":"2.f","version":"43'7592","reported_seq":7661,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.507151+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:50.507151+0000","last_peered":"2024-09-15T21:22:50.507151+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:50.507151+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7592,"log_dups_size":0,"ondisk_log_size":7592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31096832,"num_objects":7592,"num_object_clones":0,"num_object_copies":15184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7592,"num_write_kb":30368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3460},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7509","reported_seq":7609,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.281077+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:53.281077+0000","last_peered":"2024-09-15T21:22:53.281077+0000","last_clean":"2024-09-15T21:22:53.281077+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:53.281077+0000","last_undegraded":"2024-09-15T21:22:53.281077+0000","last_fullsized":"2024-09-15T21:22:53.281077+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7509,"log_dups_size":0,"ondisk_log_size":7509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30733254,"num_objects":7505,"num_object_clones":0,"num_object_copies":15010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7505,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5790,"num_write_kb":23160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7691","reported_seq":7811,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.336992+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:53.336992+0000","last_peered":"2024-09-15T21:22:53.336992+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:53.336992+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7691,"log_dups_size":0,"ondisk_log_size":7691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31485952,"num_objects":7687,"num_object_clones":0,"num_object_copies":15374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5791,"num_write_kb":23164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3420}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7388","reported_seq":7454,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.354940+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:53.354940+0000","last_peered":"2024-09-15T21:22:53.354940+0000","last_clean":"2024-09-15T21:22:53.354940+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:53.354940+0000","last_undegraded":"2024-09-15T21:22:53.354940+0000","last_fullsized":"2024-09-15T21:22:53.354940+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7388,"log_dups_size":0,"ondisk_log_size":7388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261248,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7388,"num_write_kb":29552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7399","reported_seq":7527,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:50.400568+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:50.400568+0000","last_peered":"2024-09-15T21:22:50.400568+0000","last_clean":"2024-09-15T21:22:50.400568+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:50.400568+0000","last_undegraded":"2024-09-15T21:22:50.400568+0000","last_fullsized":"2024-09-15T21:22:50.400568+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30278115,"num_objects":7393,"num_object_clones":0,"num_object_copies":14786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3854,"num_write_kb":15416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7385","reported_seq":7516,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.436929+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:50.436929+0000","last_peered":"2024-09-15T21:22:50.436929+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:50.436929+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7385,"log_dups_size":0,"ondisk_log_size":7385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30228963,"num_objects":7381,"num_object_clones":0,"num_object_copies":14762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7381,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4061,"num_write_kb":16244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3305},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7466","reported_seq":7597,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.460813+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:50.460813+0000","last_peered":"2024-09-15T21:22:50.460813+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:50.460813+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7466,"log_dups_size":0,"ondisk_log_size":7466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30557126,"num_objects":7462,"num_object_clones":0,"num_object_copies":14924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3777,"num_write_kb":15108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3301},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7567","reported_seq":7691,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:50.455389+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:50.455389+0000","last_peered":"2024-09-15T21:22:50.455389+0000","last_clean":"2024-09-15T21:22:50.455389+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:50.455389+0000","last_undegraded":"2024-09-15T21:22:50.455389+0000","last_fullsized":"2024-09-15T21:22:50.455389+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7567,"log_dups_size":0,"ondisk_log_size":7567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30986240,"num_objects":7565,"num_object_clones":0,"num_object_copies":15130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4339,"num_write_kb":17356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7499","reported_seq":7621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.306673+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:53.306673+0000","last_peered":"2024-09-15T21:22:53.306673+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:53.306673+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7499,"log_dups_size":0,"ondisk_log_size":7499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687715,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5431,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7256","reported_seq":7382,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.441826+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:50.441826+0000","last_peered":"2024-09-15T21:22:50.441826+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:50.441826+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7256,"log_dups_size":0,"ondisk_log_size":7256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29687808,"num_objects":7248,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3929,"num_write_kb":15716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3228},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7514","reported_seq":7628,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.305245+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:53.305245+0000","last_peered":"2024-09-15T21:22:53.305245+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:53.305245+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7514,"log_dups_size":0,"ondisk_log_size":7514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30757347,"num_objects":7510,"num_object_clones":0,"num_object_copies":15020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5373,"num_write_kb":21492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3420}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7566","reported_seq":12865,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:53.347588+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:53.347588+0000","last_peered":"2024-09-15T21:22:53.347588+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:53.347588+0000","last_undegraded":"2024-09-15T21:22:53.347588+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7566,"log_dups_size":0,"ondisk_log_size":7566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30965760,"num_objects":7560,"num_object_clones":0,"num_object_copies":15120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1574,"num_objects_unfound":0,"num_objects_dirty":7560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5259,"num_write_kb":21036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2590,"num_bytes_recovered":10600448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1574},{"shards":"2","objects":5986}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7427","reported_seq":7555,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.436311+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:50.436311+0000","last_peered":"2024-09-15T21:22:50.436311+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:50.436311+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7427,"log_dups_size":0,"ondisk_log_size":7427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30412800,"num_objects":7425,"num_object_clones":0,"num_object_copies":14850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4371,"num_write_kb":17484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3331},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7429","reported_seq":7556,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.488423+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:50.488423+0000","last_peered":"2024-09-15T21:22:50.488423+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:50.488423+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7429,"log_dups_size":0,"ondisk_log_size":7429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30396416,"num_objects":7421,"num_object_clones":0,"num_object_copies":14842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4338,"num_write_kb":17352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3282},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7290","reported_seq":7421,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.432080+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:50.432080+0000","last_peered":"2024-09-15T21:22:50.432080+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:50.432080+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7290,"log_dups_size":0,"ondisk_log_size":7290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29843456,"num_objects":7286,"num_object_clones":0,"num_object_copies":14572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3774,"num_write_kb":15096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3204},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7423","reported_seq":7557,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:50.477306+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:50.477306+0000","last_peered":"2024-09-15T21:22:50.477306+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:50.477306+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7423,"log_dups_size":0,"ondisk_log_size":7423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30356422,"num_objects":7413,"num_object_clones":0,"num_object_copies":14826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7413,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4127,"num_write_kb":16508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3266},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":488735454,"num_objects":119329,"num_object_clones":0,"num_object_copies":238658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1574,"num_objects_unfound":0,"num_objects_dirty":119329,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":79194,"num_write_kb":316776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2591,"num_bytes_recovered":10604544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":987873280,"data_stored":987725147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119401,"ondisk_log_size":119401,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548524,"kb_used_data":372444,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93823316,"statfs":{"total":96636764160,"available":96075075584,"internally_reserved":0,"allocated":381382656,"data_stored":381120642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":180285097},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696164,"kb_used_data":421588,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93675676,"statfs":{"total":96636764160,"available":95923892224,"internally_reserved":0,"allocated":431706112,"data_stored":431440021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":353788,"kb_used_data":169388,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":94018052,"statfs":{"total":96636764160,"available":96274485248,"internally_reserved":0,"allocated":173453312,"data_stored":173147478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":188804783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":432287744,"data_stored":432251614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":174125056,"data_stored":174049183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381460480,"data_stored":381424350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:56.495 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:22:56.495 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'7592", 'reported_seq': 7661, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.507151+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:22:50.507151+0000', 'last_peered': '2024-09-15T21:22:50.507151+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:22:50.507151+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 7592, 'log_dups_size': 0, 'ondisk_log_size': 7592, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31096832, 'num_objects': 7592, 'num_object_clones': 0, 'num_object_copies': 15184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7592, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7592, 'num_write_kb': 30368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3460}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:56.495 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:22:56.495 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'7691", 'reported_seq': 7811, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:53.336992+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:22:53.336992+0000', 'last_peered': '2024-09-15T21:22:53.336992+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:22:53.336992+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 7691, 'log_dups_size': 0, 'ondisk_log_size': 7691, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31485952, 'num_objects': 7687, 'num_object_clones': 0, 'num_object_copies': 15374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7687, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5791, 'num_write_kb': 23164, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 3420}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:56.495 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:22:56.495 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'7385", 'reported_seq': 7516, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.436929+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:22:50.436929+0000', 'last_peered': '2024-09-15T21:22:50.436929+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:22:50.436929+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 7385, 'log_dups_size': 0, 'ondisk_log_size': 7385, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30228963, 'num_objects': 7381, 'num_object_clones': 0, 'num_object_copies': 14762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7381, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4061, 'num_write_kb': 16244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3305}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:56.496 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:22:56.496 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'7466", 'reported_seq': 7597, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.460813+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:22:50.460813+0000', 'last_peered': '2024-09-15T21:22:50.460813+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:22:50.460813+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 7466, 'log_dups_size': 0, 'ondisk_log_size': 7466, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30557126, 'num_objects': 7462, 'num_object_clones': 0, 'num_object_copies': 14924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7462, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3777, 'num_write_kb': 15108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3301}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:56.496 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:22:56.496 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7499", 'reported_seq': 7621, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:53.306673+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:22:53.306673+0000', 'last_peered': '2024-09-15T21:22:53.306673+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:22:53.306673+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 7499, 'log_dups_size': 0, 'ondisk_log_size': 7499, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30687715, 'num_objects': 7493, 'num_object_clones': 0, 'num_object_copies': 14986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7493, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5431, 'num_write_kb': 21724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 3406}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:56.496 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:22:56.496 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'7256", 'reported_seq': 7382, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.441826+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:22:50.441826+0000', 'last_peered': '2024-09-15T21:22:50.441826+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:22:50.441826+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 7256, 'log_dups_size': 0, 'ondisk_log_size': 7256, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29687808, 'num_objects': 7248, 'num_object_clones': 0, 'num_object_copies': 14496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7248, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3929, 'num_write_kb': 15716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3228}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:56.496 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:22:56.496 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7514", 'reported_seq': 7628, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:53.305245+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:22:53.305245+0000', 'last_peered': '2024-09-15T21:22:53.305245+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:22:53.305245+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 7514, 'log_dups_size': 0, 'ondisk_log_size': 7514, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30757347, 'num_objects': 7510, 'num_object_clones': 0, 'num_object_copies': 15020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7510, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5373, 'num_write_kb': 21492, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 3420}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:56.496 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:22:56.496 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'7566", 'reported_seq': 12865, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:22:53.347588+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:22:53.347588+0000', 'last_peered': '2024-09-15T21:22:53.347588+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:22:53.347588+0000', 'last_undegraded': '2024-09-15T21:22:53.347588+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 7566, 'log_dups_size': 0, 'ondisk_log_size': 7566, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30965760, 'num_objects': 7560, 'num_object_clones': 0, 'num_object_copies': 15120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1574, 'num_objects_unfound': 0, 'num_objects_dirty': 7560, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5259, 'num_write_kb': 21036, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2590, 'num_bytes_recovered': 10600448, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1574}, {'shards': '2', 'objects': 5986}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:22:56.496 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:22:56.497 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'7427", 'reported_seq': 7555, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.436311+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:22:50.436311+0000', 'last_peered': '2024-09-15T21:22:50.436311+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:22:50.436311+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 7427, 'log_dups_size': 0, 'ondisk_log_size': 7427, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30412800, 'num_objects': 7425, 'num_object_clones': 0, 'num_object_copies': 14850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7425, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4371, 'num_write_kb': 17484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3331}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:56.497 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:22:56.497 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'7429", 'reported_seq': 7556, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.488423+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:22:50.488423+0000', 'last_peered': '2024-09-15T21:22:50.488423+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:22:50.488423+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 7429, 'log_dups_size': 0, 'ondisk_log_size': 7429, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30396416, 'num_objects': 7421, 'num_object_clones': 0, 'num_object_copies': 14842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7421, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4338, 'num_write_kb': 17352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3282}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:56.497 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:22:56.497 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'7290", 'reported_seq': 7421, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.432080+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:22:50.432080+0000', 'last_peered': '2024-09-15T21:22:50.432080+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:22:50.432080+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 7290, 'log_dups_size': 0, 'ondisk_log_size': 7290, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29843456, 'num_objects': 7286, 'num_object_clones': 0, 'num_object_copies': 14572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7286, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3774, 'num_write_kb': 15096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3204}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:56.497 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:22:56.497 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'7423", 'reported_seq': 7557, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:50.477306+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:22:50.477306+0000', 'last_peered': '2024-09-15T21:22:50.477306+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:22:50.477306+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 7423, 'log_dups_size': 0, 'ondisk_log_size': 7423, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30356422, 'num_objects': 7413, 'num_object_clones': 0, 'num_object_copies': 14826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7413, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4127, 'num_write_kb': 16508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3266}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:22:56.497 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:22:56.750 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 145 20 121883 121863 3.28249 1.78906 0.0238688 0.0237929 2024-09-15T21:22:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:57.204 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/238662 objects degraded (18.985%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":228,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":119331,"data_bytes":488735473,"bytes_used":1636839424,"bytes_avail":288273453056,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":238662,"degraded_ratio":0.18984589084144105,"misplaced_objects":1574,"misplaced_total":238662,"misplaced_ratio":0.0065951010215283537,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":156918,"recovering_keys_per_sec":0,"num_objects_recovered":462,"num_bytes_recovered":1884160,"num_keys_recovered":0,"write_bytes_sec":2049831,"read_op_per_sec":0,"write_op_per_sec":500},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:22:57.230 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:22:57.230 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:22:57.750 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 146 20 122343 122323 3.27231 1.79688 0.0342237 0.0238658 2024-09-15T21:22:57.803 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:22:57.806 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:22:57.836 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-15T21:22:56.727142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":495207153,"num_objects":120911,"num_object_clones":0,"num_object_copies":241822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1574,"num_objects_unfound":0,"num_objects_dirty":120911,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":80776,"num_write_kb":323098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2592,"num_bytes_recovered":10604563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120984,"ondisk_log_size":120984,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1589168,"kb_used_data":979072,"kb_used_omap":12,"kb_used_meta":610035,"kb_avail":281526352,"statfs":{"total":289910292480,"available":288282984448,"internally_reserved":0,"allocated":1002569728,"data_stored":1001735789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":624675843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,6,2,8,7,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27176960,"num_objects":6635,"num_object_clones":0,"num_object_copies":13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-318,"num_objects_unfound":0,"num_objects_dirty":6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6635,"num_write_kb":26540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009370"},"pg_stats":[{"pgid":"2.f","version":"43'7760","reported_seq":7831,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:55.484466+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:22:55.484466+0000","last_peered":"2024-09-15T21:22:55.484466+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:22:55.484466+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7760,"log_dups_size":0,"ondisk_log_size":7760,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31784960,"num_objects":7760,"num_object_clones":0,"num_object_copies":15520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7760,"num_write_kb":31040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3628},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7509","reported_seq":7609,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.281077+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:53.281077+0000","last_peered":"2024-09-15T21:22:53.281077+0000","last_clean":"2024-09-15T21:22:53.281077+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:53.281077+0000","last_undegraded":"2024-09-15T21:22:53.281077+0000","last_fullsized":"2024-09-15T21:22:53.281077+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7509,"log_dups_size":0,"ondisk_log_size":7509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30733254,"num_objects":7505,"num_object_clones":0,"num_object_copies":15010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7505,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5790,"num_write_kb":23160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7691","reported_seq":7811,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.336992+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:53.336992+0000","last_peered":"2024-09-15T21:22:53.336992+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:53.336992+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7691,"log_dups_size":0,"ondisk_log_size":7691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31485952,"num_objects":7687,"num_object_clones":0,"num_object_copies":15374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5791,"num_write_kb":23164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3420}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7388","reported_seq":7454,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:53.354940+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:53.354940+0000","last_peered":"2024-09-15T21:22:53.354940+0000","last_clean":"2024-09-15T21:22:53.354940+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:53.354940+0000","last_undegraded":"2024-09-15T21:22:53.354940+0000","last_fullsized":"2024-09-15T21:22:53.354940+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7388,"log_dups_size":0,"ondisk_log_size":7388,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30261248,"num_objects":7388,"num_object_clones":0,"num_object_copies":14776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7388,"num_write_kb":29552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7543","reported_seq":7671,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:55.519808+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:22:55.519808+0000","last_peered":"2024-09-15T21:22:55.519808+0000","last_clean":"2024-09-15T21:22:55.519808+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:22:55.519808+0000","last_undegraded":"2024-09-15T21:22:55.519808+0000","last_fullsized":"2024-09-15T21:22:55.519808+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7543,"log_dups_size":0,"ondisk_log_size":7543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30867939,"num_objects":7537,"num_object_clones":0,"num_object_copies":15074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3998,"num_write_kb":15992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7552","reported_seq":7683,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:55.513341+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:22:55.513341+0000","last_peered":"2024-09-15T21:22:55.513341+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:22:55.513341+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7552,"log_dups_size":0,"ondisk_log_size":7552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30912995,"num_objects":7548,"num_object_clones":0,"num_object_copies":15096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7548,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4228,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3472},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7601","reported_seq":7732,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:55.492252+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:22:55.492252+0000","last_peered":"2024-09-15T21:22:55.492252+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:22:55.492252+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7601,"log_dups_size":0,"ondisk_log_size":7601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31110086,"num_objects":7597,"num_object_clones":0,"num_object_copies":15194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3912,"num_write_kb":15648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3436},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7735","reported_seq":7859,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:55.451243+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:22:55.451243+0000","last_peered":"2024-09-15T21:22:55.451243+0000","last_clean":"2024-09-15T21:22:55.451243+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:22:55.451243+0000","last_undegraded":"2024-09-15T21:22:55.451243+0000","last_fullsized":"2024-09-15T21:22:55.451243+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7735,"log_dups_size":0,"ondisk_log_size":7735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31674368,"num_objects":7733,"num_object_clones":0,"num_object_copies":15466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7733,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4507,"num_write_kb":18028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7499","reported_seq":7621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.306673+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:53.306673+0000","last_peered":"2024-09-15T21:22:53.306673+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:53.306673+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7499,"log_dups_size":0,"ondisk_log_size":7499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687715,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5431,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3406}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7437","reported_seq":7563,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:55.464424+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:22:55.464424+0000","last_peered":"2024-09-15T21:22:55.464424+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:22:55.464424+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7437,"log_dups_size":0,"ondisk_log_size":7437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30429184,"num_objects":7429,"num_object_clones":0,"num_object_copies":14858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4110,"num_write_kb":16440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3409},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7514","reported_seq":7628,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:53.305245+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:53.305245+0000","last_peered":"2024-09-15T21:22:53.305245+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:53.305245+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7514,"log_dups_size":0,"ondisk_log_size":7514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30757347,"num_objects":7510,"num_object_clones":0,"num_object_copies":15020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5373,"num_write_kb":21492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3420}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7566","reported_seq":12865,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:53.347588+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:53.347588+0000","last_peered":"2024-09-15T21:22:53.347588+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:53.347588+0000","last_undegraded":"2024-09-15T21:22:53.347588+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7566,"log_dups_size":0,"ondisk_log_size":7566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30965760,"num_objects":7560,"num_object_clones":0,"num_object_copies":15120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1574,"num_objects_unfound":0,"num_objects_dirty":7560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5259,"num_write_kb":21036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2590,"num_bytes_recovered":10600448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1574},{"shards":"2","objects":5986}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7597","reported_seq":7726,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:55.467575+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:22:55.467575+0000","last_peered":"2024-09-15T21:22:55.467575+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:22:55.467575+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7597,"log_dups_size":0,"ondisk_log_size":7597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31109120,"num_objects":7595,"num_object_clones":0,"num_object_copies":15190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4541,"num_write_kb":18164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3501},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7565","reported_seq":7693,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:55.439319+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:22:55.439319+0000","last_peered":"2024-09-15T21:22:55.439319+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:22:55.439319+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7565,"log_dups_size":0,"ondisk_log_size":7565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30953472,"num_objects":7557,"num_object_clones":0,"num_object_copies":15114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4474,"num_write_kb":17896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3418},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7436","reported_seq":7566,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:55.506635+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:22:55.506635+0000","last_peered":"2024-09-15T21:22:55.506635+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:22:55.506635+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7436,"log_dups_size":0,"ondisk_log_size":7436,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30441472,"num_objects":7432,"num_object_clones":0,"num_object_copies":14864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3920,"num_write_kb":15680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3350},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7588","reported_seq":7722,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:55.498025+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:22:55.498025+0000","last_peered":"2024-09-15T21:22:55.498025+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:22:55.498025+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7588,"log_dups_size":0,"ondisk_log_size":7588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31032262,"num_objects":7578,"num_object_clones":0,"num_object_copies":15156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7578,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4292,"num_write_kb":17168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3431},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":495207134,"num_objects":120909,"num_object_clones":0,"num_object_copies":241818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1574,"num_objects_unfound":0,"num_objects_dirty":120909,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":80774,"num_write_kb":323096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2591,"num_bytes_recovered":10604544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1004056576,"data_stored":1003908443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120981,"ondisk_log_size":120981,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548524,"kb_used_data":372444,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":93823316,"statfs":{"total":96636764160,"available":96075075584,"internally_reserved":0,"allocated":381382656,"data_stored":381120642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":180285097},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691156,"kb_used_data":429060,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93680684,"statfs":{"total":96636764160,"available":95929020416,"internally_reserved":0,"allocated":439357440,"data_stored":439091349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":268365483},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349488,"kb_used_data":177568,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":94022352,"statfs":{"total":96636764160,"available":96278888448,"internally_reserved":0,"allocated":181829632,"data_stored":181523798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":176025263},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,1,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":440008704,"data_stored":439972574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":182587392,"data_stored":182511519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":381460480,"data_stored":381424350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:22:58.750 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 147 20 122865 122845 3.26392 2.03906 0.0622958 0.0239285 2024-09-15T21:22:59.750 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 148 20 123389 123369 3.2557 2.04688 0.0219707 0.0239889 2024-09-15T21:23:00.753 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 149 20 123907 123887 3.24737 2.02344 0.0221691 0.0240499 2024-09-15T21:23:00.839 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:01.397 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:01.398 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:01.424 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-15T21:23:00.728463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":505361137,"num_objects":123390,"num_object_clones":0,"num_object_copies":246780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1400,"num_objects_unfound":0,"num_objects_dirty":123390,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":83255,"num_write_kb":333014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2766,"num_bytes_recovered":11317267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123463,"ondisk_log_size":123463,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1609652,"kb_used_data":991236,"kb_used_omap":12,"kb_used_meta":618355,"kb_avail":281505868,"statfs":{"total":289910292480,"available":288262008832,"internally_reserved":0,"allocated":1015025664,"data_stored":1014191725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":633195523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,7,7,5,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-336,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6556,"num_write_kb":26224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":1376256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007204"},"pg_stats":[{"pgid":"2.f","version":"43'7922","reported_seq":7992,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.602025+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:00.602025+0000","last_peered":"2024-09-15T21:23:00.602025+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:00.602025+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7922,"log_dups_size":0,"ondisk_log_size":7922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32448512,"num_objects":7922,"num_object_clones":0,"num_object_copies":15844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7922,"num_write_kb":31688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3790},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7675","reported_seq":7774,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.269396+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:58.269396+0000","last_peered":"2024-09-15T21:22:58.269396+0000","last_clean":"2024-09-15T21:22:58.269396+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:58.269396+0000","last_undegraded":"2024-09-15T21:22:58.269396+0000","last_fullsized":"2024-09-15T21:22:58.269396+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7675,"log_dups_size":0,"ondisk_log_size":7675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31413190,"num_objects":7671,"num_object_clones":0,"num_object_copies":15342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7671,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5956,"num_write_kb":23824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7855","reported_seq":7976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.344455+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:58.344455+0000","last_peered":"2024-09-15T21:22:58.344455+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:58.344455+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7855,"log_dups_size":0,"ondisk_log_size":7855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32157696,"num_objects":7851,"num_object_clones":0,"num_object_copies":15702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5955,"num_write_kb":23820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3584}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7519","reported_seq":7585,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.317374+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:58.317374+0000","last_peered":"2024-09-15T21:22:58.317374+0000","last_clean":"2024-09-15T21:22:58.317374+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:58.317374+0000","last_undegraded":"2024-09-15T21:22:58.317374+0000","last_fullsized":"2024-09-15T21:22:58.317374+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7519,"log_dups_size":0,"ondisk_log_size":7519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30797824,"num_objects":7519,"num_object_clones":0,"num_object_copies":15038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7519,"num_write_kb":30076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7700","reported_seq":7828,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:00.558815+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:00.558815+0000","last_peered":"2024-09-15T21:23:00.558815+0000","last_clean":"2024-09-15T21:23:00.558815+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:00.558815+0000","last_undegraded":"2024-09-15T21:23:00.558815+0000","last_fullsized":"2024-09-15T21:23:00.558815+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7700,"log_dups_size":0,"ondisk_log_size":7700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31511011,"num_objects":7694,"num_object_clones":0,"num_object_copies":15388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4155,"num_write_kb":16620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7698","reported_seq":7829,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.522726+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:00.522726+0000","last_peered":"2024-09-15T21:23:00.522726+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:00.522726+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7698,"log_dups_size":0,"ondisk_log_size":7698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31511011,"num_objects":7694,"num_object_clones":0,"num_object_copies":15388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7694,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4374,"num_write_kb":17496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3618},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7772","reported_seq":7900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.551599+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:00.551599+0000","last_peered":"2024-09-15T21:23:00.551599+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:00.551599+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7772,"log_dups_size":0,"ondisk_log_size":7772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31810502,"num_objects":7768,"num_object_clones":0,"num_object_copies":15536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4083,"num_write_kb":16332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3607},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7885","reported_seq":8009,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:00.491212+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:00.491212+0000","last_peered":"2024-09-15T21:23:00.491212+0000","last_clean":"2024-09-15T21:23:00.491212+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:00.491212+0000","last_undegraded":"2024-09-15T21:23:00.491212+0000","last_fullsized":"2024-09-15T21:23:00.491212+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7885,"log_dups_size":0,"ondisk_log_size":7885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32288768,"num_objects":7883,"num_object_clones":0,"num_object_copies":15766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4657,"num_write_kb":18628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7639","reported_seq":7761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.083880+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:58.083880+0000","last_peered":"2024-09-15T21:22:58.083880+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:58.083880+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7639,"log_dups_size":0,"ondisk_log_size":7639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31261155,"num_objects":7633,"num_object_clones":0,"num_object_copies":15266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5571,"num_write_kb":22284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3546}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7592","reported_seq":7718,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.596959+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:00.596959+0000","last_peered":"2024-09-15T21:23:00.596959+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:00.596959+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7592,"log_dups_size":0,"ondisk_log_size":7592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31064064,"num_objects":7584,"num_object_clones":0,"num_object_copies":15168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4265,"num_write_kb":17060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3564},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7674","reported_seq":7785,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.308219+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:58.308219+0000","last_peered":"2024-09-15T21:22:58.308219+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:58.308219+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7674,"log_dups_size":0,"ondisk_log_size":7674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31412707,"num_objects":7670,"num_object_clones":0,"num_object_copies":15340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7670,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5533,"num_write_kb":22132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3580}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7704","reported_seq":13351,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:58.369275+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:58.369275+0000","last_peered":"2024-09-15T21:22:58.369275+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:58.369275+0000","last_undegraded":"2024-09-15T21:22:58.369275+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7704,"log_dups_size":0,"ondisk_log_size":7704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31531008,"num_objects":7698,"num_object_clones":0,"num_object_copies":15396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1400,"num_objects_unfound":0,"num_objects_dirty":7698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5397,"num_write_kb":21588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2764,"num_bytes_recovered":11313152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1400},{"shards":"2","objects":6298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7774","reported_seq":7903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.577381+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:00.577381+0000","last_peered":"2024-09-15T21:23:00.577381+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:00.577381+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7774,"log_dups_size":0,"ondisk_log_size":7774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31834112,"num_objects":7772,"num_object_clones":0,"num_object_copies":15544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4718,"num_write_kb":18872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3678},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7738","reported_seq":7866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.563415+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:00.563415+0000","last_peered":"2024-09-15T21:23:00.563415+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:00.563415+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7738,"log_dups_size":0,"ondisk_log_size":7738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31662080,"num_objects":7730,"num_object_clones":0,"num_object_copies":15460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4647,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3591},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7573","reported_seq":7702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.610472+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:00.610472+0000","last_peered":"2024-09-15T21:23:00.610472+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:00.610472+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7573,"log_dups_size":0,"ondisk_log_size":7573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31002624,"num_objects":7569,"num_object_clones":0,"num_object_copies":15138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4057,"num_write_kb":16228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3487},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7740","reported_seq":7874,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.588223+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:00.588223+0000","last_peered":"2024-09-15T21:23:00.588223+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:00.588223+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7740,"log_dups_size":0,"ondisk_log_size":7740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31654854,"num_objects":7730,"num_object_clones":0,"num_object_copies":15460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7730,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4444,"num_write_kb":17776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3583},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":505361118,"num_objects":123388,"num_object_clones":0,"num_object_copies":246776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1400,"num_objects_unfound":0,"num_objects_dirty":123388,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":83253,"num_write_kb":333012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2765,"num_bytes_recovered":11317248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1016692736,"data_stored":1016544603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123460,"ondisk_log_size":123460,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557544,"kb_used_data":377304,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93814296,"statfs":{"total":96636764160,"available":96065839104,"internally_reserved":0,"allocated":386359296,"data_stored":386097282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":184544937},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,0,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":702620,"kb_used_data":436364,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93669220,"statfs":{"total":96636764160,"available":95917281280,"internally_reserved":0,"allocated":446836736,"data_stored":446570645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349488,"kb_used_data":177568,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":94022352,"statfs":{"total":96636764160,"available":96278888448,"internally_reserved":0,"allocated":181829632,"data_stored":181523798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":176025263},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,1,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447692800,"data_stored":447656670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":182587392,"data_stored":182511519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386412544,"data_stored":386376414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:01.426 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:01.753 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 150 20 124369 124349 3.23775 1.80469 0.0514897 0.0241221 2024-09-15T21:23:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:01.996 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:02.020 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-15T21:23:00.728463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":505361137,"num_objects":123390,"num_object_clones":0,"num_object_copies":246780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1400,"num_objects_unfound":0,"num_objects_dirty":123390,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":83255,"num_write_kb":333014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2766,"num_bytes_recovered":11317267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123463,"ondisk_log_size":123463,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1609652,"kb_used_data":991236,"kb_used_omap":12,"kb_used_meta":618355,"kb_avail":281505868,"statfs":{"total":289910292480,"available":288262008832,"internally_reserved":0,"allocated":1015025664,"data_stored":1014191725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":633195523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,7,7,5,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26853376,"num_objects":6556,"num_object_clones":0,"num_object_copies":13112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-336,"num_objects_unfound":0,"num_objects_dirty":6556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6556,"num_write_kb":26224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":1376256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007204"},"pg_stats":[{"pgid":"2.f","version":"43'7922","reported_seq":7992,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.602025+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:00.602025+0000","last_peered":"2024-09-15T21:23:00.602025+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:00.602025+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7922,"log_dups_size":0,"ondisk_log_size":7922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32448512,"num_objects":7922,"num_object_clones":0,"num_object_copies":15844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7922,"num_write_kb":31688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3790},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7675","reported_seq":7774,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.269396+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:58.269396+0000","last_peered":"2024-09-15T21:22:58.269396+0000","last_clean":"2024-09-15T21:22:58.269396+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:58.269396+0000","last_undegraded":"2024-09-15T21:22:58.269396+0000","last_fullsized":"2024-09-15T21:22:58.269396+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7675,"log_dups_size":0,"ondisk_log_size":7675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31413190,"num_objects":7671,"num_object_clones":0,"num_object_copies":15342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7671,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5956,"num_write_kb":23824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7855","reported_seq":7976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.344455+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:58.344455+0000","last_peered":"2024-09-15T21:22:58.344455+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:58.344455+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7855,"log_dups_size":0,"ondisk_log_size":7855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32157696,"num_objects":7851,"num_object_clones":0,"num_object_copies":15702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5955,"num_write_kb":23820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3584}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7519","reported_seq":7585,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.317374+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:58.317374+0000","last_peered":"2024-09-15T21:22:58.317374+0000","last_clean":"2024-09-15T21:22:58.317374+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:58.317374+0000","last_undegraded":"2024-09-15T21:22:58.317374+0000","last_fullsized":"2024-09-15T21:22:58.317374+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7519,"log_dups_size":0,"ondisk_log_size":7519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30797824,"num_objects":7519,"num_object_clones":0,"num_object_copies":15038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7519,"num_write_kb":30076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7700","reported_seq":7828,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:00.558815+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:00.558815+0000","last_peered":"2024-09-15T21:23:00.558815+0000","last_clean":"2024-09-15T21:23:00.558815+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:00.558815+0000","last_undegraded":"2024-09-15T21:23:00.558815+0000","last_fullsized":"2024-09-15T21:23:00.558815+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7700,"log_dups_size":0,"ondisk_log_size":7700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31511011,"num_objects":7694,"num_object_clones":0,"num_object_copies":15388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4155,"num_write_kb":16620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7698","reported_seq":7829,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.522726+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:00.522726+0000","last_peered":"2024-09-15T21:23:00.522726+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:00.522726+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7698,"log_dups_size":0,"ondisk_log_size":7698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31511011,"num_objects":7694,"num_object_clones":0,"num_object_copies":15388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7694,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4374,"num_write_kb":17496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3618},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7772","reported_seq":7900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.551599+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:00.551599+0000","last_peered":"2024-09-15T21:23:00.551599+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:00.551599+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7772,"log_dups_size":0,"ondisk_log_size":7772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31810502,"num_objects":7768,"num_object_clones":0,"num_object_copies":15536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4083,"num_write_kb":16332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3607},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7885","reported_seq":8009,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:00.491212+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:00.491212+0000","last_peered":"2024-09-15T21:23:00.491212+0000","last_clean":"2024-09-15T21:23:00.491212+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:00.491212+0000","last_undegraded":"2024-09-15T21:23:00.491212+0000","last_fullsized":"2024-09-15T21:23:00.491212+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7885,"log_dups_size":0,"ondisk_log_size":7885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32288768,"num_objects":7883,"num_object_clones":0,"num_object_copies":15766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4657,"num_write_kb":18628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7639","reported_seq":7761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.083880+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:58.083880+0000","last_peered":"2024-09-15T21:22:58.083880+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:58.083880+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7639,"log_dups_size":0,"ondisk_log_size":7639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31261155,"num_objects":7633,"num_object_clones":0,"num_object_copies":15266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5571,"num_write_kb":22284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3546}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7592","reported_seq":7718,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.596959+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:00.596959+0000","last_peered":"2024-09-15T21:23:00.596959+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:00.596959+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7592,"log_dups_size":0,"ondisk_log_size":7592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31064064,"num_objects":7584,"num_object_clones":0,"num_object_copies":15168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4265,"num_write_kb":17060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3564},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7674","reported_seq":7785,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.308219+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:58.308219+0000","last_peered":"2024-09-15T21:22:58.308219+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:58.308219+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7674,"log_dups_size":0,"ondisk_log_size":7674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31412707,"num_objects":7670,"num_object_clones":0,"num_object_copies":15340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7670,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5533,"num_write_kb":22132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3580}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7704","reported_seq":13351,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:58.369275+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:58.369275+0000","last_peered":"2024-09-15T21:22:58.369275+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:58.369275+0000","last_undegraded":"2024-09-15T21:22:58.369275+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7704,"log_dups_size":0,"ondisk_log_size":7704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31531008,"num_objects":7698,"num_object_clones":0,"num_object_copies":15396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1400,"num_objects_unfound":0,"num_objects_dirty":7698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5397,"num_write_kb":21588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2764,"num_bytes_recovered":11313152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1400},{"shards":"2","objects":6298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7774","reported_seq":7903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.577381+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:00.577381+0000","last_peered":"2024-09-15T21:23:00.577381+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:00.577381+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7774,"log_dups_size":0,"ondisk_log_size":7774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31834112,"num_objects":7772,"num_object_clones":0,"num_object_copies":15544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4718,"num_write_kb":18872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3678},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7738","reported_seq":7866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.563415+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:00.563415+0000","last_peered":"2024-09-15T21:23:00.563415+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:00.563415+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7738,"log_dups_size":0,"ondisk_log_size":7738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31662080,"num_objects":7730,"num_object_clones":0,"num_object_copies":15460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4647,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3591},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7573","reported_seq":7702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.610472+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:00.610472+0000","last_peered":"2024-09-15T21:23:00.610472+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:00.610472+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7573,"log_dups_size":0,"ondisk_log_size":7573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31002624,"num_objects":7569,"num_object_clones":0,"num_object_copies":15138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4057,"num_write_kb":16228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3487},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7740","reported_seq":7874,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.588223+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:00.588223+0000","last_peered":"2024-09-15T21:23:00.588223+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:00.588223+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7740,"log_dups_size":0,"ondisk_log_size":7740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31654854,"num_objects":7730,"num_object_clones":0,"num_object_copies":15460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7730,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4444,"num_write_kb":17776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3583},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":505361118,"num_objects":123388,"num_object_clones":0,"num_object_copies":246776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1400,"num_objects_unfound":0,"num_objects_dirty":123388,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":83253,"num_write_kb":333012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2765,"num_bytes_recovered":11317248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1016692736,"data_stored":1016544603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123460,"ondisk_log_size":123460,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557544,"kb_used_data":377304,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93814296,"statfs":{"total":96636764160,"available":96065839104,"internally_reserved":0,"allocated":386359296,"data_stored":386097282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":184544937},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,0,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":702620,"kb_used_data":436364,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93669220,"statfs":{"total":96636764160,"available":95917281280,"internally_reserved":0,"allocated":446836736,"data_stored":446570645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":349488,"kb_used_data":177568,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":94022352,"statfs":{"total":96636764160,"available":96278888448,"internally_reserved":0,"allocated":181829632,"data_stored":181523798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":176025263},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,1,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447692800,"data_stored":447656670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":182587392,"data_stored":182511519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386412544,"data_stored":386376414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:02.022 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:02.022 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'7922", 'reported_seq': 7992, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:00.602025+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:23:00.602025+0000', 'last_peered': '2024-09-15T21:23:00.602025+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:23:00.602025+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 7922, 'log_dups_size': 0, 'ondisk_log_size': 7922, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32448512, 'num_objects': 7922, 'num_object_clones': 0, 'num_object_copies': 15844, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7922, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7922, 'num_write_kb': 31688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3790}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:02.022 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:23:02.022 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'7855", 'reported_seq': 7976, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:58.344455+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:22:58.344455+0000', 'last_peered': '2024-09-15T21:22:58.344455+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:22:58.344455+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 7855, 'log_dups_size': 0, 'ondisk_log_size': 7855, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32157696, 'num_objects': 7851, 'num_object_clones': 0, 'num_object_copies': 15702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5955, 'num_write_kb': 23820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 3584}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:02.022 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:02.022 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'7698", 'reported_seq': 7829, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:00.522726+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:23:00.522726+0000', 'last_peered': '2024-09-15T21:23:00.522726+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:23:00.522726+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 7698, 'log_dups_size': 0, 'ondisk_log_size': 7698, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31511011, 'num_objects': 7694, 'num_object_clones': 0, 'num_object_copies': 15388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7694, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4374, 'num_write_kb': 17496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3618}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:02.022 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:02.022 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'7772", 'reported_seq': 7900, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:00.551599+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:23:00.551599+0000', 'last_peered': '2024-09-15T21:23:00.551599+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:23:00.551599+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 7772, 'log_dups_size': 0, 'ondisk_log_size': 7772, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31810502, 'num_objects': 7768, 'num_object_clones': 0, 'num_object_copies': 15536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7768, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4083, 'num_write_kb': 16332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3607}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:02.023 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:02.023 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7639", 'reported_seq': 7761, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:58.083880+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:22:58.083880+0000', 'last_peered': '2024-09-15T21:22:58.083880+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:22:58.083880+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 7639, 'log_dups_size': 0, 'ondisk_log_size': 7639, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31261155, 'num_objects': 7633, 'num_object_clones': 0, 'num_object_copies': 15266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7633, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5571, 'num_write_kb': 22284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 3546}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:02.023 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:02.023 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'7592", 'reported_seq': 7718, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:00.596959+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:23:00.596959+0000', 'last_peered': '2024-09-15T21:23:00.596959+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:23:00.596959+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 7592, 'log_dups_size': 0, 'ondisk_log_size': 7592, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31064064, 'num_objects': 7584, 'num_object_clones': 0, 'num_object_copies': 15168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7584, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4265, 'num_write_kb': 17060, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3564}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:02.023 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:02.023 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7674", 'reported_seq': 7785, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:22:58.308219+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:22:58.308219+0000', 'last_peered': '2024-09-15T21:22:58.308219+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:22:58.308219+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 7674, 'log_dups_size': 0, 'ondisk_log_size': 7674, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31412707, 'num_objects': 7670, 'num_object_clones': 0, 'num_object_copies': 15340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7670, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5533, 'num_write_kb': 22132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 3580}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:02.023 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:02.023 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'7704", 'reported_seq': 13351, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:22:58.369275+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:22:58.369275+0000', 'last_peered': '2024-09-15T21:22:58.369275+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:22:58.369275+0000', 'last_undegraded': '2024-09-15T21:22:58.369275+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 7704, 'log_dups_size': 0, 'ondisk_log_size': 7704, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31531008, 'num_objects': 7698, 'num_object_clones': 0, 'num_object_copies': 15396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1400, 'num_objects_unfound': 0, 'num_objects_dirty': 7698, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5397, 'num_write_kb': 21588, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2764, 'num_bytes_recovered': 11313152, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1400}, {'shards': '2', 'objects': 6298}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:02.023 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:02.024 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'7774", 'reported_seq': 7903, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:00.577381+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:23:00.577381+0000', 'last_peered': '2024-09-15T21:23:00.577381+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:23:00.577381+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 7774, 'log_dups_size': 0, 'ondisk_log_size': 7774, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31834112, 'num_objects': 7772, 'num_object_clones': 0, 'num_object_copies': 15544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7772, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4718, 'num_write_kb': 18872, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3678}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:02.024 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:02.024 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'7738", 'reported_seq': 7866, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:00.563415+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:23:00.563415+0000', 'last_peered': '2024-09-15T21:23:00.563415+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:23:00.563415+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 7738, 'log_dups_size': 0, 'ondisk_log_size': 7738, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31662080, 'num_objects': 7730, 'num_object_clones': 0, 'num_object_copies': 15460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7730, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4647, 'num_write_kb': 18588, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3591}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:02.024 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:02.024 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'7573", 'reported_seq': 7702, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:00.610472+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:23:00.610472+0000', 'last_peered': '2024-09-15T21:23:00.610472+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:23:00.610472+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 7573, 'log_dups_size': 0, 'ondisk_log_size': 7573, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31002624, 'num_objects': 7569, 'num_object_clones': 0, 'num_object_copies': 15138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7569, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4057, 'num_write_kb': 16228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3487}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:02.024 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:23:02.024 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'7740", 'reported_seq': 7874, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:00.588223+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:23:00.588223+0000', 'last_peered': '2024-09-15T21:23:00.588223+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:23:00.588223+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 7740, 'log_dups_size': 0, 'ondisk_log_size': 7740, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31654854, 'num_objects': 7730, 'num_object_clones': 0, 'num_object_copies': 15460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7730, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4444, 'num_write_kb': 17776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3583}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:02.025 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:02.695 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:02.696 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/246780 objects degraded (18.360%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":234,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":123390,"data_bytes":505361137,"bytes_used":1648283648,"bytes_avail":288262008832,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":246780,"degraded_ratio":0.18360077802090932,"misplaced_objects":1400,"misplaced_total":246780,"misplaced_ratio":0.0056730691303995464,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":114619,"recovering_keys_per_sec":0,"num_objects_recovered":336,"num_bytes_recovered":1376256,"num_keys_recovered":0,"write_bytes_sec":2236438,"read_op_per_sec":0,"write_op_per_sec":546},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:02.725 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:02.725 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:02.753 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 151 20 124824 124804 3.22808 1.77734 0.0541741 0.024195 2024-09-15T21:23:03.289 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:03.291 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:03.317 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-15T21:23:02.729234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":505361137,"num_objects":123390,"num_object_clones":0,"num_object_copies":246780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1400,"num_objects_unfound":0,"num_objects_dirty":123390,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":83255,"num_write_kb":333014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2766,"num_bytes_recovered":11317267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123463,"ondisk_log_size":123463,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1622076,"kb_used_data":999500,"kb_used_omap":12,"kb_used_meta":622515,"kb_avail":281493444,"statfs":{"total":289910292480,"available":288249286656,"internally_reserved":0,"allocated":1023488000,"data_stored":1022654061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":637455363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,8,6,6,7,14],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20418560,"num_objects":4985,"num_object_clones":0,"num_object_copies":9970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-336,"num_objects_unfound":0,"num_objects_dirty":4985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4985,"num_write_kb":19940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":1376256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007488"},"pg_stats":[{"pgid":"2.f","version":"43'7922","reported_seq":7992,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.602025+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:00.602025+0000","last_peered":"2024-09-15T21:23:00.602025+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:00.602025+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7922,"log_dups_size":0,"ondisk_log_size":7922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32448512,"num_objects":7922,"num_object_clones":0,"num_object_copies":15844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7922,"num_write_kb":31688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3790},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7675","reported_seq":7774,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.269396+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:22:58.269396+0000","last_peered":"2024-09-15T21:22:58.269396+0000","last_clean":"2024-09-15T21:22:58.269396+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:22:58.269396+0000","last_undegraded":"2024-09-15T21:22:58.269396+0000","last_fullsized":"2024-09-15T21:22:58.269396+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7675,"log_dups_size":0,"ondisk_log_size":7675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31413190,"num_objects":7671,"num_object_clones":0,"num_object_copies":15342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7671,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5956,"num_write_kb":23824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7855","reported_seq":7976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.344455+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:22:58.344455+0000","last_peered":"2024-09-15T21:22:58.344455+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:22:58.344455+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":7855,"log_dups_size":0,"ondisk_log_size":7855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32157696,"num_objects":7851,"num_object_clones":0,"num_object_copies":15702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5955,"num_write_kb":23820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3584}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7519","reported_seq":7585,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:22:58.317374+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:22:58.317374+0000","last_peered":"2024-09-15T21:22:58.317374+0000","last_clean":"2024-09-15T21:22:58.317374+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:22:58.317374+0000","last_undegraded":"2024-09-15T21:22:58.317374+0000","last_fullsized":"2024-09-15T21:22:58.317374+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7519,"log_dups_size":0,"ondisk_log_size":7519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30797824,"num_objects":7519,"num_object_clones":0,"num_object_copies":15038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7519,"num_write_kb":30076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7700","reported_seq":7828,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:00.558815+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:00.558815+0000","last_peered":"2024-09-15T21:23:00.558815+0000","last_clean":"2024-09-15T21:23:00.558815+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:00.558815+0000","last_undegraded":"2024-09-15T21:23:00.558815+0000","last_fullsized":"2024-09-15T21:23:00.558815+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7700,"log_dups_size":0,"ondisk_log_size":7700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31511011,"num_objects":7694,"num_object_clones":0,"num_object_copies":15388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4155,"num_write_kb":16620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7698","reported_seq":7829,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.522726+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:00.522726+0000","last_peered":"2024-09-15T21:23:00.522726+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:00.522726+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7698,"log_dups_size":0,"ondisk_log_size":7698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31511011,"num_objects":7694,"num_object_clones":0,"num_object_copies":15388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7694,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4374,"num_write_kb":17496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3618},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7772","reported_seq":7900,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.551599+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:00.551599+0000","last_peered":"2024-09-15T21:23:00.551599+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:00.551599+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7772,"log_dups_size":0,"ondisk_log_size":7772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31810502,"num_objects":7768,"num_object_clones":0,"num_object_copies":15536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4083,"num_write_kb":16332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3607},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7885","reported_seq":8009,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:00.491212+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:00.491212+0000","last_peered":"2024-09-15T21:23:00.491212+0000","last_clean":"2024-09-15T21:23:00.491212+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:00.491212+0000","last_undegraded":"2024-09-15T21:23:00.491212+0000","last_fullsized":"2024-09-15T21:23:00.491212+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":7885,"log_dups_size":0,"ondisk_log_size":7885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32288768,"num_objects":7883,"num_object_clones":0,"num_object_copies":15766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7883,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4657,"num_write_kb":18628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7639","reported_seq":7761,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.083880+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:22:58.083880+0000","last_peered":"2024-09-15T21:22:58.083880+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:22:58.083880+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7639,"log_dups_size":0,"ondisk_log_size":7639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31261155,"num_objects":7633,"num_object_clones":0,"num_object_copies":15266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5571,"num_write_kb":22284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3546}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7592","reported_seq":7718,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.596959+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:00.596959+0000","last_peered":"2024-09-15T21:23:00.596959+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:00.596959+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7592,"log_dups_size":0,"ondisk_log_size":7592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31064064,"num_objects":7584,"num_object_clones":0,"num_object_copies":15168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4265,"num_write_kb":17060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3564},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7674","reported_seq":7785,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:22:58.308219+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:22:58.308219+0000","last_peered":"2024-09-15T21:22:58.308219+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:22:58.308219+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7674,"log_dups_size":0,"ondisk_log_size":7674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31412707,"num_objects":7670,"num_object_clones":0,"num_object_copies":15340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7670,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5533,"num_write_kb":22132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3580}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7704","reported_seq":13351,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:22:58.369275+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:22:58.369275+0000","last_peered":"2024-09-15T21:22:58.369275+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:22:58.369275+0000","last_undegraded":"2024-09-15T21:22:58.369275+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7704,"log_dups_size":0,"ondisk_log_size":7704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31531008,"num_objects":7698,"num_object_clones":0,"num_object_copies":15396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1400,"num_objects_unfound":0,"num_objects_dirty":7698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5397,"num_write_kb":21588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2764,"num_bytes_recovered":11313152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1400},{"shards":"2","objects":6298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7774","reported_seq":7903,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.577381+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:00.577381+0000","last_peered":"2024-09-15T21:23:00.577381+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:00.577381+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7774,"log_dups_size":0,"ondisk_log_size":7774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31834112,"num_objects":7772,"num_object_clones":0,"num_object_copies":15544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4718,"num_write_kb":18872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3678},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7738","reported_seq":7866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.563415+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:00.563415+0000","last_peered":"2024-09-15T21:23:00.563415+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:00.563415+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7738,"log_dups_size":0,"ondisk_log_size":7738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31662080,"num_objects":7730,"num_object_clones":0,"num_object_copies":15460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4647,"num_write_kb":18588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3591},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7573","reported_seq":7702,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.610472+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:00.610472+0000","last_peered":"2024-09-15T21:23:00.610472+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:00.610472+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7573,"log_dups_size":0,"ondisk_log_size":7573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31002624,"num_objects":7569,"num_object_clones":0,"num_object_copies":15138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4057,"num_write_kb":16228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3487},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7740","reported_seq":7874,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:00.588223+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:00.588223+0000","last_peered":"2024-09-15T21:23:00.588223+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:00.588223+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7740,"log_dups_size":0,"ondisk_log_size":7740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31654854,"num_objects":7730,"num_object_clones":0,"num_object_copies":15460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7730,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4444,"num_write_kb":17776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3583},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":505361118,"num_objects":123388,"num_object_clones":0,"num_object_copies":246776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1400,"num_objects_unfound":0,"num_objects_dirty":123388,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":83253,"num_write_kb":333012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2765,"num_bytes_recovered":11317248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1025101824,"data_stored":1024953691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":123460,"ondisk_log_size":123460,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":557544,"kb_used_data":377304,"kb_used_omap":4,"kb_used_meta":180219,"kb_avail":93814296,"statfs":{"total":96636764160,"available":96065839104,"internally_reserved":0,"allocated":386359296,"data_stored":386097282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":184544937},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,0,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":702620,"kb_used_data":436364,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93669220,"statfs":{"total":96636764160,"available":95917281280,"internally_reserved":0,"allocated":446836736,"data_stored":446570645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":361912,"kb_used_data":185832,"kb_used_omap":4,"kb_used_meta":176059,"kb_avail":94009928,"statfs":{"total":96636764160,"available":96266166272,"internally_reserved":0,"allocated":190291968,"data_stored":189986134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180285103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447692800,"data_stored":447656670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":190996480,"data_stored":190920607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":386412544,"data_stored":386376414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:03.753 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 152 20 125287 125267 3.21874 1.80859 0.0344982 0.0242623 2024-09-15T21:23:04.753 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 153 20 125809 125789 3.21103 2.03906 0.0360988 0.0243224 2024-09-15T21:23:05.753 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 154 20 126327 126307 3.20332 2.02344 0.0498511 0.0243815 2024-09-15T21:23:06.319 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:06.753 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 155 20 126823 126803 3.19515 1.9375 0.00798092 0.0244445 2024-09-15T21:23:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:06.864 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:06.889 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-15T21:23:06.730664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":515265265,"num_objects":125808,"num_object_clones":0,"num_object_copies":251616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1229,"num_objects_unfound":0,"num_objects_dirty":125808,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":85673,"num_write_kb":342686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2937,"num_bytes_recovered":12017683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125881,"ondisk_log_size":125881,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1681016,"kb_used_data":1019592,"kb_used_omap":12,"kb_used_meta":661363,"kb_avail":281434504,"statfs":{"total":289910292480,"available":288188932096,"internally_reserved":0,"allocated":1044062208,"data_stored":1043228269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":677235715},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,1,3,2,9,12,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26529792,"num_objects":6477,"num_object_clones":0,"num_object_copies":12954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-345,"num_objects_unfound":0,"num_objects_dirty":6477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6477,"num_write_kb":25908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":1413120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007613"},"pg_stats":[{"pgid":"2.f","version":"43'8070","reported_seq":8140,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.588511+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:05.588511+0000","last_peered":"2024-09-15T21:23:05.588511+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:05.588511+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8070,"log_dups_size":0,"ondisk_log_size":8070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33054720,"num_objects":8070,"num_object_clones":0,"num_object_copies":16140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8070,"num_write_kb":32280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3938},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7836","reported_seq":7937,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.356268+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:03.356268+0000","last_peered":"2024-09-15T21:23:03.356268+0000","last_clean":"2024-09-15T21:23:03.356268+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:03.356268+0000","last_undegraded":"2024-09-15T21:23:03.356268+0000","last_fullsized":"2024-09-15T21:23:03.356268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7836,"log_dups_size":0,"ondisk_log_size":7836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32072646,"num_objects":7832,"num_object_clones":0,"num_object_copies":15664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7832,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6117,"num_write_kb":24468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8016","reported_seq":8137,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.240853+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:03.240853+0000","last_peered":"2024-09-15T21:23:03.240853+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:03.240853+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8016,"log_dups_size":0,"ondisk_log_size":8016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32817152,"num_objects":8012,"num_object_clones":0,"num_object_copies":16024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6116,"num_write_kb":24464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3745}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7660","reported_seq":7726,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.319064+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:03.319064+0000","last_peered":"2024-09-15T21:23:03.319064+0000","last_clean":"2024-09-15T21:23:03.319064+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:03.319064+0000","last_undegraded":"2024-09-15T21:23:03.319064+0000","last_fullsized":"2024-09-15T21:23:03.319064+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7660,"log_dups_size":0,"ondisk_log_size":7660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31375360,"num_objects":7660,"num_object_clones":0,"num_object_copies":15320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7660,"num_write_kb":30640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7856","reported_seq":7984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:05.613496+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:05.613496+0000","last_peered":"2024-09-15T21:23:05.613496+0000","last_clean":"2024-09-15T21:23:05.613496+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:05.613496+0000","last_undegraded":"2024-09-15T21:23:05.613496+0000","last_fullsized":"2024-09-15T21:23:05.613496+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7856,"log_dups_size":0,"ondisk_log_size":7856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149987,"num_objects":7850,"num_object_clones":0,"num_object_copies":15700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4311,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7866","reported_seq":7997,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.575758+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:05.575758+0000","last_peered":"2024-09-15T21:23:05.575758+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:05.575758+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7866,"log_dups_size":0,"ondisk_log_size":7866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32199139,"num_objects":7862,"num_object_clones":0,"num_object_copies":15724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4542,"num_write_kb":18168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7917","reported_seq":8048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.505016+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:05.505016+0000","last_peered":"2024-09-15T21:23:05.505016+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:05.505016+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7917,"log_dups_size":0,"ondisk_log_size":7917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32404422,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4228,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3752},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8036","reported_seq":8160,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:05.541914+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:05.541914+0000","last_peered":"2024-09-15T21:23:05.541914+0000","last_clean":"2024-09-15T21:23:05.541914+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:05.541914+0000","last_undegraded":"2024-09-15T21:23:05.541914+0000","last_fullsized":"2024-09-15T21:23:05.541914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8036,"log_dups_size":0,"ondisk_log_size":8036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32907264,"num_objects":8034,"num_object_clones":0,"num_object_copies":16068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4808,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7798","reported_seq":7920,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.276444+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:03.276444+0000","last_peered":"2024-09-15T21:23:03.276444+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:03.276444+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7798,"log_dups_size":0,"ondisk_log_size":7798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31912419,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5730,"num_write_kb":22920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7731","reported_seq":7857,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.638159+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:05.638159+0000","last_peered":"2024-09-15T21:23:05.638159+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:05.638159+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7731,"log_dups_size":0,"ondisk_log_size":7731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31633408,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4404,"num_write_kb":17616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3703},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7823","reported_seq":7935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.322577+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:03.322577+0000","last_peered":"2024-09-15T21:23:03.322577+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:03.322577+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7823,"log_dups_size":0,"ondisk_log_size":7823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32023011,"num_objects":7819,"num_object_clones":0,"num_object_copies":15638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7819,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5682,"num_write_kb":22728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3729}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7863","reported_seq":13852,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:03.369436+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:03.369436+0000","last_peered":"2024-09-15T21:23:03.369436+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:03.369436+0000","last_undegraded":"2024-09-15T21:23:03.369436+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7863,"log_dups_size":0,"ondisk_log_size":7863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32182272,"num_objects":7857,"num_object_clones":0,"num_object_copies":15714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1229,"num_objects_unfound":0,"num_objects_dirty":7857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5556,"num_write_kb":22224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2935,"num_bytes_recovered":12013568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1229},{"shards":"2","objects":6628}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7903","reported_seq":8032,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.624204+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:05.624204+0000","last_peered":"2024-09-15T21:23:05.624204+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:05.624204+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32362496,"num_objects":7901,"num_object_clones":0,"num_object_copies":15802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4847,"num_write_kb":19388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3807},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7872","reported_seq":8000,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.645974+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:05.645974+0000","last_peered":"2024-09-15T21:23:05.645974+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:05.645974+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7872,"log_dups_size":0,"ondisk_log_size":7872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32210944,"num_objects":7864,"num_object_clones":0,"num_object_copies":15728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4781,"num_write_kb":19124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3725},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7735","reported_seq":7866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.588433+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:05.588433+0000","last_peered":"2024-09-15T21:23:05.588433+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:05.588433+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7735,"log_dups_size":0,"ondisk_log_size":7735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4219,"num_write_kb":16876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3649},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7896","reported_seq":8030,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.608213+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:05.608213+0000","last_peered":"2024-09-15T21:23:05.608213+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:05.608213+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7896,"log_dups_size":0,"ondisk_log_size":7896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32293830,"num_objects":7886,"num_object_clones":0,"num_object_copies":15772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7886,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4600,"num_write_kb":18400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3739},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":515265246,"num_objects":125806,"num_object_clones":0,"num_object_copies":251612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1229,"num_objects_unfound":0,"num_objects_dirty":125806,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":85671,"num_write_kb":342684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2936,"num_bytes_recovered":12017664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1045876736,"data_stored":1045728603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125878,"ondisk_log_size":125878,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584772,"kb_used_data":382324,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93787068,"statfs":{"total":96636764160,"available":96037957632,"internally_reserved":0,"allocated":391499776,"data_stored":391237762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":207285929},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718012,"kb_used_data":443436,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93653828,"statfs":{"total":96636764160,"available":95901519872,"internally_reserved":0,"allocated":454078464,"data_stored":453812373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,0,1,2,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":378232,"kb_used_data":193832,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93993608,"statfs":{"total":96636764160,"available":96249454592,"internally_reserved":0,"allocated":198483968,"data_stored":198178134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":188804783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":455012352,"data_stored":454976222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199458816,"data_stored":199382943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391405568,"data_stored":391369438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:06.890 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:07.445 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:07.469 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-15T21:23:06.730664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":515265265,"num_objects":125808,"num_object_clones":0,"num_object_copies":251616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1229,"num_objects_unfound":0,"num_objects_dirty":125808,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":85673,"num_write_kb":342686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2937,"num_bytes_recovered":12017683,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125881,"ondisk_log_size":125881,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1681016,"kb_used_data":1019592,"kb_used_omap":12,"kb_used_meta":661363,"kb_avail":281434504,"statfs":{"total":289910292480,"available":288188932096,"internally_reserved":0,"allocated":1044062208,"data_stored":1043228269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":677235715},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,1,3,2,9,12,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26529792,"num_objects":6477,"num_object_clones":0,"num_object_copies":12954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-345,"num_objects_unfound":0,"num_objects_dirty":6477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6477,"num_write_kb":25908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":345,"num_bytes_recovered":1413120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007613"},"pg_stats":[{"pgid":"2.f","version":"43'8070","reported_seq":8140,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.588511+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:05.588511+0000","last_peered":"2024-09-15T21:23:05.588511+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:05.588511+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8070,"log_dups_size":0,"ondisk_log_size":8070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33054720,"num_objects":8070,"num_object_clones":0,"num_object_copies":16140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8070,"num_write_kb":32280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3938},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7836","reported_seq":7937,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.356268+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:03.356268+0000","last_peered":"2024-09-15T21:23:03.356268+0000","last_clean":"2024-09-15T21:23:03.356268+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:03.356268+0000","last_undegraded":"2024-09-15T21:23:03.356268+0000","last_fullsized":"2024-09-15T21:23:03.356268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7836,"log_dups_size":0,"ondisk_log_size":7836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32072646,"num_objects":7832,"num_object_clones":0,"num_object_copies":15664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7832,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6117,"num_write_kb":24468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8016","reported_seq":8137,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.240853+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:03.240853+0000","last_peered":"2024-09-15T21:23:03.240853+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:03.240853+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8016,"log_dups_size":0,"ondisk_log_size":8016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32817152,"num_objects":8012,"num_object_clones":0,"num_object_copies":16024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6116,"num_write_kb":24464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3745}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7660","reported_seq":7726,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:03.319064+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:03.319064+0000","last_peered":"2024-09-15T21:23:03.319064+0000","last_clean":"2024-09-15T21:23:03.319064+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:03.319064+0000","last_undegraded":"2024-09-15T21:23:03.319064+0000","last_fullsized":"2024-09-15T21:23:03.319064+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7660,"log_dups_size":0,"ondisk_log_size":7660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31375360,"num_objects":7660,"num_object_clones":0,"num_object_copies":15320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7660,"num_write_kb":30640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7856","reported_seq":7984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:05.613496+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:05.613496+0000","last_peered":"2024-09-15T21:23:05.613496+0000","last_clean":"2024-09-15T21:23:05.613496+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:05.613496+0000","last_undegraded":"2024-09-15T21:23:05.613496+0000","last_fullsized":"2024-09-15T21:23:05.613496+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7856,"log_dups_size":0,"ondisk_log_size":7856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149987,"num_objects":7850,"num_object_clones":0,"num_object_copies":15700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4311,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7866","reported_seq":7997,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.575758+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:05.575758+0000","last_peered":"2024-09-15T21:23:05.575758+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:05.575758+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7866,"log_dups_size":0,"ondisk_log_size":7866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32199139,"num_objects":7862,"num_object_clones":0,"num_object_copies":15724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4542,"num_write_kb":18168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7917","reported_seq":8048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.505016+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:05.505016+0000","last_peered":"2024-09-15T21:23:05.505016+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:05.505016+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7917,"log_dups_size":0,"ondisk_log_size":7917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32404422,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4228,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3752},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8036","reported_seq":8160,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:05.541914+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:05.541914+0000","last_peered":"2024-09-15T21:23:05.541914+0000","last_clean":"2024-09-15T21:23:05.541914+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:05.541914+0000","last_undegraded":"2024-09-15T21:23:05.541914+0000","last_fullsized":"2024-09-15T21:23:05.541914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8036,"log_dups_size":0,"ondisk_log_size":8036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32907264,"num_objects":8034,"num_object_clones":0,"num_object_copies":16068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4808,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7798","reported_seq":7920,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.276444+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:03.276444+0000","last_peered":"2024-09-15T21:23:03.276444+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:03.276444+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7798,"log_dups_size":0,"ondisk_log_size":7798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31912419,"num_objects":7792,"num_object_clones":0,"num_object_copies":15584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7792,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5730,"num_write_kb":22920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3705}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7731","reported_seq":7857,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.638159+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:05.638159+0000","last_peered":"2024-09-15T21:23:05.638159+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:05.638159+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7731,"log_dups_size":0,"ondisk_log_size":7731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31633408,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4404,"num_write_kb":17616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3703},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7823","reported_seq":7935,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:03.322577+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:03.322577+0000","last_peered":"2024-09-15T21:23:03.322577+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:03.322577+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7823,"log_dups_size":0,"ondisk_log_size":7823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32023011,"num_objects":7819,"num_object_clones":0,"num_object_copies":15638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7819,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5682,"num_write_kb":22728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3729}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7863","reported_seq":13852,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:03.369436+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:03.369436+0000","last_peered":"2024-09-15T21:23:03.369436+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:03.369436+0000","last_undegraded":"2024-09-15T21:23:03.369436+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":7863,"log_dups_size":0,"ondisk_log_size":7863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32182272,"num_objects":7857,"num_object_clones":0,"num_object_copies":15714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1229,"num_objects_unfound":0,"num_objects_dirty":7857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5556,"num_write_kb":22224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2935,"num_bytes_recovered":12013568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1229},{"shards":"2","objects":6628}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7903","reported_seq":8032,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.624204+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:05.624204+0000","last_peered":"2024-09-15T21:23:05.624204+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:05.624204+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32362496,"num_objects":7901,"num_object_clones":0,"num_object_copies":15802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4847,"num_write_kb":19388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3807},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7872","reported_seq":8000,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.645974+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:05.645974+0000","last_peered":"2024-09-15T21:23:05.645974+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:05.645974+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7872,"log_dups_size":0,"ondisk_log_size":7872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32210944,"num_objects":7864,"num_object_clones":0,"num_object_copies":15728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4781,"num_write_kb":19124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3725},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7735","reported_seq":7866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.588433+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:05.588433+0000","last_peered":"2024-09-15T21:23:05.588433+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:05.588433+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7735,"log_dups_size":0,"ondisk_log_size":7735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4219,"num_write_kb":16876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3649},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7896","reported_seq":8030,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.608213+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:05.608213+0000","last_peered":"2024-09-15T21:23:05.608213+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:05.608213+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7896,"log_dups_size":0,"ondisk_log_size":7896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32293830,"num_objects":7886,"num_object_clones":0,"num_object_copies":15772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7886,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4600,"num_write_kb":18400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3739},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":515265246,"num_objects":125806,"num_object_clones":0,"num_object_copies":251612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1229,"num_objects_unfound":0,"num_objects_dirty":125806,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":85671,"num_write_kb":342684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2936,"num_bytes_recovered":12017664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1045876736,"data_stored":1045728603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":125878,"ondisk_log_size":125878,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584772,"kb_used_data":382324,"kb_used_omap":4,"kb_used_meta":202427,"kb_avail":93787068,"statfs":{"total":96636764160,"available":96037957632,"internally_reserved":0,"allocated":391499776,"data_stored":391237762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":207285929},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,4,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718012,"kb_used_data":443436,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93653828,"statfs":{"total":96636764160,"available":95901519872,"internally_reserved":0,"allocated":454078464,"data_stored":453812373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,0,1,2,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":378232,"kb_used_data":193832,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93993608,"statfs":{"total":96636764160,"available":96249454592,"internally_reserved":0,"allocated":198483968,"data_stored":198178134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":188804783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":455012352,"data_stored":454976222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199458816,"data_stored":199382943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":391405568,"data_stored":391369438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:07.470 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:07.470 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8070", 'reported_seq': 8140, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:05.588511+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:23:05.588511+0000', 'last_peered': '2024-09-15T21:23:05.588511+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:23:05.588511+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 8070, 'log_dups_size': 0, 'ondisk_log_size': 8070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33054720, 'num_objects': 8070, 'num_object_clones': 0, 'num_object_copies': 16140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8070, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8070, 'num_write_kb': 32280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3938}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:07.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:23:07.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8016", 'reported_seq': 8137, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:03.240853+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:23:03.240853+0000', 'last_peered': '2024-09-15T21:23:03.240853+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:23:03.240853+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 8016, 'log_dups_size': 0, 'ondisk_log_size': 8016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32817152, 'num_objects': 8012, 'num_object_clones': 0, 'num_object_copies': 16024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8012, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6116, 'num_write_kb': 24464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 3745}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:07.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:07.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'7866", 'reported_seq': 7997, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:05.575758+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:23:05.575758+0000', 'last_peered': '2024-09-15T21:23:05.575758+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:23:05.575758+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 7866, 'log_dups_size': 0, 'ondisk_log_size': 7866, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32199139, 'num_objects': 7862, 'num_object_clones': 0, 'num_object_copies': 15724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7862, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4542, 'num_write_kb': 18168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3786}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:07.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:07.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'7917", 'reported_seq': 8048, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:05.505016+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:23:05.505016+0000', 'last_peered': '2024-09-15T21:23:05.505016+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:23:05.505016+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 7917, 'log_dups_size': 0, 'ondisk_log_size': 7917, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32404422, 'num_objects': 7913, 'num_object_clones': 0, 'num_object_copies': 15826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7913, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4228, 'num_write_kb': 16912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3752}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:07.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:07.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7798", 'reported_seq': 7920, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:03.276444+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:23:03.276444+0000', 'last_peered': '2024-09-15T21:23:03.276444+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:23:03.276444+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 7798, 'log_dups_size': 0, 'ondisk_log_size': 7798, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31912419, 'num_objects': 7792, 'num_object_clones': 0, 'num_object_copies': 15584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7792, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5730, 'num_write_kb': 22920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 3705}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:07.471 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:07.471 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'7731", 'reported_seq': 7857, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:05.638159+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:23:05.638159+0000', 'last_peered': '2024-09-15T21:23:05.638159+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:23:05.638159+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 7731, 'log_dups_size': 0, 'ondisk_log_size': 7731, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31633408, 'num_objects': 7723, 'num_object_clones': 0, 'num_object_copies': 15446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7723, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4404, 'num_write_kb': 17616, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3703}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:07.472 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:07.472 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7823", 'reported_seq': 7935, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:03.322577+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:23:03.322577+0000', 'last_peered': '2024-09-15T21:23:03.322577+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:23:03.322577+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 7823, 'log_dups_size': 0, 'ondisk_log_size': 7823, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32023011, 'num_objects': 7819, 'num_object_clones': 0, 'num_object_copies': 15638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7819, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5682, 'num_write_kb': 22728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 3729}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:07.472 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:07.472 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'7863", 'reported_seq': 13852, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:23:03.369436+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:23:03.369436+0000', 'last_peered': '2024-09-15T21:23:03.369436+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:23:03.369436+0000', 'last_undegraded': '2024-09-15T21:23:03.369436+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 7863, 'log_dups_size': 0, 'ondisk_log_size': 7863, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32182272, 'num_objects': 7857, 'num_object_clones': 0, 'num_object_copies': 15714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1229, 'num_objects_unfound': 0, 'num_objects_dirty': 7857, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5556, 'num_write_kb': 22224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2935, 'num_bytes_recovered': 12013568, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1229}, {'shards': '2', 'objects': 6628}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:07.472 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:07.472 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'7903", 'reported_seq': 8032, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:05.624204+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:23:05.624204+0000', 'last_peered': '2024-09-15T21:23:05.624204+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:23:05.624204+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 7903, 'log_dups_size': 0, 'ondisk_log_size': 7903, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32362496, 'num_objects': 7901, 'num_object_clones': 0, 'num_object_copies': 15802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7901, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4847, 'num_write_kb': 19388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3807}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:07.472 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:07.472 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'7872", 'reported_seq': 8000, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:05.645974+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:23:05.645974+0000', 'last_peered': '2024-09-15T21:23:05.645974+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:23:05.645974+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 7872, 'log_dups_size': 0, 'ondisk_log_size': 7872, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32210944, 'num_objects': 7864, 'num_object_clones': 0, 'num_object_copies': 15728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7864, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4781, 'num_write_kb': 19124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3725}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:07.472 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:07.472 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'7735", 'reported_seq': 7866, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:05.588433+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:23:05.588433+0000', 'last_peered': '2024-09-15T21:23:05.588433+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:23:05.588433+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 7735, 'log_dups_size': 0, 'ondisk_log_size': 7735, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31666176, 'num_objects': 7731, 'num_object_clones': 0, 'num_object_copies': 15462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7731, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4219, 'num_write_kb': 16876, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3649}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:07.472 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:23:07.473 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'7896", 'reported_seq': 8030, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:05.608213+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:23:05.608213+0000', 'last_peered': '2024-09-15T21:23:05.608213+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:23:05.608213+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 7896, 'log_dups_size': 0, 'ondisk_log_size': 7896, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32293830, 'num_objects': 7886, 'num_object_clones': 0, 'num_object_copies': 15772, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7886, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4600, 'num_write_kb': 18400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3739}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:07.473 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:07.754 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 156 20 127275 127255 3.18599 1.76562 0.0294457 0.0245138 2024-09-15T21:23:08.178 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:08.178 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/251616 objects degraded (18.007%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":239,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":125808,"data_bytes":515265265,"bytes_used":1721360384,"bytes_avail":288188932096,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":251616,"degraded_ratio":0.18007201449828311,"misplaced_objects":1229,"misplaced_total":251616,"misplaced_ratio":0.0048844270634617829,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":117685,"recovering_keys_per_sec":0,"num_objects_recovered":345,"num_bytes_recovered":1413120,"num_keys_recovered":0,"write_bytes_sec":2209414,"read_op_per_sec":0,"write_op_per_sec":539},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:08.207 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:08.207 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:08.753 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:08.755 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 157 20 127721 127701 3.17679 1.74219 0.0225189 0.0245834 2024-09-15T21:23:08.755 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:08.784 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-15T21:23:08.731358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":519164657,"num_objects":126760,"num_object_clones":0,"num_object_copies":253520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1063,"num_objects_unfound":0,"num_objects_dirty":126760,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":86625,"num_write_kb":346494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3103,"num_bytes_recovered":12697619,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126833,"ondisk_log_size":126833,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1727720,"kb_used_data":1024632,"kb_used_omap":12,"kb_used_meta":703027,"kb_avail":281387800,"statfs":{"total":289910292480,"available":288141107200,"internally_reserved":0,"allocated":1049223168,"data_stored":1048389229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":719899651},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,1,0,2,7,12,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23957504,"num_objects":5849,"num_object_clones":0,"num_object_copies":11698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-511,"num_objects_unfound":0,"num_objects_dirty":5849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5849,"num_write_kb":23396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":511,"num_bytes_recovered":2093056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004215"},"pg_stats":[{"pgid":"2.f","version":"43'8070","reported_seq":8140,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.588511+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:05.588511+0000","last_peered":"2024-09-15T21:23:05.588511+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:05.588511+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8070,"log_dups_size":0,"ondisk_log_size":8070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33054720,"num_objects":8070,"num_object_clones":0,"num_object_copies":16140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8070,"num_write_kb":32280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3938},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7992","reported_seq":8093,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:08.368806+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:08.368806+0000","last_peered":"2024-09-15T21:23:08.368806+0000","last_clean":"2024-09-15T21:23:08.368806+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:08.368806+0000","last_undegraded":"2024-09-15T21:23:08.368806+0000","last_fullsized":"2024-09-15T21:23:08.368806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7992,"log_dups_size":0,"ondisk_log_size":7992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32711622,"num_objects":7988,"num_object_clones":0,"num_object_copies":15976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7988,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6273,"num_write_kb":25092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8181","reported_seq":8302,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.328712+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:08.328712+0000","last_peered":"2024-09-15T21:23:08.328712+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:08.328712+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8181,"log_dups_size":0,"ondisk_log_size":8181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33492992,"num_objects":8177,"num_object_clones":0,"num_object_copies":16354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6281,"num_write_kb":25124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3910}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7817","reported_seq":7883,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:08.339055+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:08.339055+0000","last_peered":"2024-09-15T21:23:08.339055+0000","last_clean":"2024-09-15T21:23:08.339055+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:08.339055+0000","last_undegraded":"2024-09-15T21:23:08.339055+0000","last_fullsized":"2024-09-15T21:23:08.339055+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7817,"log_dups_size":0,"ondisk_log_size":7817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32018432,"num_objects":7817,"num_object_clones":0,"num_object_copies":15634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7817,"num_write_kb":31268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7856","reported_seq":7984,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:05.613496+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:05.613496+0000","last_peered":"2024-09-15T21:23:05.613496+0000","last_clean":"2024-09-15T21:23:05.613496+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:05.613496+0000","last_undegraded":"2024-09-15T21:23:05.613496+0000","last_fullsized":"2024-09-15T21:23:05.613496+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7856,"log_dups_size":0,"ondisk_log_size":7856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149987,"num_objects":7850,"num_object_clones":0,"num_object_copies":15700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4311,"num_write_kb":17244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7866","reported_seq":7997,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.575758+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:05.575758+0000","last_peered":"2024-09-15T21:23:05.575758+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:05.575758+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":7866,"log_dups_size":0,"ondisk_log_size":7866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32199139,"num_objects":7862,"num_object_clones":0,"num_object_copies":15724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7862,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4542,"num_write_kb":18168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3786},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7917","reported_seq":8048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.505016+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:05.505016+0000","last_peered":"2024-09-15T21:23:05.505016+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:05.505016+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":7917,"log_dups_size":0,"ondisk_log_size":7917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32404422,"num_objects":7913,"num_object_clones":0,"num_object_copies":15826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4228,"num_write_kb":16912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3752},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8036","reported_seq":8160,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:05.541914+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:05.541914+0000","last_peered":"2024-09-15T21:23:05.541914+0000","last_clean":"2024-09-15T21:23:05.541914+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:05.541914+0000","last_undegraded":"2024-09-15T21:23:05.541914+0000","last_fullsized":"2024-09-15T21:23:05.541914+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8036,"log_dups_size":0,"ondisk_log_size":8036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32907264,"num_objects":8034,"num_object_clones":0,"num_object_copies":16068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4808,"num_write_kb":19232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7960","reported_seq":8082,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.256984+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:08.256984+0000","last_peered":"2024-09-15T21:23:08.256984+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:08.256984+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7960,"log_dups_size":0,"ondisk_log_size":7960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32575971,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5892,"num_write_kb":23568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3867}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7731","reported_seq":7857,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.638159+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:05.638159+0000","last_peered":"2024-09-15T21:23:05.638159+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:05.638159+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7731,"log_dups_size":0,"ondisk_log_size":7731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31633408,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4404,"num_write_kb":17616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3703},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7980","reported_seq":8094,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.272042+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:08.272042+0000","last_peered":"2024-09-15T21:23:08.272042+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:08.272042+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7980,"log_dups_size":0,"ondisk_log_size":7980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32666083,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5839,"num_write_kb":23356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3886}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8018","reported_seq":14339,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:08.375342+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:08.375342+0000","last_peered":"2024-09-15T21:23:08.375342+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:08.375342+0000","last_undegraded":"2024-09-15T21:23:08.375342+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8018,"log_dups_size":0,"ondisk_log_size":8018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32817152,"num_objects":8012,"num_object_clones":0,"num_object_copies":16024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1063,"num_objects_unfound":0,"num_objects_dirty":8012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5711,"num_write_kb":22844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3101,"num_bytes_recovered":12693504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1063},{"shards":"2","objects":6949}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7903","reported_seq":8032,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.624204+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:05.624204+0000","last_peered":"2024-09-15T21:23:05.624204+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:05.624204+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32362496,"num_objects":7901,"num_object_clones":0,"num_object_copies":15802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4847,"num_write_kb":19388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3807},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7872","reported_seq":8000,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.645974+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:05.645974+0000","last_peered":"2024-09-15T21:23:05.645974+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:05.645974+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":7872,"log_dups_size":0,"ondisk_log_size":7872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32210944,"num_objects":7864,"num_object_clones":0,"num_object_copies":15728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4781,"num_write_kb":19124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3725},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7735","reported_seq":7866,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.588433+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:05.588433+0000","last_peered":"2024-09-15T21:23:05.588433+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:05.588433+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7735,"log_dups_size":0,"ondisk_log_size":7735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4219,"num_write_kb":16876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3649},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7896","reported_seq":8030,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:05.608213+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:05.608213+0000","last_peered":"2024-09-15T21:23:05.608213+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:05.608213+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":7896,"log_dups_size":0,"ondisk_log_size":7896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32293830,"num_objects":7886,"num_object_clones":0,"num_object_copies":15772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7886,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4600,"num_write_kb":18400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3739},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":519164638,"num_objects":126758,"num_object_clones":0,"num_object_copies":253516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1063,"num_objects_unfound":0,"num_objects_dirty":126758,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":86623,"num_write_kb":346492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3102,"num_bytes_recovered":12697600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1050984448,"data_stored":1050836315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":126830,"ondisk_log_size":126830,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631476,"kb_used_data":387364,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93740364,"statfs":{"total":96636764160,"available":95990132736,"internally_reserved":0,"allocated":396660736,"data_stored":396398722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":249949865},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718012,"kb_used_data":443436,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93653828,"statfs":{"total":96636764160,"available":95901519872,"internally_reserved":0,"allocated":454078464,"data_stored":453812373,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,0,0,1,2,4,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":378232,"kb_used_data":193832,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93993608,"statfs":{"total":96636764160,"available":96249454592,"internally_reserved":0,"allocated":198483968,"data_stored":198178134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":188804783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":455012352,"data_stored":454976222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199458816,"data_stored":199382943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396513280,"data_stored":396477150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:09.754 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 158 20 128227 128207 3.16919 1.97656 0.0220995 0.0246422 2024-09-15T21:23:10.754 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 159 20 128737 128717 3.16179 1.99219 0.0250415 0.0247016 2024-09-15T21:23:11.754 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:2024-09-15T21:23:11.752069+0000 min lat: 0.00235499 max lat: 2.60418 avg lat: 0.0247578 2024-09-15T21:23:11.754 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:23:11.754 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 160 20 129249 129229 3.15453 2 0.0194625 0.0247578 2024-09-15T21:23:11.785 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:12.335 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:12.336 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:12.358 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-15T21:23:10.732100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525452017,"num_objects":128295,"num_object_clones":0,"num_object_copies":256590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1063,"num_objects_unfound":0,"num_objects_dirty":128295,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":88160,"num_write_kb":352634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3103,"num_bytes_recovered":12697619,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128368,"ondisk_log_size":128368,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1740580,"kb_used_data":1031924,"kb_used_omap":12,"kb_used_meta":708595,"kb_avail":281374940,"statfs":{"total":289910292480,"available":288127938560,"internally_reserved":0,"allocated":1056690176,"data_stored":1055856237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":725601283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,13,11,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26562560,"num_objects":6485,"num_object_clones":0,"num_object_copies":12970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":6485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6485,"num_write_kb":25940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004290"},"pg_stats":[{"pgid":"2.f","version":"43'8239","reported_seq":8310,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.638110+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:10.638110+0000","last_peered":"2024-09-15T21:23:10.638110+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:10.638110+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8239,"log_dups_size":0,"ondisk_log_size":8239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33746944,"num_objects":8239,"num_object_clones":0,"num_object_copies":16478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8239,"num_write_kb":32956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4107},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7992","reported_seq":8093,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:08.368806+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:08.368806+0000","last_peered":"2024-09-15T21:23:08.368806+0000","last_clean":"2024-09-15T21:23:08.368806+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:08.368806+0000","last_undegraded":"2024-09-15T21:23:08.368806+0000","last_fullsized":"2024-09-15T21:23:08.368806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7992,"log_dups_size":0,"ondisk_log_size":7992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32711622,"num_objects":7988,"num_object_clones":0,"num_object_copies":15976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7988,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6273,"num_write_kb":25092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8181","reported_seq":8302,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.328712+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:08.328712+0000","last_peered":"2024-09-15T21:23:08.328712+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:08.328712+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8181,"log_dups_size":0,"ondisk_log_size":8181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33492992,"num_objects":8177,"num_object_clones":0,"num_object_copies":16354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6281,"num_write_kb":25124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3910}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7817","reported_seq":7883,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:08.339055+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:08.339055+0000","last_peered":"2024-09-15T21:23:08.339055+0000","last_clean":"2024-09-15T21:23:08.339055+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:08.339055+0000","last_undegraded":"2024-09-15T21:23:08.339055+0000","last_fullsized":"2024-09-15T21:23:08.339055+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7817,"log_dups_size":0,"ondisk_log_size":7817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32018432,"num_objects":7817,"num_object_clones":0,"num_object_copies":15634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7817,"num_write_kb":31268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7992","reported_seq":8120,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:10.587953+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:10.587953+0000","last_peered":"2024-09-15T21:23:10.587953+0000","last_clean":"2024-09-15T21:23:10.587953+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:10.587953+0000","last_undegraded":"2024-09-15T21:23:10.587953+0000","last_fullsized":"2024-09-15T21:23:10.587953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7992,"log_dups_size":0,"ondisk_log_size":7992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32707043,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4447,"num_write_kb":17788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8032","reported_seq":8162,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.698493+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:10.698493+0000","last_peered":"2024-09-15T21:23:10.698493+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:10.698493+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8032,"log_dups_size":0,"ondisk_log_size":8032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32879075,"num_objects":8028,"num_object_clones":0,"num_object_copies":16056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4708,"num_write_kb":18832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3952},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8059","reported_seq":8190,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.677680+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:10.677680+0000","last_peered":"2024-09-15T21:23:10.677680+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:10.677680+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8059,"log_dups_size":0,"ondisk_log_size":8059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32986054,"num_objects":8055,"num_object_clones":0,"num_object_copies":16110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4370,"num_write_kb":17480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3894},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8177","reported_seq":8301,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:10.609353+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:10.609353+0000","last_peered":"2024-09-15T21:23:10.609353+0000","last_clean":"2024-09-15T21:23:10.609353+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:10.609353+0000","last_undegraded":"2024-09-15T21:23:10.609353+0000","last_fullsized":"2024-09-15T21:23:10.609353+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8177,"log_dups_size":0,"ondisk_log_size":8177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33484800,"num_objects":8175,"num_object_clones":0,"num_object_copies":16350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4949,"num_write_kb":19796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.357547+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.357547+0000","last_peered":"2024-09-15T21:21:14.357547+0000","last_clean":"2024-09-15T21:21:14.357547+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.357547+0000","last_undegraded":"2024-09-15T21:21:14.357547+0000","last_fullsized":"2024-09-15T21:21:14.357547+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7960","reported_seq":8082,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.256984+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:08.256984+0000","last_peered":"2024-09-15T21:23:08.256984+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:08.256984+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7960,"log_dups_size":0,"ondisk_log_size":7960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32575971,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5892,"num_write_kb":23568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3867}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7899","reported_seq":8021,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.655220+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:10.655220+0000","last_peered":"2024-09-15T21:23:10.655220+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:10.655220+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7899,"log_dups_size":0,"ondisk_log_size":7899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32321536,"num_objects":7891,"num_object_clones":0,"num_object_copies":15782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4572,"num_write_kb":18288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3871},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7980","reported_seq":8094,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.272042+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:08.272042+0000","last_peered":"2024-09-15T21:23:08.272042+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:08.272042+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7980,"log_dups_size":0,"ondisk_log_size":7980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32666083,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5839,"num_write_kb":23356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3886}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8018","reported_seq":14339,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:08.375342+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:08.375342+0000","last_peered":"2024-09-15T21:23:08.375342+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:08.375342+0000","last_undegraded":"2024-09-15T21:23:08.375342+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8018,"log_dups_size":0,"ondisk_log_size":8018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32817152,"num_objects":8012,"num_object_clones":0,"num_object_copies":16024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1063,"num_objects_unfound":0,"num_objects_dirty":8012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5711,"num_write_kb":22844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3101,"num_bytes_recovered":12693504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1063},{"shards":"2","objects":6949}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8042","reported_seq":8168,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.688428+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:10.688428+0000","last_peered":"2024-09-15T21:23:10.688428+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:10.688428+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8042,"log_dups_size":0,"ondisk_log_size":8042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32931840,"num_objects":8040,"num_object_clones":0,"num_object_copies":16080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4986,"num_write_kb":19944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3946},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8041","reported_seq":8169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.660655+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:10.660655+0000","last_peered":"2024-09-15T21:23:10.660655+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:10.660655+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8041,"log_dups_size":0,"ondisk_log_size":8041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32903168,"num_objects":8033,"num_object_clones":0,"num_object_copies":16066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4950,"num_write_kb":19800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3894},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7894","reported_seq":8025,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.663301+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:10.663301+0000","last_peered":"2024-09-15T21:23:10.663301+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:10.663301+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7894,"log_dups_size":0,"ondisk_log_size":7894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32317440,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4378,"num_write_kb":17512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3808},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8042","reported_seq":8176,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.582316+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:10.582316+0000","last_peered":"2024-09-15T21:23:10.582316+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:10.582316+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8042,"log_dups_size":0,"ondisk_log_size":8042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32891846,"num_objects":8032,"num_object_clones":0,"num_object_copies":16064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8032,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4746,"num_write_kb":18984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3885},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525451998,"num_objects":128293,"num_object_clones":0,"num_object_copies":256586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1063,"num_objects_unfound":0,"num_objects_dirty":128293,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":88158,"num_write_kb":352632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3102,"num_bytes_recovered":12697600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1058529280,"data_stored":1058381147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128365,"ondisk_log_size":128365,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631476,"kb_used_data":387364,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93740364,"statfs":{"total":96636764160,"available":95990132736,"internally_reserved":0,"allocated":396660736,"data_stored":396398722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":249949865},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730872,"kb_used_data":450728,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93640968,"statfs":{"total":96636764160,"available":95888351232,"internally_reserved":0,"allocated":461545472,"data_stored":461279381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286846635},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,8,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":378232,"kb_used_data":193832,"kb_used_omap":4,"kb_used_meta":184379,"kb_avail":93993608,"statfs":{"total":96636764160,"available":96249454592,"internally_reserved":0,"allocated":198483968,"data_stored":198178134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":188804783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462557184,"data_stored":462521054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":199458816,"data_stored":199382943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396513280,"data_stored":396477150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:12.359 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:12.758 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 161 20 129703 129683 3.14592 1.77344 0.0170016 0.0248241 2024-09-15T21:23:12.913 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:12.914 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:12.950 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-15T21:23:12.732911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525452017,"num_objects":128295,"num_object_clones":0,"num_object_copies":256590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1063,"num_objects_unfound":0,"num_objects_dirty":128295,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":88160,"num_write_kb":352634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3103,"num_bytes_recovered":12697619,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128368,"ondisk_log_size":128368,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1736396,"kb_used_data":1040220,"kb_used_omap":12,"kb_used_meta":696115,"kb_avail":281379124,"statfs":{"total":289910292480,"available":288132222976,"internally_reserved":0,"allocated":1065185280,"data_stored":1064351341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":712821763},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9,17,10,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20090880,"num_objects":4905,"num_object_clones":0,"num_object_copies":9810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":4905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4905,"num_write_kb":19620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004448"},"pg_stats":[{"pgid":"2.f","version":"43'8239","reported_seq":8310,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.638110+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:10.638110+0000","last_peered":"2024-09-15T21:23:10.638110+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:10.638110+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8239,"log_dups_size":0,"ondisk_log_size":8239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33746944,"num_objects":8239,"num_object_clones":0,"num_object_copies":16478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8239,"num_write_kb":32956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4107},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7992","reported_seq":8093,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:08.368806+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:08.368806+0000","last_peered":"2024-09-15T21:23:08.368806+0000","last_clean":"2024-09-15T21:23:08.368806+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:08.368806+0000","last_undegraded":"2024-09-15T21:23:08.368806+0000","last_fullsized":"2024-09-15T21:23:08.368806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7992,"log_dups_size":0,"ondisk_log_size":7992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32711622,"num_objects":7988,"num_object_clones":0,"num_object_copies":15976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7988,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6273,"num_write_kb":25092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8181","reported_seq":8302,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.328712+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:08.328712+0000","last_peered":"2024-09-15T21:23:08.328712+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:08.328712+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8181,"log_dups_size":0,"ondisk_log_size":8181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33492992,"num_objects":8177,"num_object_clones":0,"num_object_copies":16354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6281,"num_write_kb":25124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3910}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7817","reported_seq":7883,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:08.339055+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:08.339055+0000","last_peered":"2024-09-15T21:23:08.339055+0000","last_clean":"2024-09-15T21:23:08.339055+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:08.339055+0000","last_undegraded":"2024-09-15T21:23:08.339055+0000","last_fullsized":"2024-09-15T21:23:08.339055+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7817,"log_dups_size":0,"ondisk_log_size":7817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32018432,"num_objects":7817,"num_object_clones":0,"num_object_copies":15634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7817,"num_write_kb":31268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7992","reported_seq":8120,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:10.587953+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:10.587953+0000","last_peered":"2024-09-15T21:23:10.587953+0000","last_clean":"2024-09-15T21:23:10.587953+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:10.587953+0000","last_undegraded":"2024-09-15T21:23:10.587953+0000","last_fullsized":"2024-09-15T21:23:10.587953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7992,"log_dups_size":0,"ondisk_log_size":7992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32707043,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4447,"num_write_kb":17788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8032","reported_seq":8162,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.698493+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:10.698493+0000","last_peered":"2024-09-15T21:23:10.698493+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:10.698493+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8032,"log_dups_size":0,"ondisk_log_size":8032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32879075,"num_objects":8028,"num_object_clones":0,"num_object_copies":16056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4708,"num_write_kb":18832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3952},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8059","reported_seq":8190,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.677680+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:10.677680+0000","last_peered":"2024-09-15T21:23:10.677680+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:10.677680+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8059,"log_dups_size":0,"ondisk_log_size":8059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32986054,"num_objects":8055,"num_object_clones":0,"num_object_copies":16110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4370,"num_write_kb":17480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3894},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8177","reported_seq":8301,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:10.609353+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:10.609353+0000","last_peered":"2024-09-15T21:23:10.609353+0000","last_clean":"2024-09-15T21:23:10.609353+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:10.609353+0000","last_undegraded":"2024-09-15T21:23:10.609353+0000","last_fullsized":"2024-09-15T21:23:10.609353+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8177,"log_dups_size":0,"ondisk_log_size":8177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33484800,"num_objects":8175,"num_object_clones":0,"num_object_copies":16350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4949,"num_write_kb":19796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7960","reported_seq":8082,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.256984+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:08.256984+0000","last_peered":"2024-09-15T21:23:08.256984+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:08.256984+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7960,"log_dups_size":0,"ondisk_log_size":7960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32575971,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5892,"num_write_kb":23568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3867}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7899","reported_seq":8021,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.655220+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:10.655220+0000","last_peered":"2024-09-15T21:23:10.655220+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:10.655220+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7899,"log_dups_size":0,"ondisk_log_size":7899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32321536,"num_objects":7891,"num_object_clones":0,"num_object_copies":15782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4572,"num_write_kb":18288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3871},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7980","reported_seq":8094,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.272042+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:08.272042+0000","last_peered":"2024-09-15T21:23:08.272042+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:08.272042+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7980,"log_dups_size":0,"ondisk_log_size":7980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32666083,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5839,"num_write_kb":23356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3886}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8018","reported_seq":14339,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:08.375342+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:08.375342+0000","last_peered":"2024-09-15T21:23:08.375342+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:08.375342+0000","last_undegraded":"2024-09-15T21:23:08.375342+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8018,"log_dups_size":0,"ondisk_log_size":8018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32817152,"num_objects":8012,"num_object_clones":0,"num_object_copies":16024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1063,"num_objects_unfound":0,"num_objects_dirty":8012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5711,"num_write_kb":22844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3101,"num_bytes_recovered":12693504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1063},{"shards":"2","objects":6949}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8042","reported_seq":8168,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.688428+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:10.688428+0000","last_peered":"2024-09-15T21:23:10.688428+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:10.688428+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8042,"log_dups_size":0,"ondisk_log_size":8042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32931840,"num_objects":8040,"num_object_clones":0,"num_object_copies":16080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4986,"num_write_kb":19944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3946},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8041","reported_seq":8169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.660655+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:10.660655+0000","last_peered":"2024-09-15T21:23:10.660655+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:10.660655+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8041,"log_dups_size":0,"ondisk_log_size":8041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32903168,"num_objects":8033,"num_object_clones":0,"num_object_copies":16066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4950,"num_write_kb":19800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3894},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7894","reported_seq":8025,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.663301+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:10.663301+0000","last_peered":"2024-09-15T21:23:10.663301+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:10.663301+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7894,"log_dups_size":0,"ondisk_log_size":7894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32317440,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4378,"num_write_kb":17512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3808},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8042","reported_seq":8176,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.582316+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:10.582316+0000","last_peered":"2024-09-15T21:23:10.582316+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:10.582316+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8042,"log_dups_size":0,"ondisk_log_size":8042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32891846,"num_objects":8032,"num_object_clones":0,"num_object_copies":16064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8032,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4746,"num_write_kb":18984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3885},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525451998,"num_objects":128293,"num_object_clones":0,"num_object_copies":256586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1063,"num_objects_unfound":0,"num_objects_dirty":128293,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":88158,"num_write_kb":352632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3102,"num_bytes_recovered":12697600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1066926080,"data_stored":1066777947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128365,"ondisk_log_size":128365,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631476,"kb_used_data":387364,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93740364,"statfs":{"total":96636764160,"available":95990132736,"internally_reserved":0,"allocated":396660736,"data_stored":396398722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":249949865},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730872,"kb_used_data":450728,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93640968,"statfs":{"total":96636764160,"available":95888351232,"internally_reserved":0,"allocated":461545472,"data_stored":461279381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286846635},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,8,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374048,"kb_used_data":202128,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93997792,"statfs":{"total":96636764160,"available":96253739008,"internally_reserved":0,"allocated":206979072,"data_stored":206673238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":176025263},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,7,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462557184,"data_stored":462521054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207855616,"data_stored":207779743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396513280,"data_stored":396477150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:12.952 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:12.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8239", 'reported_seq': 8310, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:10.638110+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:23:10.638110+0000', 'last_peered': '2024-09-15T21:23:10.638110+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:23:10.638110+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 8239, 'log_dups_size': 0, 'ondisk_log_size': 8239, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33746944, 'num_objects': 8239, 'num_object_clones': 0, 'num_object_copies': 16478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8239, 'num_write_kb': 32956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4107}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:12.952 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:23:12.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8181", 'reported_seq': 8302, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:08.328712+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:23:08.328712+0000', 'last_peered': '2024-09-15T21:23:08.328712+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:23:08.328712+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 8181, 'log_dups_size': 0, 'ondisk_log_size': 8181, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33492992, 'num_objects': 8177, 'num_object_clones': 0, 'num_object_copies': 16354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6281, 'num_write_kb': 25124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 3910}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:12.952 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:12.952 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8032", 'reported_seq': 8162, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:10.698493+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:23:10.698493+0000', 'last_peered': '2024-09-15T21:23:10.698493+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:23:10.698493+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 8032, 'log_dups_size': 0, 'ondisk_log_size': 8032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32879075, 'num_objects': 8028, 'num_object_clones': 0, 'num_object_copies': 16056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8028, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4708, 'num_write_kb': 18832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3952}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:12.953 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:12.953 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8059", 'reported_seq': 8190, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:10.677680+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:23:10.677680+0000', 'last_peered': '2024-09-15T21:23:10.677680+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:23:10.677680+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 8059, 'log_dups_size': 0, 'ondisk_log_size': 8059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32986054, 'num_objects': 8055, 'num_object_clones': 0, 'num_object_copies': 16110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8055, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4370, 'num_write_kb': 17480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3894}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:12.953 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:12.953 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'7960", 'reported_seq': 8082, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:08.256984+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:23:08.256984+0000', 'last_peered': '2024-09-15T21:23:08.256984+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:23:08.256984+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 7960, 'log_dups_size': 0, 'ondisk_log_size': 7960, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32575971, 'num_objects': 7954, 'num_object_clones': 0, 'num_object_copies': 15908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7954, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5892, 'num_write_kb': 23568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 3867}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:12.953 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:12.953 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'7899", 'reported_seq': 8021, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:10.655220+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:23:10.655220+0000', 'last_peered': '2024-09-15T21:23:10.655220+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:23:10.655220+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 7899, 'log_dups_size': 0, 'ondisk_log_size': 7899, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32321536, 'num_objects': 7891, 'num_object_clones': 0, 'num_object_copies': 15782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7891, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4572, 'num_write_kb': 18288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3871}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:12.953 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:12.953 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'7980", 'reported_seq': 8094, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:08.272042+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:23:08.272042+0000', 'last_peered': '2024-09-15T21:23:08.272042+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:23:08.272042+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 7980, 'log_dups_size': 0, 'ondisk_log_size': 7980, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32666083, 'num_objects': 7976, 'num_object_clones': 0, 'num_object_copies': 15952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7976, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5839, 'num_write_kb': 23356, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 3886}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:12.953 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:12.953 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8018", 'reported_seq': 14339, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:23:08.375342+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:23:08.375342+0000', 'last_peered': '2024-09-15T21:23:08.375342+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:23:08.375342+0000', 'last_undegraded': '2024-09-15T21:23:08.375342+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 8018, 'log_dups_size': 0, 'ondisk_log_size': 8018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32817152, 'num_objects': 8012, 'num_object_clones': 0, 'num_object_copies': 16024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1063, 'num_objects_unfound': 0, 'num_objects_dirty': 8012, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5711, 'num_write_kb': 22844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3101, 'num_bytes_recovered': 12693504, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1063}, {'shards': '2', 'objects': 6949}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:12.954 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:12.954 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8042", 'reported_seq': 8168, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:10.688428+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:23:10.688428+0000', 'last_peered': '2024-09-15T21:23:10.688428+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:23:10.688428+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 8042, 'log_dups_size': 0, 'ondisk_log_size': 8042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32931840, 'num_objects': 8040, 'num_object_clones': 0, 'num_object_copies': 16080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8040, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4986, 'num_write_kb': 19944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3946}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:12.954 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:12.954 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8041", 'reported_seq': 8169, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:10.660655+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:23:10.660655+0000', 'last_peered': '2024-09-15T21:23:10.660655+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:23:10.660655+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 8041, 'log_dups_size': 0, 'ondisk_log_size': 8041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32903168, 'num_objects': 8033, 'num_object_clones': 0, 'num_object_copies': 16066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8033, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4950, 'num_write_kb': 19800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3894}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:12.954 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:12.954 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'7894", 'reported_seq': 8025, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:10.663301+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:23:10.663301+0000', 'last_peered': '2024-09-15T21:23:10.663301+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:23:10.663301+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 7894, 'log_dups_size': 0, 'ondisk_log_size': 7894, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32317440, 'num_objects': 7890, 'num_object_clones': 0, 'num_object_copies': 15780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7890, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4378, 'num_write_kb': 17512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3808}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:12.954 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:23:12.954 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8042", 'reported_seq': 8176, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:10.582316+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:23:10.582316+0000', 'last_peered': '2024-09-15T21:23:10.582316+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:23:10.582316+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 8042, 'log_dups_size': 0, 'ondisk_log_size': 8042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32891846, 'num_objects': 8032, 'num_object_clones': 0, 'num_object_copies': 16064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8032, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4746, 'num_write_kb': 18984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3885}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:12.955 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:13.674 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:13.674 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/256590 objects degraded (17.658%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":245,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":128295,"data_bytes":525452017,"bytes_used":1778069504,"bytes_avail":288132222976,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":256590,"degraded_ratio":0.17658131649713554,"misplaced_objects":1063,"misplaced_total":256590,"misplaced_ratio":0.0041427959000740477,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":114986,"recovering_keys_per_sec":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"write_bytes_sec":1673619,"read_op_per_sec":0,"write_op_per_sec":408},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:13.715 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:13.715 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:13.755 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 162 20 130154 130134 3.13738 1.76172 0.0690142 0.0248952 2024-09-15T21:23:14.252 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:14.253 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:14.276 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-15T21:23:12.732911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":525452017,"num_objects":128295,"num_object_clones":0,"num_object_copies":256590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1063,"num_objects_unfound":0,"num_objects_dirty":128295,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":88160,"num_write_kb":352634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3103,"num_bytes_recovered":12697619,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128368,"ondisk_log_size":128368,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1736396,"kb_used_data":1040220,"kb_used_omap":12,"kb_used_meta":696115,"kb_avail":281379124,"statfs":{"total":289910292480,"available":288132222976,"internally_reserved":0,"allocated":1065185280,"data_stored":1064351341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":712821763},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,9,17,10,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20090880,"num_objects":4905,"num_object_clones":0,"num_object_copies":9810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-337,"num_objects_unfound":0,"num_objects_dirty":4905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4905,"num_write_kb":19620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004448"},"pg_stats":[{"pgid":"2.f","version":"43'8239","reported_seq":8310,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.638110+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:10.638110+0000","last_peered":"2024-09-15T21:23:10.638110+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:10.638110+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8239,"log_dups_size":0,"ondisk_log_size":8239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33746944,"num_objects":8239,"num_object_clones":0,"num_object_copies":16478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8239,"num_write_kb":32956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4107},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7992","reported_seq":8093,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:08.368806+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:08.368806+0000","last_peered":"2024-09-15T21:23:08.368806+0000","last_clean":"2024-09-15T21:23:08.368806+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:08.368806+0000","last_undegraded":"2024-09-15T21:23:08.368806+0000","last_fullsized":"2024-09-15T21:23:08.368806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":7992,"log_dups_size":0,"ondisk_log_size":7992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32711622,"num_objects":7988,"num_object_clones":0,"num_object_copies":15976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7988,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6273,"num_write_kb":25092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8181","reported_seq":8302,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.328712+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:08.328712+0000","last_peered":"2024-09-15T21:23:08.328712+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:08.328712+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8181,"log_dups_size":0,"ondisk_log_size":8181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33492992,"num_objects":8177,"num_object_clones":0,"num_object_copies":16354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6281,"num_write_kb":25124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":3910}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7817","reported_seq":7883,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:08.339055+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:08.339055+0000","last_peered":"2024-09-15T21:23:08.339055+0000","last_clean":"2024-09-15T21:23:08.339055+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:08.339055+0000","last_undegraded":"2024-09-15T21:23:08.339055+0000","last_fullsized":"2024-09-15T21:23:08.339055+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7817,"log_dups_size":0,"ondisk_log_size":7817,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32018432,"num_objects":7817,"num_object_clones":0,"num_object_copies":15634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7817,"num_write_kb":31268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7992","reported_seq":8120,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:10.587953+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:10.587953+0000","last_peered":"2024-09-15T21:23:10.587953+0000","last_clean":"2024-09-15T21:23:10.587953+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:10.587953+0000","last_undegraded":"2024-09-15T21:23:10.587953+0000","last_fullsized":"2024-09-15T21:23:10.587953+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":7992,"log_dups_size":0,"ondisk_log_size":7992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32707043,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4447,"num_write_kb":17788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8032","reported_seq":8162,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.698493+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:10.698493+0000","last_peered":"2024-09-15T21:23:10.698493+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:10.698493+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8032,"log_dups_size":0,"ondisk_log_size":8032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32879075,"num_objects":8028,"num_object_clones":0,"num_object_copies":16056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8028,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4708,"num_write_kb":18832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3952},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8059","reported_seq":8190,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.677680+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:10.677680+0000","last_peered":"2024-09-15T21:23:10.677680+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:10.677680+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8059,"log_dups_size":0,"ondisk_log_size":8059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32986054,"num_objects":8055,"num_object_clones":0,"num_object_copies":16110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4370,"num_write_kb":17480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3894},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8177","reported_seq":8301,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:10.609353+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:10.609353+0000","last_peered":"2024-09-15T21:23:10.609353+0000","last_clean":"2024-09-15T21:23:10.609353+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:10.609353+0000","last_undegraded":"2024-09-15T21:23:10.609353+0000","last_fullsized":"2024-09-15T21:23:10.609353+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8177,"log_dups_size":0,"ondisk_log_size":8177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33484800,"num_objects":8175,"num_object_clones":0,"num_object_copies":16350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4949,"num_write_kb":19796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7960","reported_seq":8082,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.256984+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:08.256984+0000","last_peered":"2024-09-15T21:23:08.256984+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:08.256984+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":7960,"log_dups_size":0,"ondisk_log_size":7960,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32575971,"num_objects":7954,"num_object_clones":0,"num_object_copies":15908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5892,"num_write_kb":23568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":3867}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7899","reported_seq":8021,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.655220+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:10.655220+0000","last_peered":"2024-09-15T21:23:10.655220+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:10.655220+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":7899,"log_dups_size":0,"ondisk_log_size":7899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32321536,"num_objects":7891,"num_object_clones":0,"num_object_copies":15782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4572,"num_write_kb":18288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3871},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7980","reported_seq":8094,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:08.272042+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:08.272042+0000","last_peered":"2024-09-15T21:23:08.272042+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:08.272042+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":7980,"log_dups_size":0,"ondisk_log_size":7980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32666083,"num_objects":7976,"num_object_clones":0,"num_object_copies":15952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7976,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5839,"num_write_kb":23356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":3886}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.417778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.417778+0000","last_peered":"2024-09-15T21:21:14.417778+0000","last_clean":"2024-09-15T21:21:14.417778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.417778+0000","last_undegraded":"2024-09-15T21:21:14.417778+0000","last_fullsized":"2024-09-15T21:21:14.417778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8018","reported_seq":14339,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:08.375342+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:08.375342+0000","last_peered":"2024-09-15T21:23:08.375342+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:08.375342+0000","last_undegraded":"2024-09-15T21:23:08.375342+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8018,"log_dups_size":0,"ondisk_log_size":8018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32817152,"num_objects":8012,"num_object_clones":0,"num_object_copies":16024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1063,"num_objects_unfound":0,"num_objects_dirty":8012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5711,"num_write_kb":22844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3101,"num_bytes_recovered":12693504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1063},{"shards":"2","objects":6949}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8042","reported_seq":8168,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.688428+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:10.688428+0000","last_peered":"2024-09-15T21:23:10.688428+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:10.688428+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8042,"log_dups_size":0,"ondisk_log_size":8042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32931840,"num_objects":8040,"num_object_clones":0,"num_object_copies":16080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4986,"num_write_kb":19944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3946},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.526739+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.526739+0000","last_peered":"2024-09-15T21:21:14.526739+0000","last_clean":"2024-09-15T21:21:14.526739+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.526739+0000","last_undegraded":"2024-09-15T21:21:14.526739+0000","last_fullsized":"2024-09-15T21:21:14.526739+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8041","reported_seq":8169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.660655+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:10.660655+0000","last_peered":"2024-09-15T21:23:10.660655+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:10.660655+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8041,"log_dups_size":0,"ondisk_log_size":8041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32903168,"num_objects":8033,"num_object_clones":0,"num_object_copies":16066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4950,"num_write_kb":19800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3894},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7894","reported_seq":8025,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.663301+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:10.663301+0000","last_peered":"2024-09-15T21:23:10.663301+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:10.663301+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":7894,"log_dups_size":0,"ondisk_log_size":7894,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32317440,"num_objects":7890,"num_object_clones":0,"num_object_copies":15780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4378,"num_write_kb":17512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3808},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.546691+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.546691+0000","last_peered":"2024-09-15T21:21:14.546691+0000","last_clean":"2024-09-15T21:21:14.546691+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.546691+0000","last_undegraded":"2024-09-15T21:21:14.546691+0000","last_fullsized":"2024-09-15T21:21:14.546691+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8042","reported_seq":8176,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:10.582316+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:10.582316+0000","last_peered":"2024-09-15T21:23:10.582316+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:10.582316+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8042,"log_dups_size":0,"ondisk_log_size":8042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32891846,"num_objects":8032,"num_object_clones":0,"num_object_copies":16064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8032,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4746,"num_write_kb":18984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3885},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":525451998,"num_objects":128293,"num_object_clones":0,"num_object_copies":256586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":1063,"num_objects_unfound":0,"num_objects_dirty":128293,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":88158,"num_write_kb":352632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3102,"num_bytes_recovered":12697600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1066926080,"data_stored":1066777947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128365,"ondisk_log_size":128365,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631476,"kb_used_data":387364,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93740364,"statfs":{"total":96636764160,"available":95990132736,"internally_reserved":0,"allocated":396660736,"data_stored":396398722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":249949865},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730872,"kb_used_data":450728,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93640968,"statfs":{"total":96636764160,"available":95888351232,"internally_reserved":0,"allocated":461545472,"data_stored":461279381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286846635},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,8,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":374048,"kb_used_data":202128,"kb_used_omap":4,"kb_used_meta":171899,"kb_avail":93997792,"statfs":{"total":96636764160,"available":96253739008,"internally_reserved":0,"allocated":206979072,"data_stored":206673238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":176025263},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,7,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":462557184,"data_stored":462521054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":207855616,"data_stored":207779743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396513280,"data_stored":396477150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:14.756 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 163 20 130630 130610 3.12953 1.85938 0.0301807 0.0249556 2024-09-15T21:23:15.756 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 164 20 131139 131119 3.12258 1.98828 0.0424725 0.025011 2024-09-15T21:23:16.756 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 165 20 131653 131633 3.11582 2.00781 0.0358259 0.0250664 2024-09-15T21:23:17.279 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:17.756 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 166 20 132108 132088 3.10776 1.77734 0.0733733 0.0251306 2024-09-15T21:23:17.819 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:17.821 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:17.843 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-15T21:23:16.734082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":535306993,"num_objects":130701,"num_object_clones":0,"num_object_copies":261402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":879,"num_objects_unfound":0,"num_objects_dirty":130701,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":90566,"num_write_kb":362258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3287,"num_bytes_recovered":13451283,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130774,"ondisk_log_size":130774,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1786448,"kb_used_data":1059740,"kb_used_omap":12,"kb_used_meta":726643,"kb_avail":281329072,"statfs":{"total":289910292480,"available":288080969728,"internally_reserved":0,"allocated":1085173760,"data_stored":1084339821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":744082435},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,11,9,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26136576,"num_objects":6381,"num_object_clones":0,"num_object_copies":12762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-350,"num_objects_unfound":0,"num_objects_dirty":6381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6381,"num_write_kb":25524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":1433600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004025"},"pg_stats":[{"pgid":"2.f","version":"43'8371","reported_seq":8442,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.639368+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:15.639368+0000","last_peered":"2024-09-15T21:23:15.639368+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:15.639368+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8371,"log_dups_size":0,"ondisk_log_size":8371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34287616,"num_objects":8371,"num_object_clones":0,"num_object_copies":16742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8371,"num_write_kb":33484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4239},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8144","reported_seq":8245,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.395271+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:13.395271+0000","last_peered":"2024-09-15T21:23:13.395271+0000","last_clean":"2024-09-15T21:23:13.395271+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:13.395271+0000","last_undegraded":"2024-09-15T21:23:13.395271+0000","last_fullsized":"2024-09-15T21:23:13.395271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8144,"log_dups_size":0,"ondisk_log_size":8144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33334214,"num_objects":8140,"num_object_clones":0,"num_object_copies":16280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8140,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6425,"num_write_kb":25700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8339","reported_seq":8460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.297482+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:13.297482+0000","last_peered":"2024-09-15T21:23:13.297482+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:13.297482+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8339,"log_dups_size":0,"ondisk_log_size":8339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34140160,"num_objects":8335,"num_object_clones":0,"num_object_copies":16670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6439,"num_write_kb":25756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4068}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7959","reported_seq":8025,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.251865+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:13.251865+0000","last_peered":"2024-09-15T21:23:13.251865+0000","last_clean":"2024-09-15T21:23:13.251865+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:13.251865+0000","last_undegraded":"2024-09-15T21:23:13.251865+0000","last_fullsized":"2024-09-15T21:23:13.251865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7959,"log_dups_size":0,"ondisk_log_size":7959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32600064,"num_objects":7959,"num_object_clones":0,"num_object_copies":15918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7959,"num_write_kb":31836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8133","reported_seq":8261,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:15.547480+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:15.547480+0000","last_peered":"2024-09-15T21:23:15.547480+0000","last_clean":"2024-09-15T21:23:15.547480+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:15.547480+0000","last_undegraded":"2024-09-15T21:23:15.547480+0000","last_fullsized":"2024-09-15T21:23:15.547480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8133,"log_dups_size":0,"ondisk_log_size":8133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33284579,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4588,"num_write_kb":18352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8172","reported_seq":8304,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.616430+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:15.616430+0000","last_peered":"2024-09-15T21:23:15.616430+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:15.616430+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8172,"log_dups_size":0,"ondisk_log_size":8172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33452515,"num_objects":8168,"num_object_clones":0,"num_object_copies":16336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8168,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4848,"num_write_kb":19392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4092},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8224","reported_seq":8351,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.678991+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:15.678991+0000","last_peered":"2024-09-15T21:23:15.678991+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:15.678991+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8224,"log_dups_size":0,"ondisk_log_size":8224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33661894,"num_objects":8220,"num_object_clones":0,"num_object_copies":16440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4535,"num_write_kb":18140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4059},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8336","reported_seq":8460,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:15.690482+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:15.690482+0000","last_peered":"2024-09-15T21:23:15.690482+0000","last_clean":"2024-09-15T21:23:15.690482+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:15.690482+0000","last_undegraded":"2024-09-15T21:23:15.690482+0000","last_fullsized":"2024-09-15T21:23:15.690482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8336,"log_dups_size":0,"ondisk_log_size":8336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34136064,"num_objects":8334,"num_object_clones":0,"num_object_copies":16668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5108,"num_write_kb":20432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8101","reported_seq":8223,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.308445+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:13.308445+0000","last_peered":"2024-09-15T21:23:13.308445+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:13.308445+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8101,"log_dups_size":0,"ondisk_log_size":8101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33153507,"num_objects":8095,"num_object_clones":0,"num_object_copies":16190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8095,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6033,"num_write_kb":24132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4008}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8066","reported_seq":8192,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.577273+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:15.577273+0000","last_peered":"2024-09-15T21:23:15.577273+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:15.577273+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8066,"log_dups_size":0,"ondisk_log_size":8066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33005568,"num_objects":8058,"num_object_clones":0,"num_object_copies":16116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4739,"num_write_kb":18956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4038},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8137","reported_seq":8251,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.377567+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:13.377567+0000","last_peered":"2024-09-15T21:23:13.377567+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:13.377567+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8137,"log_dups_size":0,"ondisk_log_size":8137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33309155,"num_objects":8133,"num_object_clones":0,"num_object_copies":16266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8133,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5996,"num_write_kb":23984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4043}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.839588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.839588+0000","last_peered":"2024-09-15T21:21:14.839588+0000","last_clean":"2024-09-15T21:21:14.839588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.839588+0000","last_undegraded":"2024-09-15T21:21:14.839588+0000","last_fullsized":"2024-09-15T21:21:14.839588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8172","reported_seq":14860,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:13.408949+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:13.408949+0000","last_peered":"2024-09-15T21:23:13.408949+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:13.408949+0000","last_undegraded":"2024-09-15T21:23:13.408949+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8172,"log_dups_size":0,"ondisk_log_size":8172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33447936,"num_objects":8166,"num_object_clones":0,"num_object_copies":16332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":879,"num_objects_unfound":0,"num_objects_dirty":8166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5865,"num_write_kb":23460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3285,"num_bytes_recovered":13447168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":879},{"shards":"2","objects":7287}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8187","reported_seq":8316,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.692395+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:15.692395+0000","last_peered":"2024-09-15T21:23:15.692395+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:15.692395+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8187,"log_dups_size":0,"ondisk_log_size":8187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33525760,"num_objects":8185,"num_object_clones":0,"num_object_copies":16370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5131,"num_write_kb":20524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4091},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.537989+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.537989+0000","last_peered":"2024-09-15T21:21:14.537989+0000","last_clean":"2024-09-15T21:21:14.537989+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.537989+0000","last_undegraded":"2024-09-15T21:21:14.537989+0000","last_fullsized":"2024-09-15T21:21:14.537989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8164","reported_seq":8292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.703564+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:15.703564+0000","last_peered":"2024-09-15T21:23:15.703564+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:15.703564+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8164,"log_dups_size":0,"ondisk_log_size":8164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33406976,"num_objects":8156,"num_object_clones":0,"num_object_copies":16312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5073,"num_write_kb":20292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4017},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8053","reported_seq":8183,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.687660+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:15.687660+0000","last_peered":"2024-09-15T21:23:15.687660+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:15.687660+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8053,"log_dups_size":0,"ondisk_log_size":8053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32968704,"num_objects":8049,"num_object_clones":0,"num_object_copies":16098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4537,"num_write_kb":18148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3967},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.549401+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.549401+0000","last_peered":"2024-09-15T21:21:14.549401+0000","last_clean":"2024-09-15T21:21:14.549401+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.549401+0000","last_undegraded":"2024-09-15T21:21:14.549401+0000","last_fullsized":"2024-09-15T21:21:14.549401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8213","reported_seq":8347,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.720407+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:15.720407+0000","last_peered":"2024-09-15T21:23:15.720407+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:15.720407+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8213,"log_dups_size":0,"ondisk_log_size":8213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33592262,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4917,"num_write_kb":19668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4056},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":535306974,"num_objects":130699,"num_object_clones":0,"num_object_copies":261398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":879,"num_objects_unfound":0,"num_objects_dirty":130699,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":90564,"num_write_kb":362256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3286,"num_bytes_recovered":13451264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1087410176,"data_stored":1087262043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130771,"ondisk_log_size":130771,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640328,"kb_used_data":392056,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93731512,"statfs":{"total":96636764160,"available":95981068288,"internally_reserved":0,"allocated":401465344,"data_stored":401203330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":254209705},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":742080,"kb_used_data":457776,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93629760,"statfs":{"total":96636764160,"available":95876874240,"internally_reserved":0,"allocated":468762624,"data_stored":468496533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291106475},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404040,"kb_used_data":209908,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93967800,"statfs":{"total":96636764160,"available":96223027200,"internally_reserved":0,"allocated":214945792,"data_stored":214639958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":198766255},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,6,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":469954560,"data_stored":469918430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":216043520,"data_stored":215967647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401412096,"data_stored":401375966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:17.844 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:18.388 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:18.389 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:18.410 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-15T21:23:16.734082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":535306993,"num_objects":130701,"num_object_clones":0,"num_object_copies":261402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":879,"num_objects_unfound":0,"num_objects_dirty":130701,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":90566,"num_write_kb":362258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3287,"num_bytes_recovered":13451283,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130774,"ondisk_log_size":130774,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1786448,"kb_used_data":1059740,"kb_used_omap":12,"kb_used_meta":726643,"kb_avail":281329072,"statfs":{"total":289910292480,"available":288080969728,"internally_reserved":0,"allocated":1085173760,"data_stored":1084339821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":744082435},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,11,9,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26136576,"num_objects":6381,"num_object_clones":0,"num_object_copies":12762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-350,"num_objects_unfound":0,"num_objects_dirty":6381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6381,"num_write_kb":25524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":1433600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004025"},"pg_stats":[{"pgid":"2.f","version":"43'8371","reported_seq":8442,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.639368+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:15.639368+0000","last_peered":"2024-09-15T21:23:15.639368+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:15.639368+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8371,"log_dups_size":0,"ondisk_log_size":8371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34287616,"num_objects":8371,"num_object_clones":0,"num_object_copies":16742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8371,"num_write_kb":33484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4239},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8144","reported_seq":8245,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.395271+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:13.395271+0000","last_peered":"2024-09-15T21:23:13.395271+0000","last_clean":"2024-09-15T21:23:13.395271+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:13.395271+0000","last_undegraded":"2024-09-15T21:23:13.395271+0000","last_fullsized":"2024-09-15T21:23:13.395271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8144,"log_dups_size":0,"ondisk_log_size":8144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33334214,"num_objects":8140,"num_object_clones":0,"num_object_copies":16280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8140,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6425,"num_write_kb":25700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8339","reported_seq":8460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.297482+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:13.297482+0000","last_peered":"2024-09-15T21:23:13.297482+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:13.297482+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8339,"log_dups_size":0,"ondisk_log_size":8339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34140160,"num_objects":8335,"num_object_clones":0,"num_object_copies":16670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6439,"num_write_kb":25756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4068}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7959","reported_seq":8025,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:13.251865+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:13.251865+0000","last_peered":"2024-09-15T21:23:13.251865+0000","last_clean":"2024-09-15T21:23:13.251865+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:13.251865+0000","last_undegraded":"2024-09-15T21:23:13.251865+0000","last_fullsized":"2024-09-15T21:23:13.251865+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":7959,"log_dups_size":0,"ondisk_log_size":7959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32600064,"num_objects":7959,"num_object_clones":0,"num_object_copies":15918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7959,"num_write_kb":31836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8133","reported_seq":8261,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:15.547480+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:15.547480+0000","last_peered":"2024-09-15T21:23:15.547480+0000","last_clean":"2024-09-15T21:23:15.547480+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:15.547480+0000","last_undegraded":"2024-09-15T21:23:15.547480+0000","last_fullsized":"2024-09-15T21:23:15.547480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8133,"log_dups_size":0,"ondisk_log_size":8133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33284579,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4588,"num_write_kb":18352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8172","reported_seq":8304,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.616430+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:15.616430+0000","last_peered":"2024-09-15T21:23:15.616430+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:15.616430+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8172,"log_dups_size":0,"ondisk_log_size":8172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33452515,"num_objects":8168,"num_object_clones":0,"num_object_copies":16336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8168,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4848,"num_write_kb":19392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4092},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8224","reported_seq":8351,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.678991+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:15.678991+0000","last_peered":"2024-09-15T21:23:15.678991+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:15.678991+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8224,"log_dups_size":0,"ondisk_log_size":8224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33661894,"num_objects":8220,"num_object_clones":0,"num_object_copies":16440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4535,"num_write_kb":18140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4059},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8336","reported_seq":8460,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:15.690482+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:15.690482+0000","last_peered":"2024-09-15T21:23:15.690482+0000","last_clean":"2024-09-15T21:23:15.690482+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:15.690482+0000","last_undegraded":"2024-09-15T21:23:15.690482+0000","last_fullsized":"2024-09-15T21:23:15.690482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8336,"log_dups_size":0,"ondisk_log_size":8336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34136064,"num_objects":8334,"num_object_clones":0,"num_object_copies":16668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5108,"num_write_kb":20432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8101","reported_seq":8223,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.308445+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:13.308445+0000","last_peered":"2024-09-15T21:23:13.308445+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:13.308445+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8101,"log_dups_size":0,"ondisk_log_size":8101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33153507,"num_objects":8095,"num_object_clones":0,"num_object_copies":16190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8095,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6033,"num_write_kb":24132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4008}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8066","reported_seq":8192,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.577273+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:15.577273+0000","last_peered":"2024-09-15T21:23:15.577273+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:15.577273+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8066,"log_dups_size":0,"ondisk_log_size":8066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33005568,"num_objects":8058,"num_object_clones":0,"num_object_copies":16116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4739,"num_write_kb":18956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4038},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8137","reported_seq":8251,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:13.377567+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:13.377567+0000","last_peered":"2024-09-15T21:23:13.377567+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:13.377567+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8137,"log_dups_size":0,"ondisk_log_size":8137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33309155,"num_objects":8133,"num_object_clones":0,"num_object_copies":16266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8133,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5996,"num_write_kb":23984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4043}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.839588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.839588+0000","last_peered":"2024-09-15T21:21:14.839588+0000","last_clean":"2024-09-15T21:21:14.839588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.839588+0000","last_undegraded":"2024-09-15T21:21:14.839588+0000","last_fullsized":"2024-09-15T21:21:14.839588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8172","reported_seq":14860,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:13.408949+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:13.408949+0000","last_peered":"2024-09-15T21:23:13.408949+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:13.408949+0000","last_undegraded":"2024-09-15T21:23:13.408949+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8172,"log_dups_size":0,"ondisk_log_size":8172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33447936,"num_objects":8166,"num_object_clones":0,"num_object_copies":16332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":879,"num_objects_unfound":0,"num_objects_dirty":8166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5865,"num_write_kb":23460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3285,"num_bytes_recovered":13447168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":879},{"shards":"2","objects":7287}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8187","reported_seq":8316,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.692395+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:15.692395+0000","last_peered":"2024-09-15T21:23:15.692395+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:15.692395+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8187,"log_dups_size":0,"ondisk_log_size":8187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33525760,"num_objects":8185,"num_object_clones":0,"num_object_copies":16370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5131,"num_write_kb":20524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4091},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.537989+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.537989+0000","last_peered":"2024-09-15T21:21:14.537989+0000","last_clean":"2024-09-15T21:21:14.537989+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.537989+0000","last_undegraded":"2024-09-15T21:21:14.537989+0000","last_fullsized":"2024-09-15T21:21:14.537989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8164","reported_seq":8292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.703564+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:15.703564+0000","last_peered":"2024-09-15T21:23:15.703564+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:15.703564+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8164,"log_dups_size":0,"ondisk_log_size":8164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33406976,"num_objects":8156,"num_object_clones":0,"num_object_copies":16312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5073,"num_write_kb":20292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4017},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8053","reported_seq":8183,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.687660+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:15.687660+0000","last_peered":"2024-09-15T21:23:15.687660+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:15.687660+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8053,"log_dups_size":0,"ondisk_log_size":8053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32968704,"num_objects":8049,"num_object_clones":0,"num_object_copies":16098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4537,"num_write_kb":18148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3967},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.549401+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.549401+0000","last_peered":"2024-09-15T21:21:14.549401+0000","last_clean":"2024-09-15T21:21:14.549401+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.549401+0000","last_undegraded":"2024-09-15T21:21:14.549401+0000","last_fullsized":"2024-09-15T21:21:14.549401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8213","reported_seq":8347,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.720407+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:15.720407+0000","last_peered":"2024-09-15T21:23:15.720407+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:15.720407+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8213,"log_dups_size":0,"ondisk_log_size":8213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33592262,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4917,"num_write_kb":19668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4056},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":535306974,"num_objects":130699,"num_object_clones":0,"num_object_copies":261398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":879,"num_objects_unfound":0,"num_objects_dirty":130699,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":90564,"num_write_kb":362256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3286,"num_bytes_recovered":13451264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1087410176,"data_stored":1087262043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130771,"ondisk_log_size":130771,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640328,"kb_used_data":392056,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93731512,"statfs":{"total":96636764160,"available":95981068288,"internally_reserved":0,"allocated":401465344,"data_stored":401203330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":254209705},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":742080,"kb_used_data":457776,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93629760,"statfs":{"total":96636764160,"available":95876874240,"internally_reserved":0,"allocated":468762624,"data_stored":468496533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291106475},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404040,"kb_used_data":209908,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93967800,"statfs":{"total":96636764160,"available":96223027200,"internally_reserved":0,"allocated":214945792,"data_stored":214639958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":198766255},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,6,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":469954560,"data_stored":469918430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":216043520,"data_stored":215967647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401412096,"data_stored":401375966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:18.411 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:18.411 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8371", 'reported_seq': 8442, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:15.639368+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:23:15.639368+0000', 'last_peered': '2024-09-15T21:23:15.639368+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:23:15.639368+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 8371, 'log_dups_size': 0, 'ondisk_log_size': 8371, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34287616, 'num_objects': 8371, 'num_object_clones': 0, 'num_object_copies': 16742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8371, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8371, 'num_write_kb': 33484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4239}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:18.412 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:23:18.412 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8339", 'reported_seq': 8460, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:13.297482+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:23:13.297482+0000', 'last_peered': '2024-09-15T21:23:13.297482+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:23:13.297482+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 8339, 'log_dups_size': 0, 'ondisk_log_size': 8339, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34140160, 'num_objects': 8335, 'num_object_clones': 0, 'num_object_copies': 16670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6439, 'num_write_kb': 25756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 4068}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:18.412 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:18.412 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8172", 'reported_seq': 8304, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:15.616430+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:23:15.616430+0000', 'last_peered': '2024-09-15T21:23:15.616430+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:23:15.616430+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 8172, 'log_dups_size': 0, 'ondisk_log_size': 8172, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33452515, 'num_objects': 8168, 'num_object_clones': 0, 'num_object_copies': 16336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8168, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4848, 'num_write_kb': 19392, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4092}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:18.412 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:18.412 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8224", 'reported_seq': 8351, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:15.678991+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:23:15.678991+0000', 'last_peered': '2024-09-15T21:23:15.678991+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:23:15.678991+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 8224, 'log_dups_size': 0, 'ondisk_log_size': 8224, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33661894, 'num_objects': 8220, 'num_object_clones': 0, 'num_object_copies': 16440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4535, 'num_write_kb': 18140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4059}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:18.412 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:18.412 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'8101", 'reported_seq': 8223, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:13.308445+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:23:13.308445+0000', 'last_peered': '2024-09-15T21:23:13.308445+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:23:13.308445+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 8101, 'log_dups_size': 0, 'ondisk_log_size': 8101, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33153507, 'num_objects': 8095, 'num_object_clones': 0, 'num_object_copies': 16190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8095, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6033, 'num_write_kb': 24132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 4008}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:18.413 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:18.413 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8066", 'reported_seq': 8192, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:15.577273+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:23:15.577273+0000', 'last_peered': '2024-09-15T21:23:15.577273+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:23:15.577273+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 8066, 'log_dups_size': 0, 'ondisk_log_size': 8066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33005568, 'num_objects': 8058, 'num_object_clones': 0, 'num_object_copies': 16116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8058, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4739, 'num_write_kb': 18956, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4038}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:18.413 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:18.413 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8137", 'reported_seq': 8251, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:13.377567+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:23:13.377567+0000', 'last_peered': '2024-09-15T21:23:13.377567+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:23:13.377567+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 8137, 'log_dups_size': 0, 'ondisk_log_size': 8137, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33309155, 'num_objects': 8133, 'num_object_clones': 0, 'num_object_copies': 16266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8133, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5996, 'num_write_kb': 23984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 4043}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:18.413 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:18.413 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8172", 'reported_seq': 14860, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:23:13.408949+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:23:13.408949+0000', 'last_peered': '2024-09-15T21:23:13.408949+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:23:13.408949+0000', 'last_undegraded': '2024-09-15T21:23:13.408949+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 8172, 'log_dups_size': 0, 'ondisk_log_size': 8172, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33447936, 'num_objects': 8166, 'num_object_clones': 0, 'num_object_copies': 16332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 879, 'num_objects_unfound': 0, 'num_objects_dirty': 8166, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5865, 'num_write_kb': 23460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3285, 'num_bytes_recovered': 13447168, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 879}, {'shards': '2', 'objects': 7287}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:18.413 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:18.413 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8187", 'reported_seq': 8316, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:15.692395+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:23:15.692395+0000', 'last_peered': '2024-09-15T21:23:15.692395+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:23:15.692395+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 8187, 'log_dups_size': 0, 'ondisk_log_size': 8187, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33525760, 'num_objects': 8185, 'num_object_clones': 0, 'num_object_copies': 16370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5131, 'num_write_kb': 20524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4091}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:18.413 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:18.413 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8164", 'reported_seq': 8292, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:15.703564+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:23:15.703564+0000', 'last_peered': '2024-09-15T21:23:15.703564+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:23:15.703564+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 8164, 'log_dups_size': 0, 'ondisk_log_size': 8164, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33406976, 'num_objects': 8156, 'num_object_clones': 0, 'num_object_copies': 16312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5073, 'num_write_kb': 20292, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4017}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:18.414 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:18.414 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8053", 'reported_seq': 8183, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:15.687660+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:23:15.687660+0000', 'last_peered': '2024-09-15T21:23:15.687660+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:23:15.687660+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 8053, 'log_dups_size': 0, 'ondisk_log_size': 8053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32968704, 'num_objects': 8049, 'num_object_clones': 0, 'num_object_copies': 16098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8049, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4537, 'num_write_kb': 18148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3967}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:18.414 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:23:18.414 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8213", 'reported_seq': 8347, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:15.720407+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:23:15.720407+0000', 'last_peered': '2024-09-15T21:23:15.720407+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:23:15.720407+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 8213, 'log_dups_size': 0, 'ondisk_log_size': 8213, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33592262, 'num_objects': 8203, 'num_object_clones': 0, 'num_object_copies': 16406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8203, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4917, 'num_write_kb': 19668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4056}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:18.415 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:18.756 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 167 20 132545 132525 3.09937 1.70703 0.0340072 0.0251994 2024-09-15T21:23:19.114 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:19.114 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/261402 objects degraded (17.333%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":250,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":130701,"data_bytes":535306993,"bytes_used":1829322752,"bytes_avail":288080969728,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":261402,"degraded_ratio":0.17333073197603693,"misplaced_objects":879,"misplaced_total":261402,"misplaced_ratio":0.0033626368581724702,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":119426,"recovering_keys_per_sec":0,"num_objects_recovered":350,"num_bytes_recovered":1433600,"num_keys_recovered":0,"write_bytes_sec":2177317,"read_op_per_sec":0,"write_op_per_sec":531},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:19.140 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:19.140 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:19.679 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:19.679 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:19.703 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":161,"stamp":"2024-09-15T21:23:18.735018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":538985201,"num_objects":131599,"num_object_clones":0,"num_object_copies":263198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":131599,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":91464,"num_write_kb":365850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3481,"num_bytes_recovered":14245907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131672,"ondisk_log_size":131672,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1795472,"kb_used_data":1064604,"kb_used_omap":12,"kb_used_meta":730803,"kb_avail":281320048,"statfs":{"total":289910292480,"available":288071729152,"internally_reserved":0,"allocated":1090154496,"data_stored":1089320557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":748342275},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,9,12,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23719936,"num_objects":5791,"num_object_clones":0,"num_object_copies":11582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-544,"num_objects_unfound":0,"num_objects_dirty":5791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5791,"num_write_kb":23164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":544,"num_bytes_recovered":2228224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004354"},"pg_stats":[{"pgid":"2.f","version":"43'8371","reported_seq":8442,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.639368+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:15.639368+0000","last_peered":"2024-09-15T21:23:15.639368+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:15.639368+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8371,"log_dups_size":0,"ondisk_log_size":8371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34287616,"num_objects":8371,"num_object_clones":0,"num_object_copies":16742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8371,"num_write_kb":33484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4239},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8272","reported_seq":8373,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.353243+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:18.353243+0000","last_peered":"2024-09-15T21:23:18.353243+0000","last_clean":"2024-09-15T21:23:18.353243+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:18.353243+0000","last_undegraded":"2024-09-15T21:23:18.353243+0000","last_fullsized":"2024-09-15T21:23:18.353243+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8272,"log_dups_size":0,"ondisk_log_size":8272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33858502,"num_objects":8268,"num_object_clones":0,"num_object_copies":16536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8268,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6553,"num_write_kb":26212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8481","reported_seq":8602,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.405539+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:18.405539+0000","last_peered":"2024-09-15T21:23:18.405539+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:18.405539+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8481,"log_dups_size":0,"ondisk_log_size":8481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34721792,"num_objects":8477,"num_object_clones":0,"num_object_copies":16954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6581,"num_write_kb":26324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4210}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8119","reported_seq":8185,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.366213+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:18.366213+0000","last_peered":"2024-09-15T21:23:18.366213+0000","last_clean":"2024-09-15T21:23:18.366213+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:18.366213+0000","last_undegraded":"2024-09-15T21:23:18.366213+0000","last_fullsized":"2024-09-15T21:23:18.366213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8119,"log_dups_size":0,"ondisk_log_size":8119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33255424,"num_objects":8119,"num_object_clones":0,"num_object_copies":16238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8119,"num_write_kb":32476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8133","reported_seq":8261,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:15.547480+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:15.547480+0000","last_peered":"2024-09-15T21:23:15.547480+0000","last_clean":"2024-09-15T21:23:15.547480+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:15.547480+0000","last_undegraded":"2024-09-15T21:23:15.547480+0000","last_fullsized":"2024-09-15T21:23:15.547480+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8133,"log_dups_size":0,"ondisk_log_size":8133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33284579,"num_objects":8127,"num_object_clones":0,"num_object_copies":16254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4588,"num_write_kb":18352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8172","reported_seq":8304,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.616430+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:15.616430+0000","last_peered":"2024-09-15T21:23:15.616430+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:15.616430+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8172,"log_dups_size":0,"ondisk_log_size":8172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33452515,"num_objects":8168,"num_object_clones":0,"num_object_copies":16336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8168,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4848,"num_write_kb":19392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4092},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8224","reported_seq":8351,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.678991+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:15.678991+0000","last_peered":"2024-09-15T21:23:15.678991+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:15.678991+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8224,"log_dups_size":0,"ondisk_log_size":8224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33661894,"num_objects":8220,"num_object_clones":0,"num_object_copies":16440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4535,"num_write_kb":18140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4059},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8336","reported_seq":8460,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:15.690482+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:15.690482+0000","last_peered":"2024-09-15T21:23:15.690482+0000","last_clean":"2024-09-15T21:23:15.690482+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:15.690482+0000","last_undegraded":"2024-09-15T21:23:15.690482+0000","last_fullsized":"2024-09-15T21:23:15.690482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8336,"log_dups_size":0,"ondisk_log_size":8336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34136064,"num_objects":8334,"num_object_clones":0,"num_object_copies":16668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5108,"num_write_kb":20432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8270","reported_seq":8390,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.358136+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:18.358136+0000","last_peered":"2024-09-15T21:23:18.358136+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:18.358136+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8270,"log_dups_size":0,"ondisk_log_size":8270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33845731,"num_objects":8264,"num_object_clones":0,"num_object_copies":16528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8264,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6202,"num_write_kb":24808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4177}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8066","reported_seq":8192,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.577273+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:15.577273+0000","last_peered":"2024-09-15T21:23:15.577273+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:15.577273+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8066,"log_dups_size":0,"ondisk_log_size":8066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33005568,"num_objects":8058,"num_object_clones":0,"num_object_copies":16116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4739,"num_write_kb":18956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4038},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8284","reported_seq":8398,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.345282+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:18.345282+0000","last_peered":"2024-09-15T21:23:18.345282+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:18.345282+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8284,"log_dups_size":0,"ondisk_log_size":8284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33911267,"num_objects":8280,"num_object_clones":0,"num_object_copies":16560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8280,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6143,"num_write_kb":24572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4190}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.839588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.839588+0000","last_peered":"2024-09-15T21:21:14.839588+0000","last_clean":"2024-09-15T21:21:14.839588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.839588+0000","last_undegraded":"2024-09-15T21:21:14.839588+0000","last_fullsized":"2024-09-15T21:21:14.839588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8324","reported_seq":15401,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:18.388956+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:18.388956+0000","last_peered":"2024-09-15T21:23:18.388956+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:18.388956+0000","last_undegraded":"2024-09-15T21:23:18.388956+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8324,"log_dups_size":0,"ondisk_log_size":8324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34070528,"num_objects":8318,"num_object_clones":0,"num_object_copies":16636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":8318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6017,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3479,"num_bytes_recovered":14241792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":685},{"shards":"2","objects":7633}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8187","reported_seq":8316,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.692395+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:15.692395+0000","last_peered":"2024-09-15T21:23:15.692395+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:15.692395+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8187,"log_dups_size":0,"ondisk_log_size":8187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33525760,"num_objects":8185,"num_object_clones":0,"num_object_copies":16370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5131,"num_write_kb":20524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4091},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.537989+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.537989+0000","last_peered":"2024-09-15T21:21:14.537989+0000","last_clean":"2024-09-15T21:21:14.537989+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.537989+0000","last_undegraded":"2024-09-15T21:21:14.537989+0000","last_fullsized":"2024-09-15T21:21:14.537989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8164","reported_seq":8292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.703564+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:15.703564+0000","last_peered":"2024-09-15T21:23:15.703564+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:15.703564+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8164,"log_dups_size":0,"ondisk_log_size":8164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33406976,"num_objects":8156,"num_object_clones":0,"num_object_copies":16312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5073,"num_write_kb":20292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4017},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8053","reported_seq":8183,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.687660+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:15.687660+0000","last_peered":"2024-09-15T21:23:15.687660+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:15.687660+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8053,"log_dups_size":0,"ondisk_log_size":8053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32968704,"num_objects":8049,"num_object_clones":0,"num_object_copies":16098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4537,"num_write_kb":18148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3967},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.549401+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.549401+0000","last_peered":"2024-09-15T21:21:14.549401+0000","last_clean":"2024-09-15T21:21:14.549401+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.549401+0000","last_undegraded":"2024-09-15T21:21:14.549401+0000","last_fullsized":"2024-09-15T21:21:14.549401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.355429+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.355429+0000","last_peered":"2024-09-15T21:21:14.355429+0000","last_clean":"2024-09-15T21:21:14.355429+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.355429+0000","last_undegraded":"2024-09-15T21:21:14.355429+0000","last_fullsized":"2024-09-15T21:21:14.355429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8213","reported_seq":8347,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:15.720407+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:15.720407+0000","last_peered":"2024-09-15T21:23:15.720407+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:15.720407+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8213,"log_dups_size":0,"ondisk_log_size":8213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33592262,"num_objects":8203,"num_object_clones":0,"num_object_copies":16406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8203,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4917,"num_write_kb":19668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4056},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":538985182,"num_objects":131597,"num_object_clones":0,"num_object_copies":263194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":131597,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":91462,"num_write_kb":365848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1092300800,"data_stored":1092152667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131669,"ondisk_log_size":131669,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649352,"kb_used_data":396920,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93722488,"statfs":{"total":96636764160,"available":95971827712,"internally_reserved":0,"allocated":406446080,"data_stored":406184066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":258469545},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":742080,"kb_used_data":457776,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93629760,"statfs":{"total":96636764160,"available":95876874240,"internally_reserved":0,"allocated":468762624,"data_stored":468496533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291106475},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404040,"kb_used_data":209908,"kb_used_omap":4,"kb_used_meta":194107,"kb_avail":93967800,"statfs":{"total":96636764160,"available":96223027200,"internally_reserved":0,"allocated":214945792,"data_stored":214639958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":198766255},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,6,3,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":469954560,"data_stored":469918430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":216043520,"data_stored":215967647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406302720,"data_stored":406266590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:19.756 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 168 20 132978 132958 3.09099 1.69141 0.0449627 0.0252676 2024-09-15T21:23:20.756 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 169 20 133477 133457 3.08423 1.94922 0.0216415 0.0253226 2024-09-15T21:23:21.756 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 170 20 133969 133949 3.07739 1.92188 0.0190798 0.0253768 2024-09-15T21:23:22.705 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:22.760 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 171 20 134475 134455 3.07095 1.97656 0.0717372 0.0254332 2024-09-15T21:23:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:23.276 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:23.307 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-15T21:23:22.735947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":545084145,"num_objects":133088,"num_object_clones":0,"num_object_copies":266176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":133088,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":92953,"num_write_kb":371806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3481,"num_bytes_recovered":14245907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133161,"ondisk_log_size":133161,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1820232,"kb_used_data":1081048,"kb_used_omap":12,"kb_used_meta":739123,"kb_avail":281295288,"statfs":{"total":289910292480,"available":288046374912,"internally_reserved":0,"allocated":1106993152,"data_stored":1106159213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":756861955},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,11,11,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19632128,"num_objects":4793,"num_object_clones":0,"num_object_copies":9586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-378,"num_objects_unfound":0,"num_objects_dirty":4793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4793,"num_write_kb":19172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":378,"num_bytes_recovered":1548288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003847"},"pg_stats":[{"pgid":"2.f","version":"43'8547","reported_seq":8617,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.698668+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:20.698668+0000","last_peered":"2024-09-15T21:23:20.698668+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:20.698668+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8547,"log_dups_size":0,"ondisk_log_size":8547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35008512,"num_objects":8547,"num_object_clones":0,"num_object_copies":17094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8547,"num_write_kb":34188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4415},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8272","reported_seq":8373,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.353243+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:18.353243+0000","last_peered":"2024-09-15T21:23:18.353243+0000","last_clean":"2024-09-15T21:23:18.353243+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:18.353243+0000","last_undegraded":"2024-09-15T21:23:18.353243+0000","last_fullsized":"2024-09-15T21:23:18.353243+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8272,"log_dups_size":0,"ondisk_log_size":8272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33858502,"num_objects":8268,"num_object_clones":0,"num_object_copies":16536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8268,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6553,"num_write_kb":26212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8481","reported_seq":8602,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.405539+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:18.405539+0000","last_peered":"2024-09-15T21:23:18.405539+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:18.405539+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8481,"log_dups_size":0,"ondisk_log_size":8481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34721792,"num_objects":8477,"num_object_clones":0,"num_object_copies":16954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6581,"num_write_kb":26324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4210}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8119","reported_seq":8185,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.366213+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:18.366213+0000","last_peered":"2024-09-15T21:23:18.366213+0000","last_clean":"2024-09-15T21:23:18.366213+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:18.366213+0000","last_undegraded":"2024-09-15T21:23:18.366213+0000","last_fullsized":"2024-09-15T21:23:18.366213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8119,"log_dups_size":0,"ondisk_log_size":8119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33255424,"num_objects":8119,"num_object_clones":0,"num_object_copies":16238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8119,"num_write_kb":32476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8284","reported_seq":8412,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:20.657733+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:20.657733+0000","last_peered":"2024-09-15T21:23:20.657733+0000","last_clean":"2024-09-15T21:23:20.657733+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:20.657733+0000","last_undegraded":"2024-09-15T21:23:20.657733+0000","last_fullsized":"2024-09-15T21:23:20.657733+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8284,"log_dups_size":0,"ondisk_log_size":8284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33903075,"num_objects":8278,"num_object_clones":0,"num_object_copies":16556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4739,"num_write_kb":18956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8321","reported_seq":8453,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.703798+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:20.703798+0000","last_peered":"2024-09-15T21:23:20.703798+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:20.703798+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8321,"log_dups_size":0,"ondisk_log_size":8321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34062819,"num_objects":8317,"num_object_clones":0,"num_object_copies":16634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8317,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4997,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4241},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8368","reported_seq":8498,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.693410+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:20.693410+0000","last_peered":"2024-09-15T21:23:20.693410+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:20.693410+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8368,"log_dups_size":0,"ondisk_log_size":8368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34251718,"num_objects":8364,"num_object_clones":0,"num_object_copies":16728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4679,"num_write_kb":18716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4203},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8475","reported_seq":8599,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:20.704842+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:20.704842+0000","last_peered":"2024-09-15T21:23:20.704842+0000","last_clean":"2024-09-15T21:23:20.704842+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:20.704842+0000","last_undegraded":"2024-09-15T21:23:20.704842+0000","last_fullsized":"2024-09-15T21:23:20.704842+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8475,"log_dups_size":0,"ondisk_log_size":8475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34705408,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5247,"num_write_kb":20988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8270","reported_seq":8390,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.358136+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:18.358136+0000","last_peered":"2024-09-15T21:23:18.358136+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:18.358136+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8270,"log_dups_size":0,"ondisk_log_size":8270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33845731,"num_objects":8264,"num_object_clones":0,"num_object_copies":16528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8264,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6202,"num_write_kb":24808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4177}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8207","reported_seq":8333,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.701255+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:20.701255+0000","last_peered":"2024-09-15T21:23:20.701255+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:20.701255+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33583104,"num_objects":8199,"num_object_clones":0,"num_object_copies":16398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4880,"num_write_kb":19520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4179},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8284","reported_seq":8398,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.345282+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:18.345282+0000","last_peered":"2024-09-15T21:23:18.345282+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:18.345282+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8284,"log_dups_size":0,"ondisk_log_size":8284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33911267,"num_objects":8280,"num_object_clones":0,"num_object_copies":16560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8280,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6143,"num_write_kb":24572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4190}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.839588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.839588+0000","last_peered":"2024-09-15T21:21:14.839588+0000","last_clean":"2024-09-15T21:21:14.839588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.839588+0000","last_undegraded":"2024-09-15T21:21:14.839588+0000","last_fullsized":"2024-09-15T21:21:14.839588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8324","reported_seq":15401,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:18.388956+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:18.388956+0000","last_peered":"2024-09-15T21:23:18.388956+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:18.388956+0000","last_undegraded":"2024-09-15T21:23:18.388956+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8324,"log_dups_size":0,"ondisk_log_size":8324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34070528,"num_objects":8318,"num_object_clones":0,"num_object_copies":16636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":8318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6017,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3479,"num_bytes_recovered":14241792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":685},{"shards":"2","objects":7633}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8337","reported_seq":8466,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.623045+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:20.623045+0000","last_peered":"2024-09-15T21:23:20.623045+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:20.623045+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8337,"log_dups_size":0,"ondisk_log_size":8337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34140160,"num_objects":8335,"num_object_clones":0,"num_object_copies":16670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5281,"num_write_kb":21124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4241},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.537989+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.537989+0000","last_peered":"2024-09-15T21:21:14.537989+0000","last_clean":"2024-09-15T21:21:14.537989+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.537989+0000","last_undegraded":"2024-09-15T21:21:14.537989+0000","last_fullsized":"2024-09-15T21:21:14.537989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8306","reported_seq":8434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.501549+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:20.501549+0000","last_peered":"2024-09-15T21:23:20.501549+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:20.501549+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8306,"log_dups_size":0,"ondisk_log_size":8306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33988608,"num_objects":8298,"num_object_clones":0,"num_object_copies":16596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5215,"num_write_kb":20860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4159},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8195","reported_seq":8326,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.675210+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:20.675210+0000","last_peered":"2024-09-15T21:23:20.675210+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:20.675210+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8195,"log_dups_size":0,"ondisk_log_size":8195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33550336,"num_objects":8191,"num_object_clones":0,"num_object_copies":16382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4679,"num_write_kb":18716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4109},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.549401+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.549401+0000","last_peered":"2024-09-15T21:21:14.549401+0000","last_clean":"2024-09-15T21:21:14.549401+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.549401+0000","last_undegraded":"2024-09-15T21:21:14.549401+0000","last_fullsized":"2024-09-15T21:21:14.549401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363284+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.363284+0000","last_peered":"2024-09-15T21:21:14.363284+0000","last_clean":"2024-09-15T21:21:14.363284+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.363284+0000","last_undegraded":"2024-09-15T21:21:14.363284+0000","last_fullsized":"2024-09-15T21:21:14.363284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8368","reported_seq":8502,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.723575+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:20.723575+0000","last_peered":"2024-09-15T21:23:20.723575+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:20.723575+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8368,"log_dups_size":0,"ondisk_log_size":8368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34227142,"num_objects":8358,"num_object_clones":0,"num_object_copies":16716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5072,"num_write_kb":20288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4211},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":545084126,"num_objects":133086,"num_object_clones":0,"num_object_copies":266172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":133086,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":92951,"num_write_kb":371804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1107767296,"data_stored":1107619163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133158,"ondisk_log_size":133158,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649352,"kb_used_data":396920,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93722488,"statfs":{"total":96636764160,"available":95971827712,"internally_reserved":0,"allocated":406446080,"data_stored":406184066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":258469545},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754708,"kb_used_data":466244,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93617132,"statfs":{"total":96636764160,"available":95863943168,"internally_reserved":0,"allocated":477433856,"data_stored":477167765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295366315},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,6,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416172,"kb_used_data":217884,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93955668,"statfs":{"total":96636764160,"available":96210604032,"internally_reserved":0,"allocated":223113216,"data_stored":222807382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":203026095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477143040,"data_stored":477106910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224321536,"data_stored":224245663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406302720,"data_stored":406266590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:23.309 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:23.757 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 172 20 134908 134888 3.06292 1.69141 0.0856222 0.0254983 2024-09-15T21:23:23.902 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:23.903 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:23.923 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-15T21:23:22.735947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":545084145,"num_objects":133088,"num_object_clones":0,"num_object_copies":266176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":133088,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":92953,"num_write_kb":371806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3481,"num_bytes_recovered":14245907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133161,"ondisk_log_size":133161,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1820232,"kb_used_data":1081048,"kb_used_omap":12,"kb_used_meta":739123,"kb_avail":281295288,"statfs":{"total":289910292480,"available":288046374912,"internally_reserved":0,"allocated":1106993152,"data_stored":1106159213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":756861955},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,11,11,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19632128,"num_objects":4793,"num_object_clones":0,"num_object_copies":9586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-378,"num_objects_unfound":0,"num_objects_dirty":4793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4793,"num_write_kb":19172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":378,"num_bytes_recovered":1548288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003847"},"pg_stats":[{"pgid":"2.f","version":"43'8547","reported_seq":8617,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.698668+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:20.698668+0000","last_peered":"2024-09-15T21:23:20.698668+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:20.698668+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8547,"log_dups_size":0,"ondisk_log_size":8547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35008512,"num_objects":8547,"num_object_clones":0,"num_object_copies":17094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8547,"num_write_kb":34188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4415},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8272","reported_seq":8373,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.353243+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:18.353243+0000","last_peered":"2024-09-15T21:23:18.353243+0000","last_clean":"2024-09-15T21:23:18.353243+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:18.353243+0000","last_undegraded":"2024-09-15T21:23:18.353243+0000","last_fullsized":"2024-09-15T21:23:18.353243+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8272,"log_dups_size":0,"ondisk_log_size":8272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33858502,"num_objects":8268,"num_object_clones":0,"num_object_copies":16536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8268,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6553,"num_write_kb":26212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8481","reported_seq":8602,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.405539+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:18.405539+0000","last_peered":"2024-09-15T21:23:18.405539+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:18.405539+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8481,"log_dups_size":0,"ondisk_log_size":8481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34721792,"num_objects":8477,"num_object_clones":0,"num_object_copies":16954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6581,"num_write_kb":26324,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4210}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8119","reported_seq":8185,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:18.366213+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:18.366213+0000","last_peered":"2024-09-15T21:23:18.366213+0000","last_clean":"2024-09-15T21:23:18.366213+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:18.366213+0000","last_undegraded":"2024-09-15T21:23:18.366213+0000","last_fullsized":"2024-09-15T21:23:18.366213+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8119,"log_dups_size":0,"ondisk_log_size":8119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33255424,"num_objects":8119,"num_object_clones":0,"num_object_copies":16238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8119,"num_write_kb":32476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8284","reported_seq":8412,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:20.657733+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:20.657733+0000","last_peered":"2024-09-15T21:23:20.657733+0000","last_clean":"2024-09-15T21:23:20.657733+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:20.657733+0000","last_undegraded":"2024-09-15T21:23:20.657733+0000","last_fullsized":"2024-09-15T21:23:20.657733+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8284,"log_dups_size":0,"ondisk_log_size":8284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33903075,"num_objects":8278,"num_object_clones":0,"num_object_copies":16556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4739,"num_write_kb":18956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8321","reported_seq":8453,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.703798+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:20.703798+0000","last_peered":"2024-09-15T21:23:20.703798+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:20.703798+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8321,"log_dups_size":0,"ondisk_log_size":8321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34062819,"num_objects":8317,"num_object_clones":0,"num_object_copies":16634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8317,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4997,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4241},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8368","reported_seq":8498,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.693410+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:20.693410+0000","last_peered":"2024-09-15T21:23:20.693410+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:20.693410+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8368,"log_dups_size":0,"ondisk_log_size":8368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34251718,"num_objects":8364,"num_object_clones":0,"num_object_copies":16728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4679,"num_write_kb":18716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4203},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8475","reported_seq":8599,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:20.704842+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:20.704842+0000","last_peered":"2024-09-15T21:23:20.704842+0000","last_clean":"2024-09-15T21:23:20.704842+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:20.704842+0000","last_undegraded":"2024-09-15T21:23:20.704842+0000","last_fullsized":"2024-09-15T21:23:20.704842+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8475,"log_dups_size":0,"ondisk_log_size":8475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34705408,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5247,"num_write_kb":20988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8270","reported_seq":8390,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.358136+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:18.358136+0000","last_peered":"2024-09-15T21:23:18.358136+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:18.358136+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8270,"log_dups_size":0,"ondisk_log_size":8270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33845731,"num_objects":8264,"num_object_clones":0,"num_object_copies":16528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8264,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6202,"num_write_kb":24808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4177}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8207","reported_seq":8333,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.701255+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:20.701255+0000","last_peered":"2024-09-15T21:23:20.701255+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:20.701255+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33583104,"num_objects":8199,"num_object_clones":0,"num_object_copies":16398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4880,"num_write_kb":19520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4179},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8284","reported_seq":8398,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:18.345282+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:18.345282+0000","last_peered":"2024-09-15T21:23:18.345282+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:18.345282+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8284,"log_dups_size":0,"ondisk_log_size":8284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33911267,"num_objects":8280,"num_object_clones":0,"num_object_copies":16560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8280,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6143,"num_write_kb":24572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4190}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.839588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.839588+0000","last_peered":"2024-09-15T21:21:14.839588+0000","last_clean":"2024-09-15T21:21:14.839588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.839588+0000","last_undegraded":"2024-09-15T21:21:14.839588+0000","last_fullsized":"2024-09-15T21:21:14.839588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8324","reported_seq":15401,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:18.388956+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:18.388956+0000","last_peered":"2024-09-15T21:23:18.388956+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:18.388956+0000","last_undegraded":"2024-09-15T21:23:18.388956+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8324,"log_dups_size":0,"ondisk_log_size":8324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34070528,"num_objects":8318,"num_object_clones":0,"num_object_copies":16636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":8318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6017,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3479,"num_bytes_recovered":14241792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":685},{"shards":"2","objects":7633}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8337","reported_seq":8466,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.623045+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:20.623045+0000","last_peered":"2024-09-15T21:23:20.623045+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:20.623045+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8337,"log_dups_size":0,"ondisk_log_size":8337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34140160,"num_objects":8335,"num_object_clones":0,"num_object_copies":16670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5281,"num_write_kb":21124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4241},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.537989+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.537989+0000","last_peered":"2024-09-15T21:21:14.537989+0000","last_clean":"2024-09-15T21:21:14.537989+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.537989+0000","last_undegraded":"2024-09-15T21:21:14.537989+0000","last_fullsized":"2024-09-15T21:21:14.537989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8306","reported_seq":8434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.501549+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:20.501549+0000","last_peered":"2024-09-15T21:23:20.501549+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:20.501549+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8306,"log_dups_size":0,"ondisk_log_size":8306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33988608,"num_objects":8298,"num_object_clones":0,"num_object_copies":16596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5215,"num_write_kb":20860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4159},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8195","reported_seq":8326,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.675210+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:20.675210+0000","last_peered":"2024-09-15T21:23:20.675210+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:20.675210+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8195,"log_dups_size":0,"ondisk_log_size":8195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33550336,"num_objects":8191,"num_object_clones":0,"num_object_copies":16382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4679,"num_write_kb":18716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4109},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.549401+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.549401+0000","last_peered":"2024-09-15T21:21:14.549401+0000","last_clean":"2024-09-15T21:21:14.549401+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.549401+0000","last_undegraded":"2024-09-15T21:21:14.549401+0000","last_fullsized":"2024-09-15T21:21:14.549401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363284+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.363284+0000","last_peered":"2024-09-15T21:21:14.363284+0000","last_clean":"2024-09-15T21:21:14.363284+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.363284+0000","last_undegraded":"2024-09-15T21:21:14.363284+0000","last_fullsized":"2024-09-15T21:21:14.363284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8368","reported_seq":8502,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.723575+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:20.723575+0000","last_peered":"2024-09-15T21:23:20.723575+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:20.723575+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8368,"log_dups_size":0,"ondisk_log_size":8368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34227142,"num_objects":8358,"num_object_clones":0,"num_object_copies":16716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5072,"num_write_kb":20288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4211},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":545084126,"num_objects":133086,"num_object_clones":0,"num_object_copies":266172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":685,"num_objects_unfound":0,"num_objects_dirty":133086,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":92951,"num_write_kb":371804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3480,"num_bytes_recovered":14245888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1107767296,"data_stored":1107619163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":133158,"ondisk_log_size":133158,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649352,"kb_used_data":396920,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93722488,"statfs":{"total":96636764160,"available":95971827712,"internally_reserved":0,"allocated":406446080,"data_stored":406184066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":258469545},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754708,"kb_used_data":466244,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93617132,"statfs":{"total":96636764160,"available":95863943168,"internally_reserved":0,"allocated":477433856,"data_stored":477167765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295366315},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,6,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416172,"kb_used_data":217884,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93955668,"statfs":{"total":96636764160,"available":96210604032,"internally_reserved":0,"allocated":223113216,"data_stored":222807382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":203026095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477143040,"data_stored":477106910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224321536,"data_stored":224245663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406302720,"data_stored":406266590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:23.924 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:23.924 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8547", 'reported_seq': 8617, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:20.698668+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:23:20.698668+0000', 'last_peered': '2024-09-15T21:23:20.698668+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:23:20.698668+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 8547, 'log_dups_size': 0, 'ondisk_log_size': 8547, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35008512, 'num_objects': 8547, 'num_object_clones': 0, 'num_object_copies': 17094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8547, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8547, 'num_write_kb': 34188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4415}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:23.924 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:23:23.924 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8481", 'reported_seq': 8602, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:18.405539+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:23:18.405539+0000', 'last_peered': '2024-09-15T21:23:18.405539+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:23:18.405539+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 8481, 'log_dups_size': 0, 'ondisk_log_size': 8481, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34721792, 'num_objects': 8477, 'num_object_clones': 0, 'num_object_copies': 16954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6581, 'num_write_kb': 26324, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 4210}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:23.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:23.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8321", 'reported_seq': 8453, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:20.703798+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:23:20.703798+0000', 'last_peered': '2024-09-15T21:23:20.703798+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:23:20.703798+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 8321, 'log_dups_size': 0, 'ondisk_log_size': 8321, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34062819, 'num_objects': 8317, 'num_object_clones': 0, 'num_object_copies': 16634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8317, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 4997, 'num_write_kb': 19988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4241}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:23.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:23.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8368", 'reported_seq': 8498, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:20.693410+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:23:20.693410+0000', 'last_peered': '2024-09-15T21:23:20.693410+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:23:20.693410+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 8368, 'log_dups_size': 0, 'ondisk_log_size': 8368, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34251718, 'num_objects': 8364, 'num_object_clones': 0, 'num_object_copies': 16728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8364, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4679, 'num_write_kb': 18716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4203}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:23.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:23.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'8270", 'reported_seq': 8390, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:18.358136+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:23:18.358136+0000', 'last_peered': '2024-09-15T21:23:18.358136+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:23:18.358136+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 8270, 'log_dups_size': 0, 'ondisk_log_size': 8270, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33845731, 'num_objects': 8264, 'num_object_clones': 0, 'num_object_copies': 16528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8264, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6202, 'num_write_kb': 24808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 4177}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:23.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:23.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8207", 'reported_seq': 8333, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:20.701255+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:23:20.701255+0000', 'last_peered': '2024-09-15T21:23:20.701255+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:23:20.701255+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 8207, 'log_dups_size': 0, 'ondisk_log_size': 8207, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33583104, 'num_objects': 8199, 'num_object_clones': 0, 'num_object_copies': 16398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4880, 'num_write_kb': 19520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4179}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:23.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:23.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8284", 'reported_seq': 8398, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:18.345282+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:23:18.345282+0000', 'last_peered': '2024-09-15T21:23:18.345282+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:23:18.345282+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 8284, 'log_dups_size': 0, 'ondisk_log_size': 8284, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33911267, 'num_objects': 8280, 'num_object_clones': 0, 'num_object_copies': 16560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8280, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6143, 'num_write_kb': 24572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 4190}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:23.926 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:23.926 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8324", 'reported_seq': 15401, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:23:18.388956+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:23:18.388956+0000', 'last_peered': '2024-09-15T21:23:18.388956+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:23:18.388956+0000', 'last_undegraded': '2024-09-15T21:23:18.388956+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 8324, 'log_dups_size': 0, 'ondisk_log_size': 8324, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34070528, 'num_objects': 8318, 'num_object_clones': 0, 'num_object_copies': 16636, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 685, 'num_objects_unfound': 0, 'num_objects_dirty': 8318, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6017, 'num_write_kb': 24068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3479, 'num_bytes_recovered': 14241792, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 685}, {'shards': '2', 'objects': 7633}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:23.926 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:23.926 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8337", 'reported_seq': 8466, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:20.623045+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:23:20.623045+0000', 'last_peered': '2024-09-15T21:23:20.623045+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:23:20.623045+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 8337, 'log_dups_size': 0, 'ondisk_log_size': 8337, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34140160, 'num_objects': 8335, 'num_object_clones': 0, 'num_object_copies': 16670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5281, 'num_write_kb': 21124, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4241}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:23.926 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:23.926 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8306", 'reported_seq': 8434, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:20.501549+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:23:20.501549+0000', 'last_peered': '2024-09-15T21:23:20.501549+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:23:20.501549+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 8306, 'log_dups_size': 0, 'ondisk_log_size': 8306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33988608, 'num_objects': 8298, 'num_object_clones': 0, 'num_object_copies': 16596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8298, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5215, 'num_write_kb': 20860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4159}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:23.926 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:23.926 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8195", 'reported_seq': 8326, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:20.675210+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:23:20.675210+0000', 'last_peered': '2024-09-15T21:23:20.675210+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:23:20.675210+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 8195, 'log_dups_size': 0, 'ondisk_log_size': 8195, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33550336, 'num_objects': 8191, 'num_object_clones': 0, 'num_object_copies': 16382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8191, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4679, 'num_write_kb': 18716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4109}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:23.926 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:23:23.926 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8368", 'reported_seq': 8502, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:20.723575+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:23:20.723575+0000', 'last_peered': '2024-09-15T21:23:20.723575+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:23:20.723575+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 8368, 'log_dups_size': 0, 'ondisk_log_size': 8368, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34227142, 'num_objects': 8358, 'num_object_clones': 0, 'num_object_copies': 16716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8358, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5072, 'num_write_kb': 20288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4211}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:23.927 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:24.613 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:24.613 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/266176 objects degraded (17.022%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":256,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":133088,"data_bytes":545084145,"bytes_used":1863917568,"bytes_avail":288046374912,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":266176,"degraded_ratio":0.17022195840346238,"misplaced_objects":685,"misplaced_total":266176,"misplaced_ratio":0.0025734852127915365,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":128982,"recovering_keys_per_sec":0,"num_objects_recovered":378,"num_bytes_recovered":1548288,"num_keys_recovered":0,"write_bytes_sec":1635486,"read_op_per_sec":0,"write_op_per_sec":399},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:24.642 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:24.642 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:24.757 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 173 20 135355 135335 3.05531 1.74609 0.0748333 0.0255623 2024-09-15T21:23:25.192 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:25.216 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-09-15T21:23:24.737366+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":548549361,"num_objects":133934,"num_object_clones":0,"num_object_copies":267868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":475,"num_objects_unfound":0,"num_objects_dirty":133934,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":93799,"num_write_kb":375190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3691,"num_bytes_recovered":15106067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134007,"ondisk_log_size":134007,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1828892,"kb_used_data":1085548,"kb_used_omap":12,"kb_used_meta":743283,"kb_avail":281286628,"statfs":{"total":289910292480,"available":288037507072,"internally_reserved":0,"allocated":1111601152,"data_stored":1110767213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":761121795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,12,8,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23097344,"num_objects":5639,"num_object_clones":0,"num_object_copies":11278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-588,"num_objects_unfound":0,"num_objects_dirty":5639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5639,"num_write_kb":22556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":588,"num_bytes_recovered":2408448,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004454"},"pg_stats":[{"pgid":"2.f","version":"43'8547","reported_seq":8617,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.698668+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:20.698668+0000","last_peered":"2024-09-15T21:23:20.698668+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:20.698668+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8547,"log_dups_size":0,"ondisk_log_size":8547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35008512,"num_objects":8547,"num_object_clones":0,"num_object_copies":17094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8547,"num_write_kb":34188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4415},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8424","reported_seq":8525,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.387635+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:23.387635+0000","last_peered":"2024-09-15T21:23:23.387635+0000","last_clean":"2024-09-15T21:23:23.387635+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:23.387635+0000","last_undegraded":"2024-09-15T21:23:23.387635+0000","last_fullsized":"2024-09-15T21:23:23.387635+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8424,"log_dups_size":0,"ondisk_log_size":8424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34481094,"num_objects":8420,"num_object_clones":0,"num_object_copies":16840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8420,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6705,"num_write_kb":26820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8613","reported_seq":8734,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.271390+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:23.271390+0000","last_peered":"2024-09-15T21:23:23.271390+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:23.271390+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8613,"log_dups_size":0,"ondisk_log_size":8613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35262464,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6713,"num_write_kb":26852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4342}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8253","reported_seq":8319,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:23.348542+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:23.348542+0000","last_peered":"2024-09-15T21:23:23.348542+0000","last_clean":"2024-09-15T21:23:23.348542+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:23.348542+0000","last_undegraded":"2024-09-15T21:23:23.348542+0000","last_fullsized":"2024-09-15T21:23:23.348542+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8253,"log_dups_size":0,"ondisk_log_size":8253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33804288,"num_objects":8253,"num_object_clones":0,"num_object_copies":16506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8253,"num_write_kb":33012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8284","reported_seq":8412,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:20.657733+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:20.657733+0000","last_peered":"2024-09-15T21:23:20.657733+0000","last_clean":"2024-09-15T21:23:20.657733+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:20.657733+0000","last_undegraded":"2024-09-15T21:23:20.657733+0000","last_fullsized":"2024-09-15T21:23:20.657733+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8284,"log_dups_size":0,"ondisk_log_size":8284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33903075,"num_objects":8278,"num_object_clones":0,"num_object_copies":16556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4739,"num_write_kb":18956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8321","reported_seq":8453,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.703798+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:20.703798+0000","last_peered":"2024-09-15T21:23:20.703798+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:20.703798+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8321,"log_dups_size":0,"ondisk_log_size":8321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34062819,"num_objects":8317,"num_object_clones":0,"num_object_copies":16634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8317,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4997,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4241},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8368","reported_seq":8498,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.693410+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:20.693410+0000","last_peered":"2024-09-15T21:23:20.693410+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:20.693410+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8368,"log_dups_size":0,"ondisk_log_size":8368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34251718,"num_objects":8364,"num_object_clones":0,"num_object_copies":16728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4679,"num_write_kb":18716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4203},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8475","reported_seq":8599,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:20.704842+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:20.704842+0000","last_peered":"2024-09-15T21:23:20.704842+0000","last_clean":"2024-09-15T21:23:20.704842+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:20.704842+0000","last_undegraded":"2024-09-15T21:23:20.704842+0000","last_fullsized":"2024-09-15T21:23:20.704842+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8475,"log_dups_size":0,"ondisk_log_size":8475,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34705408,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5247,"num_write_kb":20988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8411","reported_seq":8533,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.321815+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:23.321815+0000","last_peered":"2024-09-15T21:23:23.321815+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:23.321815+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8411,"log_dups_size":0,"ondisk_log_size":8411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34423267,"num_objects":8405,"num_object_clones":0,"num_object_copies":16810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6343,"num_write_kb":25372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4318}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8207","reported_seq":8333,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.701255+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:20.701255+0000","last_peered":"2024-09-15T21:23:20.701255+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:20.701255+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33583104,"num_objects":8199,"num_object_clones":0,"num_object_copies":16398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4880,"num_write_kb":19520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4179},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8427","reported_seq":8541,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:23.391210+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:23.391210+0000","last_peered":"2024-09-15T21:23:23.391210+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:23.391210+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8427,"log_dups_size":0,"ondisk_log_size":8427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34496995,"num_objects":8423,"num_object_clones":0,"num_object_copies":16846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8423,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6286,"num_write_kb":25144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4333}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.839588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.839588+0000","last_peered":"2024-09-15T21:21:14.839588+0000","last_clean":"2024-09-15T21:21:14.839588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.839588+0000","last_undegraded":"2024-09-15T21:21:14.839588+0000","last_fullsized":"2024-09-15T21:21:14.839588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8468","reported_seq":15964,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:23.423948+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:23.423948+0000","last_peered":"2024-09-15T21:23:23.423948+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:23.423948+0000","last_undegraded":"2024-09-15T21:23:23.423948+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8468,"log_dups_size":0,"ondisk_log_size":8468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34660352,"num_objects":8462,"num_object_clones":0,"num_object_copies":16924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":475,"num_objects_unfound":0,"num_objects_dirty":8462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6161,"num_write_kb":24644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3689,"num_bytes_recovered":15101952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":475},{"shards":"2","objects":7987}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8337","reported_seq":8466,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.623045+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:20.623045+0000","last_peered":"2024-09-15T21:23:20.623045+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:20.623045+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8337,"log_dups_size":0,"ondisk_log_size":8337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34140160,"num_objects":8335,"num_object_clones":0,"num_object_copies":16670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5281,"num_write_kb":21124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4241},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.537989+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.537989+0000","last_peered":"2024-09-15T21:21:14.537989+0000","last_clean":"2024-09-15T21:21:14.537989+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.537989+0000","last_undegraded":"2024-09-15T21:21:14.537989+0000","last_fullsized":"2024-09-15T21:21:14.537989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8306","reported_seq":8434,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.501549+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:20.501549+0000","last_peered":"2024-09-15T21:23:20.501549+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:20.501549+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8306,"log_dups_size":0,"ondisk_log_size":8306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33988608,"num_objects":8298,"num_object_clones":0,"num_object_copies":16596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5215,"num_write_kb":20860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4159},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8195","reported_seq":8326,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.675210+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:20.675210+0000","last_peered":"2024-09-15T21:23:20.675210+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:20.675210+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8195,"log_dups_size":0,"ondisk_log_size":8195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33550336,"num_objects":8191,"num_object_clones":0,"num_object_copies":16382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4679,"num_write_kb":18716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4109},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.549401+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.549401+0000","last_peered":"2024-09-15T21:21:14.549401+0000","last_clean":"2024-09-15T21:21:14.549401+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.549401+0000","last_undegraded":"2024-09-15T21:21:14.549401+0000","last_fullsized":"2024-09-15T21:21:14.549401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363284+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.363284+0000","last_peered":"2024-09-15T21:21:14.363284+0000","last_clean":"2024-09-15T21:21:14.363284+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.363284+0000","last_undegraded":"2024-09-15T21:21:14.363284+0000","last_fullsized":"2024-09-15T21:21:14.363284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8368","reported_seq":8502,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:20.723575+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:20.723575+0000","last_peered":"2024-09-15T21:23:20.723575+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:20.723575+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8368,"log_dups_size":0,"ondisk_log_size":8368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34227142,"num_objects":8358,"num_object_clones":0,"num_object_copies":16716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8358,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5072,"num_write_kb":20288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4211},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":548549342,"num_objects":133932,"num_object_clones":0,"num_object_copies":267864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":475,"num_objects_unfound":0,"num_objects_dirty":133932,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":93797,"num_write_kb":375188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3690,"num_bytes_recovered":15106048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1112375296,"data_stored":1112227163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134004,"ondisk_log_size":134004,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658012,"kb_used_data":401420,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93713828,"statfs":{"total":96636764160,"available":95962959872,"internally_reserved":0,"allocated":411054080,"data_stored":410792066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":262729385},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754708,"kb_used_data":466244,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93617132,"statfs":{"total":96636764160,"available":95863943168,"internally_reserved":0,"allocated":477433856,"data_stored":477167765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":295366315},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,6,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416172,"kb_used_data":217884,"kb_used_omap":4,"kb_used_meta":198267,"kb_avail":93955668,"statfs":{"total":96636764160,"available":96210604032,"internally_reserved":0,"allocated":223113216,"data_stored":222807382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":203026095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477143040,"data_stored":477106910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224321536,"data_stored":224245663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":410910720,"data_stored":410874590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:25.757 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 174 20 135824 135804 3.04828 1.83203 0.0239367 0.0256232 2024-09-15T21:23:26.757 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 175 20 136331 136311 3.04218 1.98047 0.0377401 0.0256734 2024-09-15T21:23:27.757 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 176 20 136830 136810 3.03597 1.94922 0.053459 0.0257258 2024-09-15T21:23:28.222 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:28.757 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 177 20 137306 137286 3.02932 1.85938 0.0660854 0.025782 2024-09-15T21:23:28.794 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:28.796 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:28.826 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-15T21:23:28.738551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":558359281,"num_objects":136329,"num_object_clones":0,"num_object_copies":272658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":136329,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":96194,"num_write_kb":384770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":15953939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136402,"ondisk_log_size":136402,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898932,"kb_used_data":1105600,"kb_used_omap":12,"kb_used_meta":793267,"kb_avail":281216588,"statfs":{"total":289910292480,"available":287965786112,"internally_reserved":0,"allocated":1132134400,"data_stored":1131300461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":812305411},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,1,6,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23052288,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-611,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5628,"num_write_kb":22512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":611,"num_bytes_recovered":2502656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004469"},"pg_stats":[{"pgid":"2.f","version":"43'8717","reported_seq":8788,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.703674+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:25.703674+0000","last_peered":"2024-09-15T21:23:25.703674+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:25.703674+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8717,"log_dups_size":0,"ondisk_log_size":8717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35704832,"num_objects":8717,"num_object_clones":0,"num_object_copies":17434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8717,"num_write_kb":34868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4585},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8595","reported_seq":8696,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.426270+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:28.426270+0000","last_peered":"2024-09-15T21:23:28.426270+0000","last_clean":"2024-09-15T21:23:28.426270+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:28.426270+0000","last_undegraded":"2024-09-15T21:23:28.426270+0000","last_fullsized":"2024-09-15T21:23:28.426270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8595,"log_dups_size":0,"ondisk_log_size":8595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35181510,"num_objects":8591,"num_object_clones":0,"num_object_copies":17182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8591,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6876,"num_write_kb":27504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8756","reported_seq":8877,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.376160+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:28.376160+0000","last_peered":"2024-09-15T21:23:28.376160+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:28.376160+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8756,"log_dups_size":0,"ondisk_log_size":8756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6856,"num_write_kb":27424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4485}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8415","reported_seq":8479,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.421422+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:28.421422+0000","last_peered":"2024-09-15T21:23:28.421422+0000","last_clean":"2024-09-15T21:23:28.421422+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:28.421422+0000","last_undegraded":"2024-09-15T21:23:28.421422+0000","last_fullsized":"2024-09-15T21:23:28.421422+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8415,"log_dups_size":0,"ondisk_log_size":8415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8415,"num_write_kb":33660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8412","reported_seq":8540,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:25.652182+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:25.652182+0000","last_peered":"2024-09-15T21:23:25.652182+0000","last_clean":"2024-09-15T21:23:25.652182+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:25.652182+0000","last_undegraded":"2024-09-15T21:23:25.652182+0000","last_fullsized":"2024-09-15T21:23:25.652182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8412,"log_dups_size":0,"ondisk_log_size":8412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34427363,"num_objects":8406,"num_object_clones":0,"num_object_copies":16812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4867,"num_write_kb":19468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8459","reported_seq":8591,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.497306+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:25.497306+0000","last_peered":"2024-09-15T21:23:25.497306+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:25.497306+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8459,"log_dups_size":0,"ondisk_log_size":8459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34628067,"num_objects":8455,"num_object_clones":0,"num_object_copies":16910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8455,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5135,"num_write_kb":20540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4379},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8524","reported_seq":8656,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.703354+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:25.703354+0000","last_peered":"2024-09-15T21:23:25.703354+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:25.703354+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8524,"log_dups_size":0,"ondisk_log_size":8524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34890694,"num_objects":8520,"num_object_clones":0,"num_object_copies":17040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4835,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4359},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8608","reported_seq":8732,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:25.640076+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:25.640076+0000","last_peered":"2024-09-15T21:23:25.640076+0000","last_clean":"2024-09-15T21:23:25.640076+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:25.640076+0000","last_undegraded":"2024-09-15T21:23:25.640076+0000","last_fullsized":"2024-09-15T21:23:25.640076+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8608,"log_dups_size":0,"ondisk_log_size":8608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35250176,"num_objects":8606,"num_object_clones":0,"num_object_copies":17212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5380,"num_write_kb":21520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8552","reported_seq":8674,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.392286+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:28.392286+0000","last_peered":"2024-09-15T21:23:28.392286+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:28.392286+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8552,"log_dups_size":0,"ondisk_log_size":8552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35000803,"num_objects":8546,"num_object_clones":0,"num_object_copies":17092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8546,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6484,"num_write_kb":25936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4459}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8345","reported_seq":8471,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.710930+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:25.710930+0000","last_peered":"2024-09-15T21:23:25.710930+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:25.710930+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8345,"log_dups_size":0,"ondisk_log_size":8345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34148352,"num_objects":8337,"num_object_clones":0,"num_object_copies":16674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5018,"num_write_kb":20072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4317},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8598","reported_seq":8712,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.281505+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:28.281505+0000","last_peered":"2024-09-15T21:23:28.281505+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:28.281505+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8598,"log_dups_size":0,"ondisk_log_size":8598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35197411,"num_objects":8594,"num_object_clones":0,"num_object_copies":17188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8594,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6457,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4504}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.839588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.839588+0000","last_peered":"2024-09-15T21:21:14.839588+0000","last_clean":"2024-09-15T21:21:14.839588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.839588+0000","last_undegraded":"2024-09-15T21:21:14.839588+0000","last_fullsized":"2024-09-15T21:21:14.839588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8619","reported_seq":16530,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:28.429652+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:28.429652+0000","last_peered":"2024-09-15T21:23:28.429652+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:28.429652+0000","last_undegraded":"2024-09-15T21:23:28.429652+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8619,"log_dups_size":0,"ondisk_log_size":8619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35278848,"num_objects":8613,"num_object_clones":0,"num_object_copies":17226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":8613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6312,"num_write_kb":25248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3896,"num_bytes_recovered":15949824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":268},{"shards":"2","objects":8345}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8489","reported_seq":8618,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.627516+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:25.627516+0000","last_peered":"2024-09-15T21:23:25.627516+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:25.627516+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8489,"log_dups_size":0,"ondisk_log_size":8489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34762752,"num_objects":8487,"num_object_clones":0,"num_object_copies":16974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5433,"num_write_kb":21732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4393},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.537989+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.537989+0000","last_peered":"2024-09-15T21:21:14.537989+0000","last_clean":"2024-09-15T21:21:14.537989+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.537989+0000","last_undegraded":"2024-09-15T21:21:14.537989+0000","last_fullsized":"2024-09-15T21:21:14.537989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8443","reported_seq":8570,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.744029+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:25.744029+0000","last_peered":"2024-09-15T21:23:25.744029+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:25.744029+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8443,"log_dups_size":0,"ondisk_log_size":8443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34549760,"num_objects":8435,"num_object_clones":0,"num_object_copies":16870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5352,"num_write_kb":21408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4296},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8342","reported_seq":8473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.727140+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:25.727140+0000","last_peered":"2024-09-15T21:23:25.727140+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:25.727140+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8342,"log_dups_size":0,"ondisk_log_size":8342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34152448,"num_objects":8338,"num_object_clones":0,"num_object_copies":16676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4826,"num_write_kb":19304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4256},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.549401+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.549401+0000","last_peered":"2024-09-15T21:21:14.549401+0000","last_clean":"2024-09-15T21:21:14.549401+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.549401+0000","last_undegraded":"2024-09-15T21:21:14.549401+0000","last_fullsized":"2024-09-15T21:21:14.549401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363284+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.363284+0000","last_peered":"2024-09-15T21:21:14.363284+0000","last_clean":"2024-09-15T21:21:14.363284+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.363284+0000","last_undegraded":"2024-09-15T21:21:14.363284+0000","last_fullsized":"2024-09-15T21:21:14.363284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8525","reported_seq":8659,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.710554+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:25.710554+0000","last_peered":"2024-09-15T21:23:25.710554+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:25.710554+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8525,"log_dups_size":0,"ondisk_log_size":8525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34870214,"num_objects":8515,"num_object_clones":0,"num_object_copies":17030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5229,"num_write_kb":20916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4368},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":558359262,"num_objects":136327,"num_object_clones":0,"num_object_copies":272654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":136327,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":96192,"num_write_kb":384768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3897,"num_bytes_recovered":15953920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1132670976,"data_stored":1132522843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136399,"ondisk_log_size":136399,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680844,"kb_used_data":406204,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93690996,"statfs":{"total":96636764160,"available":95939579904,"internally_reserved":0,"allocated":415952896,"data_stored":415690882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":782688,"kb_used_data":472012,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93589152,"statfs":{"total":96636764160,"available":95835291648,"internally_reserved":0,"allocated":483340288,"data_stored":483074197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,2,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435400,"kb_used_data":227384,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93936440,"statfs":{"total":96636764160,"available":96190914560,"internally_reserved":0,"allocated":232841216,"data_stored":232535382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":212987567},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484323328,"data_stored":484287198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":232488960,"data_stored":232413087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":415858688,"data_stored":415822558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:28.828 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:29.367 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:29.367 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:29.391 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-15T21:23:28.738551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":558359281,"num_objects":136329,"num_object_clones":0,"num_object_copies":272658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":136329,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":96194,"num_write_kb":384770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":15953939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136402,"ondisk_log_size":136402,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898932,"kb_used_data":1105600,"kb_used_omap":12,"kb_used_meta":793267,"kb_avail":281216588,"statfs":{"total":289910292480,"available":287965786112,"internally_reserved":0,"allocated":1132134400,"data_stored":1131300461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":812305411},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,1,6,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23052288,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-611,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5628,"num_write_kb":22512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":611,"num_bytes_recovered":2502656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004469"},"pg_stats":[{"pgid":"2.f","version":"43'8717","reported_seq":8788,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.703674+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:25.703674+0000","last_peered":"2024-09-15T21:23:25.703674+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:25.703674+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8717,"log_dups_size":0,"ondisk_log_size":8717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35704832,"num_objects":8717,"num_object_clones":0,"num_object_copies":17434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8717,"num_write_kb":34868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4585},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8595","reported_seq":8696,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.426270+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:28.426270+0000","last_peered":"2024-09-15T21:23:28.426270+0000","last_clean":"2024-09-15T21:23:28.426270+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:28.426270+0000","last_undegraded":"2024-09-15T21:23:28.426270+0000","last_fullsized":"2024-09-15T21:23:28.426270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8595,"log_dups_size":0,"ondisk_log_size":8595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35181510,"num_objects":8591,"num_object_clones":0,"num_object_copies":17182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8591,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6876,"num_write_kb":27504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8756","reported_seq":8877,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.376160+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:28.376160+0000","last_peered":"2024-09-15T21:23:28.376160+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:28.376160+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8756,"log_dups_size":0,"ondisk_log_size":8756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6856,"num_write_kb":27424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4485}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8415","reported_seq":8479,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.421422+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:28.421422+0000","last_peered":"2024-09-15T21:23:28.421422+0000","last_clean":"2024-09-15T21:23:28.421422+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:28.421422+0000","last_undegraded":"2024-09-15T21:23:28.421422+0000","last_fullsized":"2024-09-15T21:23:28.421422+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8415,"log_dups_size":0,"ondisk_log_size":8415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8415,"num_write_kb":33660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8412","reported_seq":8540,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:25.652182+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:25.652182+0000","last_peered":"2024-09-15T21:23:25.652182+0000","last_clean":"2024-09-15T21:23:25.652182+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:25.652182+0000","last_undegraded":"2024-09-15T21:23:25.652182+0000","last_fullsized":"2024-09-15T21:23:25.652182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8412,"log_dups_size":0,"ondisk_log_size":8412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34427363,"num_objects":8406,"num_object_clones":0,"num_object_copies":16812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4867,"num_write_kb":19468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8459","reported_seq":8591,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.497306+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:25.497306+0000","last_peered":"2024-09-15T21:23:25.497306+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:25.497306+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8459,"log_dups_size":0,"ondisk_log_size":8459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34628067,"num_objects":8455,"num_object_clones":0,"num_object_copies":16910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8455,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5135,"num_write_kb":20540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4379},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8524","reported_seq":8656,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.703354+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:25.703354+0000","last_peered":"2024-09-15T21:23:25.703354+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:25.703354+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8524,"log_dups_size":0,"ondisk_log_size":8524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34890694,"num_objects":8520,"num_object_clones":0,"num_object_copies":17040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4835,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4359},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8608","reported_seq":8732,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:25.640076+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:25.640076+0000","last_peered":"2024-09-15T21:23:25.640076+0000","last_clean":"2024-09-15T21:23:25.640076+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:25.640076+0000","last_undegraded":"2024-09-15T21:23:25.640076+0000","last_fullsized":"2024-09-15T21:23:25.640076+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8608,"log_dups_size":0,"ondisk_log_size":8608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35250176,"num_objects":8606,"num_object_clones":0,"num_object_copies":17212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5380,"num_write_kb":21520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8552","reported_seq":8674,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.392286+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:28.392286+0000","last_peered":"2024-09-15T21:23:28.392286+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:28.392286+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8552,"log_dups_size":0,"ondisk_log_size":8552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35000803,"num_objects":8546,"num_object_clones":0,"num_object_copies":17092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8546,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6484,"num_write_kb":25936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4459}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8345","reported_seq":8471,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.710930+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:25.710930+0000","last_peered":"2024-09-15T21:23:25.710930+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:25.710930+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8345,"log_dups_size":0,"ondisk_log_size":8345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34148352,"num_objects":8337,"num_object_clones":0,"num_object_copies":16674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5018,"num_write_kb":20072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4317},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8598","reported_seq":8712,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.281505+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:28.281505+0000","last_peered":"2024-09-15T21:23:28.281505+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:28.281505+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8598,"log_dups_size":0,"ondisk_log_size":8598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35197411,"num_objects":8594,"num_object_clones":0,"num_object_copies":17188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8594,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6457,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4504}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.839588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.839588+0000","last_peered":"2024-09-15T21:21:14.839588+0000","last_clean":"2024-09-15T21:21:14.839588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.839588+0000","last_undegraded":"2024-09-15T21:21:14.839588+0000","last_fullsized":"2024-09-15T21:21:14.839588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8619","reported_seq":16530,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:28.429652+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:28.429652+0000","last_peered":"2024-09-15T21:23:28.429652+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:28.429652+0000","last_undegraded":"2024-09-15T21:23:28.429652+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8619,"log_dups_size":0,"ondisk_log_size":8619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35278848,"num_objects":8613,"num_object_clones":0,"num_object_copies":17226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":8613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6312,"num_write_kb":25248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3896,"num_bytes_recovered":15949824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":268},{"shards":"2","objects":8345}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8489","reported_seq":8618,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.627516+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:25.627516+0000","last_peered":"2024-09-15T21:23:25.627516+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:25.627516+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8489,"log_dups_size":0,"ondisk_log_size":8489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34762752,"num_objects":8487,"num_object_clones":0,"num_object_copies":16974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5433,"num_write_kb":21732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4393},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.537989+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.537989+0000","last_peered":"2024-09-15T21:21:14.537989+0000","last_clean":"2024-09-15T21:21:14.537989+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.537989+0000","last_undegraded":"2024-09-15T21:21:14.537989+0000","last_fullsized":"2024-09-15T21:21:14.537989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8443","reported_seq":8570,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.744029+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:25.744029+0000","last_peered":"2024-09-15T21:23:25.744029+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:25.744029+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8443,"log_dups_size":0,"ondisk_log_size":8443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34549760,"num_objects":8435,"num_object_clones":0,"num_object_copies":16870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5352,"num_write_kb":21408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4296},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8342","reported_seq":8473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.727140+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:25.727140+0000","last_peered":"2024-09-15T21:23:25.727140+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:25.727140+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8342,"log_dups_size":0,"ondisk_log_size":8342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34152448,"num_objects":8338,"num_object_clones":0,"num_object_copies":16676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4826,"num_write_kb":19304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4256},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.549401+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.549401+0000","last_peered":"2024-09-15T21:21:14.549401+0000","last_clean":"2024-09-15T21:21:14.549401+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.549401+0000","last_undegraded":"2024-09-15T21:21:14.549401+0000","last_fullsized":"2024-09-15T21:21:14.549401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363284+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.363284+0000","last_peered":"2024-09-15T21:21:14.363284+0000","last_clean":"2024-09-15T21:21:14.363284+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.363284+0000","last_undegraded":"2024-09-15T21:21:14.363284+0000","last_fullsized":"2024-09-15T21:21:14.363284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8525","reported_seq":8659,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.710554+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:25.710554+0000","last_peered":"2024-09-15T21:23:25.710554+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:25.710554+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8525,"log_dups_size":0,"ondisk_log_size":8525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34870214,"num_objects":8515,"num_object_clones":0,"num_object_copies":17030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5229,"num_write_kb":20916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4368},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":558359262,"num_objects":136327,"num_object_clones":0,"num_object_copies":272654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":136327,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":96192,"num_write_kb":384768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3897,"num_bytes_recovered":15953920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1132670976,"data_stored":1132522843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136399,"ondisk_log_size":136399,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680844,"kb_used_data":406204,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93690996,"statfs":{"total":96636764160,"available":95939579904,"internally_reserved":0,"allocated":415952896,"data_stored":415690882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":782688,"kb_used_data":472012,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93589152,"statfs":{"total":96636764160,"available":95835291648,"internally_reserved":0,"allocated":483340288,"data_stored":483074197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,2,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435400,"kb_used_data":227384,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93936440,"statfs":{"total":96636764160,"available":96190914560,"internally_reserved":0,"allocated":232841216,"data_stored":232535382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":212987567},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484323328,"data_stored":484287198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":232488960,"data_stored":232413087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":415858688,"data_stored":415822558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:29.393 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:29.393 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8717", 'reported_seq': 8788, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:25.703674+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:23:25.703674+0000', 'last_peered': '2024-09-15T21:23:25.703674+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:23:25.703674+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 8717, 'log_dups_size': 0, 'ondisk_log_size': 8717, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35704832, 'num_objects': 8717, 'num_object_clones': 0, 'num_object_copies': 17434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8717, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8717, 'num_write_kb': 34868, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4585}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:29.393 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:23:29.394 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8756", 'reported_seq': 8877, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:28.376160+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:23:28.376160+0000', 'last_peered': '2024-09-15T21:23:28.376160+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:23:28.376160+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 8756, 'log_dups_size': 0, 'ondisk_log_size': 8756, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35848192, 'num_objects': 8752, 'num_object_clones': 0, 'num_object_copies': 17504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8752, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6856, 'num_write_kb': 27424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 4485}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:29.394 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:29.394 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8459", 'reported_seq': 8591, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:25.497306+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:23:25.497306+0000', 'last_peered': '2024-09-15T21:23:25.497306+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:23:25.497306+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 8459, 'log_dups_size': 0, 'ondisk_log_size': 8459, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34628067, 'num_objects': 8455, 'num_object_clones': 0, 'num_object_copies': 16910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8455, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5135, 'num_write_kb': 20540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4379}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:29.394 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:29.394 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8524", 'reported_seq': 8656, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:25.703354+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:23:25.703354+0000', 'last_peered': '2024-09-15T21:23:25.703354+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:23:25.703354+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 8524, 'log_dups_size': 0, 'ondisk_log_size': 8524, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34890694, 'num_objects': 8520, 'num_object_clones': 0, 'num_object_copies': 17040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8520, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4835, 'num_write_kb': 19340, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4359}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:29.395 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:29.395 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'8552", 'reported_seq': 8674, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:28.392286+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:23:28.392286+0000', 'last_peered': '2024-09-15T21:23:28.392286+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:23:28.392286+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 8552, 'log_dups_size': 0, 'ondisk_log_size': 8552, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35000803, 'num_objects': 8546, 'num_object_clones': 0, 'num_object_copies': 17092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8546, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6484, 'num_write_kb': 25936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 4459}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:29.395 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:29.395 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8345", 'reported_seq': 8471, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:25.710930+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:23:25.710930+0000', 'last_peered': '2024-09-15T21:23:25.710930+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:23:25.710930+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 8345, 'log_dups_size': 0, 'ondisk_log_size': 8345, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34148352, 'num_objects': 8337, 'num_object_clones': 0, 'num_object_copies': 16674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8337, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5018, 'num_write_kb': 20072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4317}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:29.396 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:29.396 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8598", 'reported_seq': 8712, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:28.281505+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:23:28.281505+0000', 'last_peered': '2024-09-15T21:23:28.281505+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:23:28.281505+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 8598, 'log_dups_size': 0, 'ondisk_log_size': 8598, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35197411, 'num_objects': 8594, 'num_object_clones': 0, 'num_object_copies': 17188, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8594, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6457, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 4504}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:29.396 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:29.396 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8619", 'reported_seq': 16530, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:23:28.429652+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:23:28.429652+0000', 'last_peered': '2024-09-15T21:23:28.429652+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:23:28.429652+0000', 'last_undegraded': '2024-09-15T21:23:28.429652+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 8619, 'log_dups_size': 0, 'ondisk_log_size': 8619, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35278848, 'num_objects': 8613, 'num_object_clones': 0, 'num_object_copies': 17226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 268, 'num_objects_unfound': 0, 'num_objects_dirty': 8613, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6312, 'num_write_kb': 25248, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3896, 'num_bytes_recovered': 15949824, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 268}, {'shards': '2', 'objects': 8345}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:29.396 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:29.396 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8489", 'reported_seq': 8618, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:25.627516+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:23:25.627516+0000', 'last_peered': '2024-09-15T21:23:25.627516+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:23:25.627516+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 8489, 'log_dups_size': 0, 'ondisk_log_size': 8489, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34762752, 'num_objects': 8487, 'num_object_clones': 0, 'num_object_copies': 16974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8487, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5433, 'num_write_kb': 21732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4393}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:29.396 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:29.396 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8443", 'reported_seq': 8570, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:25.744029+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:23:25.744029+0000', 'last_peered': '2024-09-15T21:23:25.744029+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:23:25.744029+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 8443, 'log_dups_size': 0, 'ondisk_log_size': 8443, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34549760, 'num_objects': 8435, 'num_object_clones': 0, 'num_object_copies': 16870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8435, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5352, 'num_write_kb': 21408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4296}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:29.397 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:29.397 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8342", 'reported_seq': 8473, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:25.727140+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:23:25.727140+0000', 'last_peered': '2024-09-15T21:23:25.727140+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:23:25.727140+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 8342, 'log_dups_size': 0, 'ondisk_log_size': 8342, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34152448, 'num_objects': 8338, 'num_object_clones': 0, 'num_object_copies': 16676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8338, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4826, 'num_write_kb': 19304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4256}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:29.397 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:23:29.397 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8525", 'reported_seq': 8659, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:25.710554+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:23:25.710554+0000', 'last_peered': '2024-09-15T21:23:25.710554+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:23:25.710554+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 8525, 'log_dups_size': 0, 'ondisk_log_size': 8525, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34870214, 'num_objects': 8515, 'num_object_clones': 0, 'num_object_copies': 17030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8515, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5229, 'num_write_kb': 20916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4368}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:29.397 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:29.757 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 178 20 137747 137727 3.02198 1.72266 0.0336634 0.0258435 2024-09-15T21:23:30.093 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:30.093 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/272658 objects degraded (16.618%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":261,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":136329,"data_bytes":558359281,"bytes_used":1944506368,"bytes_avail":287965786112,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":272658,"degraded_ratio":0.16617520850295975,"misplaced_objects":268,"misplaced_total":272658,"misplaced_ratio":0.00098291632741382968,"recovering_objects_per_sec":50,"recovering_bytes_per_sec":208477,"recovering_keys_per_sec":0,"num_objects_recovered":611,"num_bytes_recovered":2502656,"num_keys_recovered":0,"write_bytes_sec":1920308,"read_op_per_sec":0,"write_op_per_sec":468},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:30.122 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:30.122 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:30.685 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:30.685 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:30.713 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-15T21:23:28.738551+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":558359281,"num_objects":136329,"num_object_clones":0,"num_object_copies":272658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":136329,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":96194,"num_write_kb":384770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":15953939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136402,"ondisk_log_size":136402,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898932,"kb_used_data":1105600,"kb_used_omap":12,"kb_used_meta":793267,"kb_avail":281216588,"statfs":{"total":289910292480,"available":287965786112,"internally_reserved":0,"allocated":1132134400,"data_stored":1131300461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":812305411},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,1,6,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23052288,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-611,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5628,"num_write_kb":22512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":611,"num_bytes_recovered":2502656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004469"},"pg_stats":[{"pgid":"2.f","version":"43'8717","reported_seq":8788,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.703674+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:25.703674+0000","last_peered":"2024-09-15T21:23:25.703674+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:25.703674+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8717,"log_dups_size":0,"ondisk_log_size":8717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35704832,"num_objects":8717,"num_object_clones":0,"num_object_copies":17434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8717,"num_write_kb":34868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4585},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8595","reported_seq":8696,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.426270+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:28.426270+0000","last_peered":"2024-09-15T21:23:28.426270+0000","last_clean":"2024-09-15T21:23:28.426270+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:28.426270+0000","last_undegraded":"2024-09-15T21:23:28.426270+0000","last_fullsized":"2024-09-15T21:23:28.426270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8595,"log_dups_size":0,"ondisk_log_size":8595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35181510,"num_objects":8591,"num_object_clones":0,"num_object_copies":17182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8591,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6876,"num_write_kb":27504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8756","reported_seq":8877,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.376160+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:28.376160+0000","last_peered":"2024-09-15T21:23:28.376160+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:28.376160+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8756,"log_dups_size":0,"ondisk_log_size":8756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6856,"num_write_kb":27424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4485}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8415","reported_seq":8479,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.421422+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:28.421422+0000","last_peered":"2024-09-15T21:23:28.421422+0000","last_clean":"2024-09-15T21:23:28.421422+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:28.421422+0000","last_undegraded":"2024-09-15T21:23:28.421422+0000","last_fullsized":"2024-09-15T21:23:28.421422+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8415,"log_dups_size":0,"ondisk_log_size":8415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8415,"num_write_kb":33660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8412","reported_seq":8540,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:25.652182+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:25.652182+0000","last_peered":"2024-09-15T21:23:25.652182+0000","last_clean":"2024-09-15T21:23:25.652182+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:25.652182+0000","last_undegraded":"2024-09-15T21:23:25.652182+0000","last_fullsized":"2024-09-15T21:23:25.652182+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8412,"log_dups_size":0,"ondisk_log_size":8412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34427363,"num_objects":8406,"num_object_clones":0,"num_object_copies":16812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4867,"num_write_kb":19468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8459","reported_seq":8591,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.497306+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:25.497306+0000","last_peered":"2024-09-15T21:23:25.497306+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:25.497306+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8459,"log_dups_size":0,"ondisk_log_size":8459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34628067,"num_objects":8455,"num_object_clones":0,"num_object_copies":16910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8455,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5135,"num_write_kb":20540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4379},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8524","reported_seq":8656,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.703354+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:25.703354+0000","last_peered":"2024-09-15T21:23:25.703354+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:25.703354+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8524,"log_dups_size":0,"ondisk_log_size":8524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34890694,"num_objects":8520,"num_object_clones":0,"num_object_copies":17040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4835,"num_write_kb":19340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4359},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8608","reported_seq":8732,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:25.640076+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:25.640076+0000","last_peered":"2024-09-15T21:23:25.640076+0000","last_clean":"2024-09-15T21:23:25.640076+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:25.640076+0000","last_undegraded":"2024-09-15T21:23:25.640076+0000","last_fullsized":"2024-09-15T21:23:25.640076+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8608,"log_dups_size":0,"ondisk_log_size":8608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35250176,"num_objects":8606,"num_object_clones":0,"num_object_copies":17212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5380,"num_write_kb":21520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8552","reported_seq":8674,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.392286+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:28.392286+0000","last_peered":"2024-09-15T21:23:28.392286+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:28.392286+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8552,"log_dups_size":0,"ondisk_log_size":8552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35000803,"num_objects":8546,"num_object_clones":0,"num_object_copies":17092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8546,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6484,"num_write_kb":25936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4459}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8345","reported_seq":8471,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.710930+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:25.710930+0000","last_peered":"2024-09-15T21:23:25.710930+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:25.710930+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8345,"log_dups_size":0,"ondisk_log_size":8345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34148352,"num_objects":8337,"num_object_clones":0,"num_object_copies":16674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5018,"num_write_kb":20072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4317},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8598","reported_seq":8712,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.281505+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:28.281505+0000","last_peered":"2024-09-15T21:23:28.281505+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:28.281505+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8598,"log_dups_size":0,"ondisk_log_size":8598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35197411,"num_objects":8594,"num_object_clones":0,"num_object_copies":17188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8594,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6457,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4504}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.839588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.839588+0000","last_peered":"2024-09-15T21:21:14.839588+0000","last_clean":"2024-09-15T21:21:14.839588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.839588+0000","last_undegraded":"2024-09-15T21:21:14.839588+0000","last_fullsized":"2024-09-15T21:21:14.839588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8619","reported_seq":16530,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:28.429652+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:28.429652+0000","last_peered":"2024-09-15T21:23:28.429652+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:28.429652+0000","last_undegraded":"2024-09-15T21:23:28.429652+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8619,"log_dups_size":0,"ondisk_log_size":8619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35278848,"num_objects":8613,"num_object_clones":0,"num_object_copies":17226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":8613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6312,"num_write_kb":25248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3896,"num_bytes_recovered":15949824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":268},{"shards":"2","objects":8345}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8489","reported_seq":8618,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.627516+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:25.627516+0000","last_peered":"2024-09-15T21:23:25.627516+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:25.627516+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8489,"log_dups_size":0,"ondisk_log_size":8489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34762752,"num_objects":8487,"num_object_clones":0,"num_object_copies":16974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5433,"num_write_kb":21732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4393},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.537989+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.537989+0000","last_peered":"2024-09-15T21:21:14.537989+0000","last_clean":"2024-09-15T21:21:14.537989+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.537989+0000","last_undegraded":"2024-09-15T21:21:14.537989+0000","last_fullsized":"2024-09-15T21:21:14.537989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8443","reported_seq":8570,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.744029+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:25.744029+0000","last_peered":"2024-09-15T21:23:25.744029+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:25.744029+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8443,"log_dups_size":0,"ondisk_log_size":8443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34549760,"num_objects":8435,"num_object_clones":0,"num_object_copies":16870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5352,"num_write_kb":21408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4296},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8342","reported_seq":8473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.727140+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:25.727140+0000","last_peered":"2024-09-15T21:23:25.727140+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:25.727140+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8342,"log_dups_size":0,"ondisk_log_size":8342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34152448,"num_objects":8338,"num_object_clones":0,"num_object_copies":16676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4826,"num_write_kb":19304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4256},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.549401+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.549401+0000","last_peered":"2024-09-15T21:21:14.549401+0000","last_clean":"2024-09-15T21:21:14.549401+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.549401+0000","last_undegraded":"2024-09-15T21:21:14.549401+0000","last_fullsized":"2024-09-15T21:21:14.549401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363284+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.363284+0000","last_peered":"2024-09-15T21:21:14.363284+0000","last_clean":"2024-09-15T21:21:14.363284+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.363284+0000","last_undegraded":"2024-09-15T21:21:14.363284+0000","last_fullsized":"2024-09-15T21:21:14.363284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8525","reported_seq":8659,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:25.710554+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:25.710554+0000","last_peered":"2024-09-15T21:23:25.710554+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:25.710554+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8525,"log_dups_size":0,"ondisk_log_size":8525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34870214,"num_objects":8515,"num_object_clones":0,"num_object_copies":17030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5229,"num_write_kb":20916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4368},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":558359262,"num_objects":136327,"num_object_clones":0,"num_object_copies":272654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":136327,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":96192,"num_write_kb":384768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3897,"num_bytes_recovered":15953920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1132670976,"data_stored":1132522843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136399,"ondisk_log_size":136399,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680844,"kb_used_data":406204,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93690996,"statfs":{"total":96636764160,"available":95939579904,"internally_reserved":0,"allocated":415952896,"data_stored":415690882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":782688,"kb_used_data":472012,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93589152,"statfs":{"total":96636764160,"available":95835291648,"internally_reserved":0,"allocated":483340288,"data_stored":483074197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,2,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435400,"kb_used_data":227384,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93936440,"statfs":{"total":96636764160,"available":96190914560,"internally_reserved":0,"allocated":232841216,"data_stored":232535382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":212987567},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484323328,"data_stored":484287198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":232488960,"data_stored":232413087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":415858688,"data_stored":415822558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:30.765 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 179 20 138194 138174 3.01485 1.74609 0.0228516 0.0259066 2024-09-15T21:23:31.758 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:2024-09-15T21:23:31.755842+0000 min lat: 0.00235499 max lat: 2.60418 avg lat: 0.0259574 2024-09-15T21:23:31.758 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:23:31.758 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 180 20 138690 138670 3.00887 1.9375 0.0247186 0.0259574 2024-09-15T21:23:32.757 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 181 20 139189 139169 3.00301 1.94922 0.0172335 0.0260078 2024-09-15T21:23:33.715 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:33.758 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 182 20 139692 139672 2.99731 1.96484 0.0342719 0.0260585 2024-09-15T21:23:34.257 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:34.258 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:34.286 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-15T21:23:32.739641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":564400881,"num_objects":137804,"num_object_clones":0,"num_object_copies":275608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":137804,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":97669,"num_write_kb":390670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":15953939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137877,"ondisk_log_size":137877,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1908316,"kb_used_data":1119148,"kb_used_omap":12,"kb_used_meta":789107,"kb_avail":281207204,"statfs":{"total":289910292480,"available":287956176896,"internally_reserved":0,"allocated":1146007552,"data_stored":1145173613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":808045571},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,2,4,8,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25415680,"num_objects":6205,"num_object_clones":0,"num_object_copies":12410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-417,"num_objects_unfound":0,"num_objects_dirty":6205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6205,"num_write_kb":24820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":1708032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004135"},"pg_stats":[{"pgid":"2.f","version":"43'8874","reported_seq":8945,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.678156+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:30.678156+0000","last_peered":"2024-09-15T21:23:30.678156+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:30.678156+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8874,"log_dups_size":0,"ondisk_log_size":8874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36347904,"num_objects":8874,"num_object_clones":0,"num_object_copies":17748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8874,"num_write_kb":35496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4742},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8595","reported_seq":8696,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.426270+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:28.426270+0000","last_peered":"2024-09-15T21:23:28.426270+0000","last_clean":"2024-09-15T21:23:28.426270+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:28.426270+0000","last_undegraded":"2024-09-15T21:23:28.426270+0000","last_fullsized":"2024-09-15T21:23:28.426270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8595,"log_dups_size":0,"ondisk_log_size":8595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35181510,"num_objects":8591,"num_object_clones":0,"num_object_copies":17182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8591,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":6876,"num_write_kb":27504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8756","reported_seq":8877,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.376160+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:28.376160+0000","last_peered":"2024-09-15T21:23:28.376160+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:28.376160+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8756,"log_dups_size":0,"ondisk_log_size":8756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6856,"num_write_kb":27424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4485}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8415","reported_seq":8479,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:28.421422+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:28.421422+0000","last_peered":"2024-09-15T21:23:28.421422+0000","last_clean":"2024-09-15T21:23:28.421422+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:28.421422+0000","last_undegraded":"2024-09-15T21:23:28.421422+0000","last_fullsized":"2024-09-15T21:23:28.421422+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8415,"log_dups_size":0,"ondisk_log_size":8415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8415,"num_write_kb":33660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8542","reported_seq":8670,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:30.635764+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:30.635764+0000","last_peered":"2024-09-15T21:23:30.635764+0000","last_clean":"2024-09-15T21:23:30.635764+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:30.635764+0000","last_undegraded":"2024-09-15T21:23:30.635764+0000","last_fullsized":"2024-09-15T21:23:30.635764+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8542,"log_dups_size":0,"ondisk_log_size":8542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959843,"num_objects":8536,"num_object_clones":0,"num_object_copies":17072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4997,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8593","reported_seq":8725,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.727075+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:30.727075+0000","last_peered":"2024-09-15T21:23:30.727075+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:30.727075+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8593,"log_dups_size":0,"ondisk_log_size":8593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35176931,"num_objects":8589,"num_object_clones":0,"num_object_copies":17178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8589,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5269,"num_write_kb":21076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4513},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8671","reported_seq":8803,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.729803+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:30.729803+0000","last_peered":"2024-09-15T21:23:30.729803+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:30.729803+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8671,"log_dups_size":0,"ondisk_log_size":8671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35492806,"num_objects":8667,"num_object_clones":0,"num_object_copies":17334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4982,"num_write_kb":19928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4506},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8770","reported_seq":8894,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:30.750004+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:30.750004+0000","last_peered":"2024-09-15T21:23:30.750004+0000","last_clean":"2024-09-15T21:23:30.750004+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:30.750004+0000","last_undegraded":"2024-09-15T21:23:30.750004+0000","last_fullsized":"2024-09-15T21:23:30.750004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8770,"log_dups_size":0,"ondisk_log_size":8770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5542,"num_write_kb":22168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8552","reported_seq":8674,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.392286+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:28.392286+0000","last_peered":"2024-09-15T21:23:28.392286+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:28.392286+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8552,"log_dups_size":0,"ondisk_log_size":8552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35000803,"num_objects":8546,"num_object_clones":0,"num_object_copies":17092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8546,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6484,"num_write_kb":25936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4459}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8495","reported_seq":8621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.644430+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:30.644430+0000","last_peered":"2024-09-15T21:23:30.644430+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:30.644430+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8495,"log_dups_size":0,"ondisk_log_size":8495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34762752,"num_objects":8487,"num_object_clones":0,"num_object_copies":16974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5168,"num_write_kb":20672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4467},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8598","reported_seq":8712,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:28.281505+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:28.281505+0000","last_peered":"2024-09-15T21:23:28.281505+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:28.281505+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8598,"log_dups_size":0,"ondisk_log_size":8598,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35197411,"num_objects":8594,"num_object_clones":0,"num_object_copies":17188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8594,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6457,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4504}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.839588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.839588+0000","last_peered":"2024-09-15T21:21:14.839588+0000","last_clean":"2024-09-15T21:21:14.839588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.839588+0000","last_undegraded":"2024-09-15T21:21:14.839588+0000","last_fullsized":"2024-09-15T21:21:14.839588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8619","reported_seq":16530,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:28.429652+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:28.429652+0000","last_peered":"2024-09-15T21:23:28.429652+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:28.429652+0000","last_undegraded":"2024-09-15T21:23:28.429652+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8619,"log_dups_size":0,"ondisk_log_size":8619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35278848,"num_objects":8613,"num_object_clones":0,"num_object_copies":17226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":8613,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6312,"num_write_kb":25248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3896,"num_bytes_recovered":15949824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":268},{"shards":"2","objects":8345}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8636","reported_seq":8762,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.720999+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:30.720999+0000","last_peered":"2024-09-15T21:23:30.720999+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:30.720999+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8636,"log_dups_size":0,"ondisk_log_size":8636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35364864,"num_objects":8634,"num_object_clones":0,"num_object_copies":17268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5580,"num_write_kb":22320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4540},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.537989+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.537989+0000","last_peered":"2024-09-15T21:21:14.537989+0000","last_clean":"2024-09-15T21:21:14.537989+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.537989+0000","last_undegraded":"2024-09-15T21:21:14.537989+0000","last_fullsized":"2024-09-15T21:21:14.537989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8577","reported_seq":8705,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.760796+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:30.760796+0000","last_peered":"2024-09-15T21:23:30.760796+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:30.760796+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8577,"log_dups_size":0,"ondisk_log_size":8577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35098624,"num_objects":8569,"num_object_clones":0,"num_object_copies":17138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5486,"num_write_kb":21944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4430},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8482","reported_seq":8613,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.726600+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:30.726600+0000","last_peered":"2024-09-15T21:23:30.726600+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:30.726600+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8482,"log_dups_size":0,"ondisk_log_size":8482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34725888,"num_objects":8478,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4966,"num_write_kb":19864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4396},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.549401+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.549401+0000","last_peered":"2024-09-15T21:21:14.549401+0000","last_clean":"2024-09-15T21:21:14.549401+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.549401+0000","last_undegraded":"2024-09-15T21:21:14.549401+0000","last_fullsized":"2024-09-15T21:21:14.549401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363284+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.363284+0000","last_peered":"2024-09-15T21:21:14.363284+0000","last_clean":"2024-09-15T21:21:14.363284+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.363284+0000","last_undegraded":"2024-09-15T21:21:14.363284+0000","last_fullsized":"2024-09-15T21:21:14.363284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8699","reported_seq":8832,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.782849+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:30.782849+0000","last_peered":"2024-09-15T21:23:30.782849+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:30.782849+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8699,"log_dups_size":0,"ondisk_log_size":8699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35582918,"num_objects":8689,"num_object_clones":0,"num_object_copies":17378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8689,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5403,"num_write_kb":21612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4542},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":564400862,"num_objects":137802,"num_object_clones":0,"num_object_copies":275604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":268,"num_objects_unfound":0,"num_objects_dirty":137802,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":97667,"num_write_kb":390668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3897,"num_bytes_recovered":15953920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1148153856,"data_stored":1148005723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137874,"ondisk_log_size":137874,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680844,"kb_used_data":406204,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93690996,"statfs":{"total":96636764160,"available":95939579904,"internally_reserved":0,"allocated":415952896,"data_stored":415690882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":781648,"kb_used_data":479296,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93590192,"statfs":{"total":96636764160,"available":95836356608,"internally_reserved":0,"allocated":490799104,"data_stored":490533013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309587627},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,0,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445824,"kb_used_data":233648,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93926016,"statfs":{"total":96636764160,"available":96180240384,"internally_reserved":0,"allocated":239255552,"data_stored":238949718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":491696128,"data_stored":491659998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":240599040,"data_stored":240523167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":415858688,"data_stored":415822558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:34.287 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:34.758 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 183 20 140121 140101 2.99008 1.67578 0.0459406 0.0261198 2024-09-15T21:23:34.889 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:35.066 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:35.067 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":170,"stamp":"2024-09-15T21:23:34.869769+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":568181489,"num_objects":138727,"num_object_clones":0,"num_object_copies":277454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":56,"num_objects_unfound":0,"num_objects_dirty":138727,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":98592,"num_write_kb":394362,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4110,"num_bytes_recovered":16822291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138800,"ondisk_log_size":138800,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1900720,"kb_used_data":1124032,"kb_used_omap":12,"kb_used_meta":776627,"kb_avail":281214800,"statfs":{"total":289910292480,"available":287963955200,"internally_reserved":0,"allocated":1151008768,"data_stored":1150174829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":795266051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,5,2,6,10,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19632128,"num_objects":4793,"num_object_clones":0,"num_object_copies":9586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-419,"num_objects_unfound":0,"num_objects_dirty":4793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4793,"num_write_kb":19172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":419,"num_bytes_recovered":1716224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.132403"},"pg_stats":[{"pgid":"2.f","version":"43'8874","reported_seq":8945,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.678156+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:30.678156+0000","last_peered":"2024-09-15T21:23:30.678156+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:30.678156+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8874,"log_dups_size":0,"ondisk_log_size":8874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36347904,"num_objects":8874,"num_object_clones":0,"num_object_copies":17748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8874,"num_write_kb":35496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4742},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8733","reported_seq":8834,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.358214+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:33.358214+0000","last_peered":"2024-09-15T21:23:33.358214+0000","last_clean":"2024-09-15T21:23:33.358214+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:33.358214+0000","last_undegraded":"2024-09-15T21:23:33.358214+0000","last_fullsized":"2024-09-15T21:23:33.358214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8733,"log_dups_size":0,"ondisk_log_size":8733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35746758,"num_objects":8729,"num_object_clones":0,"num_object_copies":17458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8729,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7014,"num_write_kb":28056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8924","reported_seq":9045,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.354164+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:33.354164+0000","last_peered":"2024-09-15T21:23:33.354164+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:33.354164+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8924,"log_dups_size":0,"ondisk_log_size":8924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36536320,"num_objects":8920,"num_object_clones":0,"num_object_copies":17840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7024,"num_write_kb":28096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4653}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8584","reported_seq":8650,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.377003+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:33.377003+0000","last_peered":"2024-09-15T21:23:33.377003+0000","last_clean":"2024-09-15T21:23:33.377003+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:33.377003+0000","last_undegraded":"2024-09-15T21:23:33.377003+0000","last_fullsized":"2024-09-15T21:23:33.377003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8584,"log_dups_size":0,"ondisk_log_size":8584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35160064,"num_objects":8584,"num_object_clones":0,"num_object_copies":17168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8584,"num_write_kb":34336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8542","reported_seq":8670,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:30.635764+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:30.635764+0000","last_peered":"2024-09-15T21:23:30.635764+0000","last_clean":"2024-09-15T21:23:30.635764+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:30.635764+0000","last_undegraded":"2024-09-15T21:23:30.635764+0000","last_fullsized":"2024-09-15T21:23:30.635764+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8542,"log_dups_size":0,"ondisk_log_size":8542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959843,"num_objects":8536,"num_object_clones":0,"num_object_copies":17072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4997,"num_write_kb":19988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8593","reported_seq":8725,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.727075+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:30.727075+0000","last_peered":"2024-09-15T21:23:30.727075+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:30.727075+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8593,"log_dups_size":0,"ondisk_log_size":8593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35176931,"num_objects":8589,"num_object_clones":0,"num_object_copies":17178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8589,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5269,"num_write_kb":21076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4513},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8671","reported_seq":8803,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.729803+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:30.729803+0000","last_peered":"2024-09-15T21:23:30.729803+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:30.729803+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8671,"log_dups_size":0,"ondisk_log_size":8671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35492806,"num_objects":8667,"num_object_clones":0,"num_object_copies":17334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4982,"num_write_kb":19928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4506},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8770","reported_seq":8894,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:30.750004+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:30.750004+0000","last_peered":"2024-09-15T21:23:30.750004+0000","last_clean":"2024-09-15T21:23:30.750004+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:30.750004+0000","last_undegraded":"2024-09-15T21:23:30.750004+0000","last_fullsized":"2024-09-15T21:23:30.750004+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8770,"log_dups_size":0,"ondisk_log_size":8770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5542,"num_write_kb":22168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8699","reported_seq":8821,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.438190+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:33.438190+0000","last_peered":"2024-09-15T21:23:33.438190+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:33.438190+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8699,"log_dups_size":0,"ondisk_log_size":8699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35602915,"num_objects":8693,"num_object_clones":0,"num_object_copies":17386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8693,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6631,"num_write_kb":26524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4606}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8495","reported_seq":8621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.644430+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:30.644430+0000","last_peered":"2024-09-15T21:23:30.644430+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:30.644430+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8495,"log_dups_size":0,"ondisk_log_size":8495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34762752,"num_objects":8487,"num_object_clones":0,"num_object_copies":16974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5168,"num_write_kb":20672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4467},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.601350+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:21:14.601350+0000","last_peered":"2024-09-15T21:21:14.601350+0000","last_clean":"2024-09-15T21:21:14.601350+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:21:14.601350+0000","last_undegraded":"2024-09-15T21:21:14.601350+0000","last_fullsized":"2024-09-15T21:21:14.601350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8756","reported_seq":8870,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.396465+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:33.396465+0000","last_peered":"2024-09-15T21:23:33.396465+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:33.396465+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8756,"log_dups_size":0,"ondisk_log_size":8756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35844579,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6615,"num_write_kb":26460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4662}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.839588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.839588+0000","last_peered":"2024-09-15T21:21:14.839588+0000","last_clean":"2024-09-15T21:21:14.839588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.839588+0000","last_undegraded":"2024-09-15T21:21:14.839588+0000","last_fullsized":"2024-09-15T21:21:14.839588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8762","reported_seq":17097,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:33.430845+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:33.430845+0000","last_peered":"2024-09-15T21:23:33.430845+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:33.430845+0000","last_undegraded":"2024-09-15T21:23:33.430845+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8762,"log_dups_size":0,"ondisk_log_size":8762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35864576,"num_objects":8756,"num_object_clones":0,"num_object_copies":17512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":56,"num_objects_unfound":0,"num_objects_dirty":8756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6455,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4108,"num_bytes_recovered":16818176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":56},{"shards":"2","objects":8700}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8636","reported_seq":8762,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.720999+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:30.720999+0000","last_peered":"2024-09-15T21:23:30.720999+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:30.720999+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8636,"log_dups_size":0,"ondisk_log_size":8636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35364864,"num_objects":8634,"num_object_clones":0,"num_object_copies":17268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5580,"num_write_kb":22320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4540},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.537989+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.537989+0000","last_peered":"2024-09-15T21:21:14.537989+0000","last_clean":"2024-09-15T21:21:14.537989+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.537989+0000","last_undegraded":"2024-09-15T21:21:14.537989+0000","last_fullsized":"2024-09-15T21:21:14.537989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8577","reported_seq":8705,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.760796+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:30.760796+0000","last_peered":"2024-09-15T21:23:30.760796+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:30.760796+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8577,"log_dups_size":0,"ondisk_log_size":8577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35098624,"num_objects":8569,"num_object_clones":0,"num_object_copies":17138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5486,"num_write_kb":21944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4430},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8482","reported_seq":8613,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.726600+0000","last_change":"2024-09-15T21:21:14.813340+0000","last_active":"2024-09-15T21:23:30.726600+0000","last_peered":"2024-09-15T21:23:30.726600+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:30.726600+0000","last_undegraded":"2024-09-15T21:21:14.454492+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8482,"log_dups_size":0,"ondisk_log_size":8482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34725888,"num_objects":8478,"num_object_clones":0,"num_object_copies":16956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4084,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4966,"num_write_kb":19864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4396},{"shards":"0,2","objects":4082}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.549401+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.549401+0000","last_peered":"2024-09-15T21:21:14.549401+0000","last_clean":"2024-09-15T21:21:14.549401+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.549401+0000","last_undegraded":"2024-09-15T21:21:14.549401+0000","last_fullsized":"2024-09-15T21:21:14.549401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363284+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.363284+0000","last_peered":"2024-09-15T21:21:14.363284+0000","last_clean":"2024-09-15T21:21:14.363284+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.363284+0000","last_undegraded":"2024-09-15T21:21:14.363284+0000","last_fullsized":"2024-09-15T21:21:14.363284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8699","reported_seq":8832,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:30.782849+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:30.782849+0000","last_peered":"2024-09-15T21:23:30.782849+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:30.782849+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8699,"log_dups_size":0,"ondisk_log_size":8699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35582918,"num_objects":8689,"num_object_clones":0,"num_object_copies":17378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8689,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5403,"num_write_kb":21612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4542},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":568181470,"num_objects":138725,"num_object_clones":0,"num_object_copies":277450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":45309,"num_objects_misplaced":56,"num_objects_unfound":0,"num_objects_dirty":138725,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":98590,"num_write_kb":394360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4109,"num_bytes_recovered":16822272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1153167360,"data_stored":1153019227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138797,"ondisk_log_size":138797,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673248,"kb_used_data":411088,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93698592,"statfs":{"total":96636764160,"available":95947358208,"internally_reserved":0,"allocated":420954112,"data_stored":420692098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":781648,"kb_used_data":479296,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93590192,"statfs":{"total":96636764160,"available":95836356608,"internally_reserved":0,"allocated":490799104,"data_stored":490533013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309587627},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,0,3,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445824,"kb_used_data":233648,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93926016,"statfs":{"total":96636764160,"available":96180240384,"internally_reserved":0,"allocated":239255552,"data_stored":238949718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":491696128,"data_stored":491659998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":240599040,"data_stored":240523167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":420872192,"data_stored":420836062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:35.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:35.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "43'8874", 'reported_seq': 8945, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:30.678156+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:23:30.678156+0000', 'last_peered': '2024-09-15T21:23:30.678156+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:23:30.678156+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 8874, 'log_dups_size': 0, 'ondisk_log_size': 8874, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36347904, 'num_objects': 8874, 'num_object_clones': 0, 'num_object_copies': 17748, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8874, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8874, 'num_write_kb': 35496, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4742}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:35.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:23:35.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "43'8924", 'reported_seq': 9045, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.354164+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:23:33.354164+0000', 'last_peered': '2024-09-15T21:23:33.354164+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:23:33.354164+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 8924, 'log_dups_size': 0, 'ondisk_log_size': 8924, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36536320, 'num_objects': 8920, 'num_object_clones': 0, 'num_object_copies': 17840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7024, 'num_write_kb': 28096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 4653}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:35.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:35.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "43'8593", 'reported_seq': 8725, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:30.727075+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:23:30.727075+0000', 'last_peered': '2024-09-15T21:23:30.727075+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:23:30.727075+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 8593, 'log_dups_size': 0, 'ondisk_log_size': 8593, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35176931, 'num_objects': 8589, 'num_object_clones': 0, 'num_object_copies': 17178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8589, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5269, 'num_write_kb': 21076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4513}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:35.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:35.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "43'8671", 'reported_seq': 8803, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:30.729803+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:23:30.729803+0000', 'last_peered': '2024-09-15T21:23:30.729803+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:23:30.729803+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 8671, 'log_dups_size': 0, 'ondisk_log_size': 8671, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35492806, 'num_objects': 8667, 'num_object_clones': 0, 'num_object_copies': 17334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8667, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4982, 'num_write_kb': 19928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4506}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:35.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:35.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "43'8699", 'reported_seq': 8821, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.438190+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:23:33.438190+0000', 'last_peered': '2024-09-15T21:23:33.438190+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:23:33.438190+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 8699, 'log_dups_size': 0, 'ondisk_log_size': 8699, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35602915, 'num_objects': 8693, 'num_object_clones': 0, 'num_object_copies': 17386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8693, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6631, 'num_write_kb': 26524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 4606}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:35.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:35.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "43'8495", 'reported_seq': 8621, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:30.644430+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:23:30.644430+0000', 'last_peered': '2024-09-15T21:23:30.644430+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:23:30.644430+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 8495, 'log_dups_size': 0, 'ondisk_log_size': 8495, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34762752, 'num_objects': 8487, 'num_object_clones': 0, 'num_object_copies': 16974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8487, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5168, 'num_write_kb': 20672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4467}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:35.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:35.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "43'8756", 'reported_seq': 8870, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:33.396465+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:23:33.396465+0000', 'last_peered': '2024-09-15T21:23:33.396465+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:23:33.396465+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 8756, 'log_dups_size': 0, 'ondisk_log_size': 8756, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35844579, 'num_objects': 8752, 'num_object_clones': 0, 'num_object_copies': 17504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8752, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6615, 'num_write_kb': 26460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 4662}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:35.069 INFO:tasks.lost_unfound.ceph_manager:PG 2.0 is not active+clean 2024-09-15T21:23:35.069 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.0', 'version': "43'8762", 'reported_seq': 17097, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:23:33.430845+0000', 'last_change': '2024-09-15T21:21:14.759353+0000', 'last_active': '2024-09-15T21:23:33.430845+0000', 'last_peered': '2024-09-15T21:23:33.430845+0000', 'last_clean': '2024-09-15T21:21:12.335143+0000', 'last_became_active': '2024-09-15T21:21:14.732411+0000', 'last_became_peered': '2024-09-15T21:21:14.732411+0000', 'last_unstale': '2024-09-15T21:23:33.430845+0000', 'last_undegraded': '2024-09-15T21:23:33.430845+0000', 'last_fullsized': '2024-09-15T21:21:14.350874+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2342", 'last_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:48.647896+0000', 'objects_scrubbed': 2301, 'log_size': 8762, 'log_dups_size': 0, 'ondisk_log_size': 8762, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 744, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35864576, 'num_objects': 8756, 'num_object_clones': 0, 'num_object_copies': 17512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 56, 'num_objects_unfound': 0, 'num_objects_dirty': 8756, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6455, 'num_write_kb': 25820, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4108, 'num_bytes_recovered': 16818176, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 56}, {'shards': '2', 'objects': 8700}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:35.070 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:35.070 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "43'8636", 'reported_seq': 8762, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:30.720999+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:23:30.720999+0000', 'last_peered': '2024-09-15T21:23:30.720999+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:23:30.720999+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 8636, 'log_dups_size': 0, 'ondisk_log_size': 8636, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35364864, 'num_objects': 8634, 'num_object_clones': 0, 'num_object_copies': 17268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8634, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5580, 'num_write_kb': 22320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4540}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:35.070 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:35.070 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "43'8577", 'reported_seq': 8705, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:30.760796+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:23:30.760796+0000', 'last_peered': '2024-09-15T21:23:30.760796+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:23:30.760796+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 8577, 'log_dups_size': 0, 'ondisk_log_size': 8577, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35098624, 'num_objects': 8569, 'num_object_clones': 0, 'num_object_copies': 17138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8569, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5486, 'num_write_kb': 21944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4430}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:35.070 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:35.070 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "43'8482", 'reported_seq': 8613, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:30.726600+0000', 'last_change': '2024-09-15T21:21:14.813340+0000', 'last_active': '2024-09-15T21:23:30.726600+0000', 'last_peered': '2024-09-15T21:23:30.726600+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:23:30.726600+0000', 'last_undegraded': '2024-09-15T21:21:14.454492+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 8482, 'log_dups_size': 0, 'ondisk_log_size': 8482, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34725888, 'num_objects': 8478, 'num_object_clones': 0, 'num_object_copies': 16956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4084, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8478, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4966, 'num_write_kb': 19864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4396}, {'shards': '0,2', 'objects': 4082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:35.070 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:23:35.070 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "43'8699", 'reported_seq': 8832, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:30.782849+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:23:30.782849+0000', 'last_peered': '2024-09-15T21:23:30.782849+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:23:30.782849+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 8699, 'log_dups_size': 0, 'ondisk_log_size': 8699, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35582918, 'num_objects': 8689, 'num_object_clones': 0, 'num_object_copies': 17378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8689, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5403, 'num_write_kb': 21612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4542}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:35.071 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:35.655 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:35.655 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 45309/277454 objects degraded (16.330%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":267,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":138727,"data_bytes":568181489,"bytes_used":1946337280,"bytes_avail":287963955200,"bytes_total":289910292480,"degraded_objects":45309,"degraded_total":277454,"degraded_ratio":0.16330274568036504,"misplaced_objects":56,"misplaced_total":277454,"misplaced_ratio":0.00020183525917809796,"recovering_objects_per_sec":52,"recovering_bytes_per_sec":214623,"recovering_keys_per_sec":0,"num_objects_recovered":629,"num_bytes_recovered":2576384,"num_keys_recovered":0,"write_bytes_sec":1924103,"read_op_per_sec":0,"write_op_per_sec":469},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:35.670 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:35.671 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:35.758 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 184 20 140467 140447 2.98118 1.35156 0.00837183 0.0261264 2024-09-15T21:23:36.094 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:36.096 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:36.117 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-15T21:23:35.875541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":573919985,"num_objects":140128,"num_object_clones":0,"num_object_copies":280256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":4099,"num_objects_unfound":0,"num_objects_dirty":140128,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":99993,"num_write_kb":399966,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":16982035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140201,"ondisk_log_size":140201,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1912064,"kb_used_data":1131212,"kb_used_omap":12,"kb_used_meta":780787,"kb_avail":281203456,"statfs":{"total":289910292480,"available":287952338944,"internally_reserved":0,"allocated":1158361088,"data_stored":1157522379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":799525891},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,7,7,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19406848,"num_objects":4738,"num_object_clones":0,"num_object_copies":9476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4084,"num_objects_misplaced":3624,"num_objects_unfound":0,"num_objects_dirty":4738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4738,"num_write_kb":18952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":1875968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.137642"},"pg_stats":[{"pgid":"2.f","version":"44'9004","reported_seq":9077,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.606202+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:35.606202+0000","last_peered":"2024-09-15T21:23:35.606202+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:35.606202+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9004,"log_dups_size":0,"ondisk_log_size":9004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9004,"num_write_kb":36016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4872},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8733","reported_seq":8834,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.358214+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:33.358214+0000","last_peered":"2024-09-15T21:23:33.358214+0000","last_clean":"2024-09-15T21:23:33.358214+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:33.358214+0000","last_undegraded":"2024-09-15T21:23:33.358214+0000","last_fullsized":"2024-09-15T21:23:33.358214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8733,"log_dups_size":0,"ondisk_log_size":8733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35746758,"num_objects":8729,"num_object_clones":0,"num_object_copies":17458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8729,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7014,"num_write_kb":28056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8924","reported_seq":9045,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.354164+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:33.354164+0000","last_peered":"2024-09-15T21:23:33.354164+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:33.354164+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":8924,"log_dups_size":0,"ondisk_log_size":8924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36536320,"num_objects":8920,"num_object_clones":0,"num_object_copies":17840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7024,"num_write_kb":28096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4653}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8584","reported_seq":8650,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:23:33.377003+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:33.377003+0000","last_peered":"2024-09-15T21:23:33.377003+0000","last_clean":"2024-09-15T21:23:33.377003+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:33.377003+0000","last_undegraded":"2024-09-15T21:23:33.377003+0000","last_fullsized":"2024-09-15T21:23:33.377003+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8584,"log_dups_size":0,"ondisk_log_size":8584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35160064,"num_objects":8584,"num_object_clones":0,"num_object_copies":17168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8584,"num_write_kb":34336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'8674","reported_seq":8804,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.653128+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:35.653128+0000","last_peered":"2024-09-15T21:23:35.653128+0000","last_clean":"2024-09-15T21:23:35.653128+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:35.653128+0000","last_undegraded":"2024-09-15T21:23:35.653128+0000","last_fullsized":"2024-09-15T21:23:35.653128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8674,"log_dups_size":0,"ondisk_log_size":8674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35500515,"num_objects":8668,"num_object_clones":0,"num_object_copies":17336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5129,"num_write_kb":20516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'8735","reported_seq":8869,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.668805+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:35.668805+0000","last_peered":"2024-09-15T21:23:35.668805+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:35.668805+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8735,"log_dups_size":0,"ondisk_log_size":8735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758563,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5411,"num_write_kb":21644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4655},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'8818","reported_seq":8952,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.579368+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:35.579368+0000","last_peered":"2024-09-15T21:23:35.579368+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:35.579368+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8818,"log_dups_size":0,"ondisk_log_size":8818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36094918,"num_objects":8814,"num_object_clones":0,"num_object_copies":17628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5129,"num_write_kb":20516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4653},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'8895","reported_seq":9021,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.659269+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:35.659269+0000","last_peered":"2024-09-15T21:23:35.659269+0000","last_clean":"2024-09-15T21:23:35.659269+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:35.659269+0000","last_undegraded":"2024-09-15T21:23:35.659269+0000","last_fullsized":"2024-09-15T21:23:35.659269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8895,"log_dups_size":0,"ondisk_log_size":8895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36425728,"num_objects":8893,"num_object_clones":0,"num_object_copies":17786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5667,"num_write_kb":22668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363286+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:21:14.363286+0000","last_peered":"2024-09-15T21:21:14.363286+0000","last_clean":"2024-09-15T21:21:14.363286+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:21:14.363286+0000","last_undegraded":"2024-09-15T21:21:14.363286+0000","last_fullsized":"2024-09-15T21:21:14.363286+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8699","reported_seq":8821,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.438190+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:33.438190+0000","last_peered":"2024-09-15T21:23:33.438190+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:33.438190+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8699,"log_dups_size":0,"ondisk_log_size":8699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35602915,"num_objects":8693,"num_object_clones":0,"num_object_copies":17386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8693,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6631,"num_write_kb":26524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4606}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.352597+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:21:14.352597+0000","last_peered":"2024-09-15T21:21:14.352597+0000","last_clean":"2024-09-15T21:21:14.352597+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:21:14.352597+0000","last_undegraded":"2024-09-15T21:21:14.352597+0000","last_fullsized":"2024-09-15T21:21:14.352597+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'8630","reported_seq":8758,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.574675+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:35.574675+0000","last_peered":"2024-09-15T21:23:35.574675+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:35.574675+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8630,"log_dups_size":0,"ondisk_log_size":8630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35315712,"num_objects":8622,"num_object_clones":0,"num_object_copies":17244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":21212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4602},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.887834+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:34.887834+0000","last_peered":"2024-09-15T21:23:34.887834+0000","last_clean":"2024-09-15T21:23:34.887834+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:34.887834+0000","last_undegraded":"2024-09-15T21:23:34.887834+0000","last_fullsized":"2024-09-15T21:23:34.887834+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8756","reported_seq":8870,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:33.396465+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:33.396465+0000","last_peered":"2024-09-15T21:23:33.396465+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:33.396465+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8756,"log_dups_size":0,"ondisk_log_size":8756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35844579,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6615,"num_write_kb":26460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4662}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.839588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:21:14.839588+0000","last_peered":"2024-09-15T21:21:14.839588+0000","last_clean":"2024-09-15T21:21:14.839588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:21:14.839588+0000","last_undegraded":"2024-09-15T21:21:14.839588+0000","last_fullsized":"2024-09-15T21:21:14.839588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8762","reported_seq":17097,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:33.430845+0000","last_change":"2024-09-15T21:21:14.759353+0000","last_active":"2024-09-15T21:23:33.430845+0000","last_peered":"2024-09-15T21:23:33.430845+0000","last_clean":"2024-09-15T21:21:12.335143+0000","last_became_active":"2024-09-15T21:21:14.732411+0000","last_became_peered":"2024-09-15T21:21:14.732411+0000","last_unstale":"2024-09-15T21:23:33.430845+0000","last_undegraded":"2024-09-15T21:23:33.430845+0000","last_fullsized":"2024-09-15T21:21:14.350874+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8762,"log_dups_size":0,"ondisk_log_size":8762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35864576,"num_objects":8756,"num_object_clones":0,"num_object_copies":17512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":56,"num_objects_unfound":0,"num_objects_dirty":8756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6455,"num_write_kb":25820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4108,"num_bytes_recovered":16818176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":56},{"shards":"2","objects":8700}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'8779","reported_seq":8910,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.593135+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:35.593135+0000","last_peered":"2024-09-15T21:23:35.593135+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:35.593135+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8779,"log_dups_size":0,"ondisk_log_size":8779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35950592,"num_objects":8777,"num_object_clones":0,"num_object_copies":17554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5723,"num_write_kb":22892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4683},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.537989+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:21:14.537989+0000","last_peered":"2024-09-15T21:21:14.537989+0000","last_clean":"2024-09-15T21:21:14.537989+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:21:14.537989+0000","last_undegraded":"2024-09-15T21:21:14.537989+0000","last_fullsized":"2024-09-15T21:21:14.537989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'8720","reported_seq":8850,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.680309+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:35.680309+0000","last_peered":"2024-09-15T21:23:35.680309+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:35.680309+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8720,"log_dups_size":0,"ondisk_log_size":8720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35684352,"num_objects":8712,"num_object_clones":0,"num_object_copies":17424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5629,"num_write_kb":22516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4573},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.354669+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:21:14.354669+0000","last_peered":"2024-09-15T21:21:14.354669+0000","last_clean":"2024-09-15T21:21:14.354669+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:21:14.354669+0000","last_undegraded":"2024-09-15T21:21:14.354669+0000","last_fullsized":"2024-09-15T21:21:14.354669+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'8631","reported_seq":8843,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:35.791081+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:35.791081+0000","last_peered":"2024-09-15T21:23:35.791081+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:35.791081+0000","last_undegraded":"2024-09-15T21:23:35.791081+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8631,"log_dups_size":0,"ondisk_log_size":8631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35336192,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4043,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5115,"num_write_kb":20460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":159744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4584},{"shards":"0,2","objects":4043}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.549401+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:21:14.549401+0000","last_peered":"2024-09-15T21:21:14.549401+0000","last_clean":"2024-09-15T21:21:14.549401+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:21:14.549401+0000","last_undegraded":"2024-09-15T21:21:14.549401+0000","last_fullsized":"2024-09-15T21:21:14.549401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-15T21:21:14.363284+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:21:14.363284+0000","last_peered":"2024-09-15T21:21:14.363284+0000","last_clean":"2024-09-15T21:21:14.363284+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:21:14.363284+0000","last_undegraded":"2024-09-15T21:21:14.363284+0000","last_fullsized":"2024-09-15T21:21:14.363284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'8854","reported_seq":8990,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.721883+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:35.721883+0000","last_peered":"2024-09-15T21:23:35.721883+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:35.721883+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8854,"log_dups_size":0,"ondisk_log_size":8854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36217798,"num_objects":8844,"num_object_clones":0,"num_object_copies":17688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8844,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5558,"num_write_kb":22232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4697},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":573919966,"num_objects":140126,"num_object_clones":0,"num_object_copies":280252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":4099,"num_objects_unfound":0,"num_objects_dirty":140126,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":99991,"num_write_kb":399964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4148,"num_bytes_recovered":16982016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1160101888,"data_stored":1159953755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140198,"ondisk_log_size":140198,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":673248,"kb_used_data":411088,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93698592,"statfs":{"total":96636764160,"available":95947358208,"internally_reserved":0,"allocated":420954112,"data_stored":420692098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":792992,"kb_used_data":486476,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93578848,"statfs":{"total":96636764160,"available":95824740352,"internally_reserved":0,"allocated":498151424,"data_stored":497880563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445824,"kb_used_data":233648,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93926016,"statfs":{"total":96636764160,"available":96180240384,"internally_reserved":0,"allocated":239255552,"data_stored":238949718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,0,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":498630656,"data_stored":498594526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":240599040,"data_stored":240523167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":420872192,"data_stored":420836062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:36.761 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 185 20 140871 140851 2.97359 1.57812 0.0280463 0.0262651 2024-09-15T21:23:37.758 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 186 20 141363 141343 2.96794 1.92188 0.0281837 0.0263151 2024-09-15T21:23:38.758 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 187 20 141858 141838 2.96241 1.93359 0.0282503 0.0263639 2024-09-15T21:23:39.120 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:39.664 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:39.667 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:39.687 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-15T21:23:38.741167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":577245937,"num_objects":140940,"num_object_clones":0,"num_object_copies":281880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":4043,"num_objects_unfound":0,"num_objects_dirty":140940,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":100805,"num_write_kb":403214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4205,"num_bytes_recovered":17211411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141013,"ondisk_log_size":141013,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1937940,"kb_used_data":1144596,"kb_used_omap":12,"kb_used_meta":793267,"kb_avail":281177580,"statfs":{"total":289910292480,"available":287925841920,"internally_reserved":0,"allocated":1172066304,"data_stored":1171208473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":812305411},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,12,8,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18886656,"num_objects":4611,"num_object_clones":0,"num_object_copies":9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4084,"num_objects_misplaced":3775,"num_objects_unfound":0,"num_objects_dirty":4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4611,"num_write_kb":18444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":1257472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001975"},"pg_stats":[{"pgid":"2.f","version":"44'9004","reported_seq":9077,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.606202+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:35.606202+0000","last_peered":"2024-09-15T21:23:35.606202+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:35.606202+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9004,"log_dups_size":0,"ondisk_log_size":9004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9004,"num_write_kb":36016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4872},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8873","reported_seq":8978,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.429271+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:38.429271+0000","last_peered":"2024-09-15T21:23:38.429271+0000","last_clean":"2024-09-15T21:23:38.429271+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:38.429271+0000","last_undegraded":"2024-09-15T21:23:38.429271+0000","last_fullsized":"2024-09-15T21:23:38.429271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8873,"log_dups_size":0,"ondisk_log_size":8873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36320198,"num_objects":8869,"num_object_clones":0,"num_object_copies":17738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8869,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7154,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9053","reported_seq":9178,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.432584+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:38.432584+0000","last_peered":"2024-09-15T21:23:38.432584+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:38.432584+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9053,"log_dups_size":0,"ondisk_log_size":9053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7153,"num_write_kb":28612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4782}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8727","reported_seq":8797,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.452289+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:38.452289+0000","last_peered":"2024-09-15T21:23:38.452289+0000","last_clean":"2024-09-15T21:23:38.452289+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:38.452289+0000","last_undegraded":"2024-09-15T21:23:38.452289+0000","last_fullsized":"2024-09-15T21:23:38.452289+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8727,"log_dups_size":0,"ondisk_log_size":8727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35745792,"num_objects":8727,"num_object_clones":0,"num_object_copies":17454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8727,"num_write_kb":34908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'8674","reported_seq":8804,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.653128+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:35.653128+0000","last_peered":"2024-09-15T21:23:35.653128+0000","last_clean":"2024-09-15T21:23:35.653128+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:35.653128+0000","last_undegraded":"2024-09-15T21:23:35.653128+0000","last_fullsized":"2024-09-15T21:23:35.653128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8674,"log_dups_size":0,"ondisk_log_size":8674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35500515,"num_objects":8668,"num_object_clones":0,"num_object_copies":17336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5129,"num_write_kb":20516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'8735","reported_seq":8869,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.668805+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:35.668805+0000","last_peered":"2024-09-15T21:23:35.668805+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:35.668805+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8735,"log_dups_size":0,"ondisk_log_size":8735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758563,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5411,"num_write_kb":21644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4655},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'8818","reported_seq":8952,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.579368+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:35.579368+0000","last_peered":"2024-09-15T21:23:35.579368+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:35.579368+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8818,"log_dups_size":0,"ondisk_log_size":8818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36094918,"num_objects":8814,"num_object_clones":0,"num_object_copies":17628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5129,"num_write_kb":20516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4653},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'8895","reported_seq":9021,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.659269+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:35.659269+0000","last_peered":"2024-09-15T21:23:35.659269+0000","last_clean":"2024-09-15T21:23:35.659269+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:35.659269+0000","last_undegraded":"2024-09-15T21:23:35.659269+0000","last_fullsized":"2024-09-15T21:23:35.659269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8895,"log_dups_size":0,"ondisk_log_size":8895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36425728,"num_objects":8893,"num_object_clones":0,"num_object_copies":17786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5667,"num_write_kb":22668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8842","reported_seq":8968,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.443010+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:38.443010+0000","last_peered":"2024-09-15T21:23:38.443010+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:38.443010+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8842,"log_dups_size":0,"ondisk_log_size":8842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36188643,"num_objects":8836,"num_object_clones":0,"num_object_copies":17672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8836,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6774,"num_write_kb":27096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4749}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'8630","reported_seq":8758,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.574675+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:35.574675+0000","last_peered":"2024-09-15T21:23:35.574675+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:35.574675+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8630,"log_dups_size":0,"ondisk_log_size":8630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35315712,"num_objects":8622,"num_object_clones":0,"num_object_copies":17244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":21212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4602},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.887834+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:34.887834+0000","last_peered":"2024-09-15T21:23:34.887834+0000","last_clean":"2024-09-15T21:23:34.887834+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:34.887834+0000","last_undegraded":"2024-09-15T21:23:34.887834+0000","last_fullsized":"2024-09-15T21:23:34.887834+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8874","reported_seq":8992,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.335510+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:38.335510+0000","last_peered":"2024-09-15T21:23:38.335510+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:38.335510+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8874,"log_dups_size":0,"ondisk_log_size":8874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36327907,"num_objects":8870,"num_object_clones":0,"num_object_copies":17740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8870,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6733,"num_write_kb":26932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4780}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8901","reported_seq":17359,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.439916+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:38.439916+0000","last_peered":"2024-09-15T21:23:38.439916+0000","last_clean":"2024-09-15T21:23:38.439916+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:38.439916+0000","last_undegraded":"2024-09-15T21:23:38.439916+0000","last_fullsized":"2024-09-15T21:23:38.439916+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8901,"log_dups_size":0,"ondisk_log_size":8901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6594,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'8779","reported_seq":8910,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.593135+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:35.593135+0000","last_peered":"2024-09-15T21:23:35.593135+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:35.593135+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8779,"log_dups_size":0,"ondisk_log_size":8779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35950592,"num_objects":8777,"num_object_clones":0,"num_object_copies":17554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5723,"num_write_kb":22892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4683},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'8720","reported_seq":8850,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.680309+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:35.680309+0000","last_peered":"2024-09-15T21:23:35.680309+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:35.680309+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8720,"log_dups_size":0,"ondisk_log_size":8720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35684352,"num_objects":8712,"num_object_clones":0,"num_object_copies":17424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5629,"num_write_kb":22516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4573},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'8631","reported_seq":8843,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:35.791081+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:35.791081+0000","last_peered":"2024-09-15T21:23:35.791081+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:35.791081+0000","last_undegraded":"2024-09-15T21:23:35.791081+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8631,"log_dups_size":0,"ondisk_log_size":8631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35336192,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4043,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5115,"num_write_kb":20460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":159744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4584},{"shards":"0,2","objects":4043}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'8854","reported_seq":8990,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.721883+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:35.721883+0000","last_peered":"2024-09-15T21:23:35.721883+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:35.721883+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8854,"log_dups_size":0,"ondisk_log_size":8854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36217798,"num_objects":8844,"num_object_clones":0,"num_object_copies":17688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8844,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5558,"num_write_kb":22232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4697},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":577245918,"num_objects":140938,"num_object_clones":0,"num_object_copies":281876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":4043,"num_objects_unfound":0,"num_objects_dirty":140938,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":100803,"num_write_kb":403212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4204,"num_bytes_recovered":17211392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1172119552,"data_stored":1171971419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141010,"ondisk_log_size":141010,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681784,"kb_used_data":415460,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93690056,"statfs":{"total":96636764160,"available":95938617344,"internally_reserved":0,"allocated":425431040,"data_stored":425159465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":272690857},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":792992,"kb_used_data":486476,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93578848,"statfs":{"total":96636764160,"available":95824740352,"internally_reserved":0,"allocated":498151424,"data_stored":497880563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463164,"kb_used_data":242660,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93908676,"statfs":{"total":96636764160,"available":96162484224,"internally_reserved":0,"allocated":248483840,"data_stored":248168445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":225767087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":498630656,"data_stored":498594526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":248213504,"data_stored":248137631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":425275392,"data_stored":425239262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:39.689 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:39.758 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 188 20 142311 142291 2.95606 1.76953 0.0184193 0.0264204 2024-09-15T21:23:40.268 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:40.268 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:40.298 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-09-15T21:23:38.741167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":577245937,"num_objects":140940,"num_object_clones":0,"num_object_copies":281880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":4043,"num_objects_unfound":0,"num_objects_dirty":140940,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":100805,"num_write_kb":403214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4205,"num_bytes_recovered":17211411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141013,"ondisk_log_size":141013,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1937940,"kb_used_data":1144596,"kb_used_omap":12,"kb_used_meta":793267,"kb_avail":281177580,"statfs":{"total":289910292480,"available":287925841920,"internally_reserved":0,"allocated":1172066304,"data_stored":1171208473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":812305411},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,12,8,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18886656,"num_objects":4611,"num_object_clones":0,"num_object_copies":9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4084,"num_objects_misplaced":3775,"num_objects_unfound":0,"num_objects_dirty":4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4611,"num_write_kb":18444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":1257472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001975"},"pg_stats":[{"pgid":"2.f","version":"44'9004","reported_seq":9077,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.606202+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:35.606202+0000","last_peered":"2024-09-15T21:23:35.606202+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:35.606202+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9004,"log_dups_size":0,"ondisk_log_size":9004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9004,"num_write_kb":36016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4872},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8873","reported_seq":8978,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.429271+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:38.429271+0000","last_peered":"2024-09-15T21:23:38.429271+0000","last_clean":"2024-09-15T21:23:38.429271+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:38.429271+0000","last_undegraded":"2024-09-15T21:23:38.429271+0000","last_fullsized":"2024-09-15T21:23:38.429271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8873,"log_dups_size":0,"ondisk_log_size":8873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36320198,"num_objects":8869,"num_object_clones":0,"num_object_copies":17738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8869,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7154,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9053","reported_seq":9178,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.432584+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:38.432584+0000","last_peered":"2024-09-15T21:23:38.432584+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:38.432584+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9053,"log_dups_size":0,"ondisk_log_size":9053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7153,"num_write_kb":28612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4782}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8727","reported_seq":8797,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.452289+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:38.452289+0000","last_peered":"2024-09-15T21:23:38.452289+0000","last_clean":"2024-09-15T21:23:38.452289+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:38.452289+0000","last_undegraded":"2024-09-15T21:23:38.452289+0000","last_fullsized":"2024-09-15T21:23:38.452289+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8727,"log_dups_size":0,"ondisk_log_size":8727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35745792,"num_objects":8727,"num_object_clones":0,"num_object_copies":17454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8727,"num_write_kb":34908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'8674","reported_seq":8804,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.653128+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:35.653128+0000","last_peered":"2024-09-15T21:23:35.653128+0000","last_clean":"2024-09-15T21:23:35.653128+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:35.653128+0000","last_undegraded":"2024-09-15T21:23:35.653128+0000","last_fullsized":"2024-09-15T21:23:35.653128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8674,"log_dups_size":0,"ondisk_log_size":8674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35500515,"num_objects":8668,"num_object_clones":0,"num_object_copies":17336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5129,"num_write_kb":20516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'8735","reported_seq":8869,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.668805+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:35.668805+0000","last_peered":"2024-09-15T21:23:35.668805+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:35.668805+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8735,"log_dups_size":0,"ondisk_log_size":8735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758563,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5411,"num_write_kb":21644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4655},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'8818","reported_seq":8952,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.579368+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:35.579368+0000","last_peered":"2024-09-15T21:23:35.579368+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:35.579368+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8818,"log_dups_size":0,"ondisk_log_size":8818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36094918,"num_objects":8814,"num_object_clones":0,"num_object_copies":17628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5129,"num_write_kb":20516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4653},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'8895","reported_seq":9021,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.659269+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:35.659269+0000","last_peered":"2024-09-15T21:23:35.659269+0000","last_clean":"2024-09-15T21:23:35.659269+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:35.659269+0000","last_undegraded":"2024-09-15T21:23:35.659269+0000","last_fullsized":"2024-09-15T21:23:35.659269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8895,"log_dups_size":0,"ondisk_log_size":8895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36425728,"num_objects":8893,"num_object_clones":0,"num_object_copies":17786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5667,"num_write_kb":22668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8842","reported_seq":8968,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.443010+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:38.443010+0000","last_peered":"2024-09-15T21:23:38.443010+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:38.443010+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8842,"log_dups_size":0,"ondisk_log_size":8842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36188643,"num_objects":8836,"num_object_clones":0,"num_object_copies":17672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8836,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6774,"num_write_kb":27096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4749}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'8630","reported_seq":8758,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.574675+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:35.574675+0000","last_peered":"2024-09-15T21:23:35.574675+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:35.574675+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8630,"log_dups_size":0,"ondisk_log_size":8630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35315712,"num_objects":8622,"num_object_clones":0,"num_object_copies":17244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":21212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4602},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.887834+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:34.887834+0000","last_peered":"2024-09-15T21:23:34.887834+0000","last_clean":"2024-09-15T21:23:34.887834+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:34.887834+0000","last_undegraded":"2024-09-15T21:23:34.887834+0000","last_fullsized":"2024-09-15T21:23:34.887834+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8874","reported_seq":8992,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.335510+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:38.335510+0000","last_peered":"2024-09-15T21:23:38.335510+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:38.335510+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8874,"log_dups_size":0,"ondisk_log_size":8874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36327907,"num_objects":8870,"num_object_clones":0,"num_object_copies":17740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8870,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6733,"num_write_kb":26932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4780}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8901","reported_seq":17359,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.439916+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:38.439916+0000","last_peered":"2024-09-15T21:23:38.439916+0000","last_clean":"2024-09-15T21:23:38.439916+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:38.439916+0000","last_undegraded":"2024-09-15T21:23:38.439916+0000","last_fullsized":"2024-09-15T21:23:38.439916+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8901,"log_dups_size":0,"ondisk_log_size":8901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6594,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'8779","reported_seq":8910,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.593135+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:35.593135+0000","last_peered":"2024-09-15T21:23:35.593135+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:35.593135+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8779,"log_dups_size":0,"ondisk_log_size":8779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35950592,"num_objects":8777,"num_object_clones":0,"num_object_copies":17554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5723,"num_write_kb":22892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4683},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'8720","reported_seq":8850,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.680309+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:35.680309+0000","last_peered":"2024-09-15T21:23:35.680309+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:35.680309+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8720,"log_dups_size":0,"ondisk_log_size":8720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35684352,"num_objects":8712,"num_object_clones":0,"num_object_copies":17424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5629,"num_write_kb":22516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4573},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'8631","reported_seq":8843,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:35.791081+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:35.791081+0000","last_peered":"2024-09-15T21:23:35.791081+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:35.791081+0000","last_undegraded":"2024-09-15T21:23:35.791081+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8631,"log_dups_size":0,"ondisk_log_size":8631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35336192,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4043,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5115,"num_write_kb":20460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":159744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4584},{"shards":"0,2","objects":4043}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'8854","reported_seq":8990,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.721883+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:35.721883+0000","last_peered":"2024-09-15T21:23:35.721883+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:35.721883+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8854,"log_dups_size":0,"ondisk_log_size":8854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36217798,"num_objects":8844,"num_object_clones":0,"num_object_copies":17688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8844,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5558,"num_write_kb":22232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4697},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":577245918,"num_objects":140938,"num_object_clones":0,"num_object_copies":281876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":4043,"num_objects_unfound":0,"num_objects_dirty":140938,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":100803,"num_write_kb":403212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4204,"num_bytes_recovered":17211392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1172119552,"data_stored":1171971419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141010,"ondisk_log_size":141010,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681784,"kb_used_data":415460,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93690056,"statfs":{"total":96636764160,"available":95938617344,"internally_reserved":0,"allocated":425431040,"data_stored":425159465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":272690857},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":792992,"kb_used_data":486476,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93578848,"statfs":{"total":96636764160,"available":95824740352,"internally_reserved":0,"allocated":498151424,"data_stored":497880563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463164,"kb_used_data":242660,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93908676,"statfs":{"total":96636764160,"available":96162484224,"internally_reserved":0,"allocated":248483840,"data_stored":248168445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":225767087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":498630656,"data_stored":498594526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":248213504,"data_stored":248137631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":425275392,"data_stored":425239262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:40.300 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:40.300 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "44'9004", 'reported_seq': 9077, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:35.606202+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:23:35.606202+0000', 'last_peered': '2024-09-15T21:23:35.606202+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:23:35.606202+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 9004, 'log_dups_size': 0, 'ondisk_log_size': 9004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36880384, 'num_objects': 9004, 'num_object_clones': 0, 'num_object_copies': 18008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9004, 'num_write_kb': 36016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4872}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:40.300 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:23:40.300 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'9053", 'reported_seq': 9178, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:38.432584+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:23:38.432584+0000', 'last_peered': '2024-09-15T21:23:38.432584+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:23:38.432584+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 9053, 'log_dups_size': 0, 'ondisk_log_size': 9053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37064704, 'num_objects': 9049, 'num_object_clones': 0, 'num_object_copies': 18098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9049, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7153, 'num_write_kb': 28612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 4782}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:40.300 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:40.300 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "44'8735", 'reported_seq': 8869, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:35.668805+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:23:35.668805+0000', 'last_peered': '2024-09-15T21:23:35.668805+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:23:35.668805+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 8735, 'log_dups_size': 0, 'ondisk_log_size': 8735, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35758563, 'num_objects': 8731, 'num_object_clones': 0, 'num_object_copies': 17462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8731, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5411, 'num_write_kb': 21644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4655}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:40.300 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:40.301 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "44'8818", 'reported_seq': 8952, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:35.579368+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:23:35.579368+0000', 'last_peered': '2024-09-15T21:23:35.579368+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:23:35.579368+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 8818, 'log_dups_size': 0, 'ondisk_log_size': 8818, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36094918, 'num_objects': 8814, 'num_object_clones': 0, 'num_object_copies': 17628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8814, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5129, 'num_write_kb': 20516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4653}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:40.301 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:40.301 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'8842", 'reported_seq': 8968, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:38.443010+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:23:38.443010+0000', 'last_peered': '2024-09-15T21:23:38.443010+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:23:38.443010+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 8842, 'log_dups_size': 0, 'ondisk_log_size': 8842, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36188643, 'num_objects': 8836, 'num_object_clones': 0, 'num_object_copies': 17672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8836, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6774, 'num_write_kb': 27096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 4749}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:40.301 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:40.301 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "44'8630", 'reported_seq': 8758, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:35.574675+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:23:35.574675+0000', 'last_peered': '2024-09-15T21:23:35.574675+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:23:35.574675+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 8630, 'log_dups_size': 0, 'ondisk_log_size': 8630, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35315712, 'num_objects': 8622, 'num_object_clones': 0, 'num_object_copies': 17244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8622, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5303, 'num_write_kb': 21212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4602}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:40.301 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:40.301 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'8874", 'reported_seq': 8992, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:38.335510+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:23:38.335510+0000', 'last_peered': '2024-09-15T21:23:38.335510+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:23:38.335510+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 8874, 'log_dups_size': 0, 'ondisk_log_size': 8874, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36327907, 'num_objects': 8870, 'num_object_clones': 0, 'num_object_copies': 17740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8870, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6733, 'num_write_kb': 26932, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 4780}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:40.301 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:40.301 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "44'8779", 'reported_seq': 8910, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:35.593135+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:23:35.593135+0000', 'last_peered': '2024-09-15T21:23:35.593135+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:23:35.593135+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 8779, 'log_dups_size': 0, 'ondisk_log_size': 8779, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35950592, 'num_objects': 8777, 'num_object_clones': 0, 'num_object_copies': 17554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5723, 'num_write_kb': 22892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4683}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:40.301 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:40.302 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "44'8720", 'reported_seq': 8850, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:35.680309+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:23:35.680309+0000', 'last_peered': '2024-09-15T21:23:35.680309+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:23:35.680309+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 8720, 'log_dups_size': 0, 'ondisk_log_size': 8720, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35684352, 'num_objects': 8712, 'num_object_clones': 0, 'num_object_copies': 17424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8712, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5629, 'num_write_kb': 22516, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4573}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:40.302 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:40.302 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "44'8631", 'reported_seq': 8843, 'reported_epoch': 44, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:23:35.791081+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:23:35.791081+0000', 'last_peered': '2024-09-15T21:23:35.791081+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:23:35.791081+0000', 'last_undegraded': '2024-09-15T21:23:35.791081+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 8631, 'log_dups_size': 0, 'ondisk_log_size': 8631, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35336192, 'num_objects': 8627, 'num_object_clones': 0, 'num_object_copies': 17254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 4043, 'num_objects_unfound': 0, 'num_objects_dirty': 8627, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5115, 'num_write_kb': 20460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 39, 'num_bytes_recovered': 159744, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4584}, {'shards': '0,2', 'objects': 4043}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:40.302 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:23:40.302 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "44'8854", 'reported_seq': 8990, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:35.721883+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:23:35.721883+0000', 'last_peered': '2024-09-15T21:23:35.721883+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:23:35.721883+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 8854, 'log_dups_size': 0, 'ondisk_log_size': 8854, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36217798, 'num_objects': 8844, 'num_object_clones': 0, 'num_object_copies': 17688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8844, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5558, 'num_write_kb': 22232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4697}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:40.303 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:40.758 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 189 20 142743 142723 2.94935 1.6875 0.024969 0.0264819 2024-09-15T21:23:41.003 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:41.003 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/281880 objects degraded (14.625%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":272,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":140940,"data_bytes":577245937,"bytes_used":1984450560,"bytes_avail":287925841920,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":281880,"degraded_ratio":0.14625017738044557,"misplaced_objects":4043,"misplaced_total":281880,"misplaced_ratio":0.014342982829572868,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":157068,"recovering_keys_per_sec":0,"num_objects_recovered":307,"num_bytes_recovered":1257472,"num_keys_recovered":0,"write_bytes_sec":1604456,"read_op_per_sec":0,"write_op_per_sec":391},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:41.026 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:41.026 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:41.577 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:41.577 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:41.601 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-15T21:23:40.745505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":577245937,"num_objects":140940,"num_object_clones":0,"num_object_copies":281880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":4043,"num_objects_unfound":0,"num_objects_dirty":140940,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":100805,"num_write_kb":403214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4205,"num_bytes_recovered":17211411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141013,"ondisk_log_size":141013,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1937940,"kb_used_data":1144596,"kb_used_omap":12,"kb_used_meta":793267,"kb_avail":281177580,"statfs":{"total":289910292480,"available":287925841920,"internally_reserved":0,"allocated":1172066304,"data_stored":1171208473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":812305411},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,12,8,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":12845056,"num_objects":3136,"num_object_clones":0,"num_object_copies":6272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4084,"num_objects_misplaced":3775,"num_objects_unfound":0,"num_objects_dirty":3136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3136,"num_write_kb":12544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":1257472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.005863"},"pg_stats":[{"pgid":"2.f","version":"44'9004","reported_seq":9077,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.606202+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:35.606202+0000","last_peered":"2024-09-15T21:23:35.606202+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:35.606202+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9004,"log_dups_size":0,"ondisk_log_size":9004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9004,"num_write_kb":36016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4872},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8873","reported_seq":8978,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.429271+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:38.429271+0000","last_peered":"2024-09-15T21:23:38.429271+0000","last_clean":"2024-09-15T21:23:38.429271+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:38.429271+0000","last_undegraded":"2024-09-15T21:23:38.429271+0000","last_fullsized":"2024-09-15T21:23:38.429271+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":8873,"log_dups_size":0,"ondisk_log_size":8873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36320198,"num_objects":8869,"num_object_clones":0,"num_object_copies":17738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8869,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7154,"num_write_kb":28616,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9053","reported_seq":9178,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.432584+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:38.432584+0000","last_peered":"2024-09-15T21:23:38.432584+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:38.432584+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9053,"log_dups_size":0,"ondisk_log_size":9053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7153,"num_write_kb":28612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4782}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8727","reported_seq":8797,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.452289+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:38.452289+0000","last_peered":"2024-09-15T21:23:38.452289+0000","last_clean":"2024-09-15T21:23:38.452289+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:38.452289+0000","last_undegraded":"2024-09-15T21:23:38.452289+0000","last_fullsized":"2024-09-15T21:23:38.452289+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8727,"log_dups_size":0,"ondisk_log_size":8727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35745792,"num_objects":8727,"num_object_clones":0,"num_object_copies":17454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8727,"num_write_kb":34908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'8674","reported_seq":8804,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.653128+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:35.653128+0000","last_peered":"2024-09-15T21:23:35.653128+0000","last_clean":"2024-09-15T21:23:35.653128+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:35.653128+0000","last_undegraded":"2024-09-15T21:23:35.653128+0000","last_fullsized":"2024-09-15T21:23:35.653128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8674,"log_dups_size":0,"ondisk_log_size":8674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35500515,"num_objects":8668,"num_object_clones":0,"num_object_copies":17336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5129,"num_write_kb":20516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'8735","reported_seq":8869,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.668805+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:35.668805+0000","last_peered":"2024-09-15T21:23:35.668805+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:35.668805+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8735,"log_dups_size":0,"ondisk_log_size":8735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35758563,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5411,"num_write_kb":21644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4655},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'8818","reported_seq":8952,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.579368+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:35.579368+0000","last_peered":"2024-09-15T21:23:35.579368+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:35.579368+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8818,"log_dups_size":0,"ondisk_log_size":8818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36094918,"num_objects":8814,"num_object_clones":0,"num_object_copies":17628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5129,"num_write_kb":20516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4653},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"44'8895","reported_seq":9021,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.659269+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:35.659269+0000","last_peered":"2024-09-15T21:23:35.659269+0000","last_clean":"2024-09-15T21:23:35.659269+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:35.659269+0000","last_undegraded":"2024-09-15T21:23:35.659269+0000","last_fullsized":"2024-09-15T21:23:35.659269+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":8895,"log_dups_size":0,"ondisk_log_size":8895,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36425728,"num_objects":8893,"num_object_clones":0,"num_object_copies":17786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5667,"num_write_kb":22668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8842","reported_seq":8968,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.443010+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:38.443010+0000","last_peered":"2024-09-15T21:23:38.443010+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:38.443010+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8842,"log_dups_size":0,"ondisk_log_size":8842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36188643,"num_objects":8836,"num_object_clones":0,"num_object_copies":17672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8836,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6774,"num_write_kb":27096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4749}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'8630","reported_seq":8758,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.574675+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:35.574675+0000","last_peered":"2024-09-15T21:23:35.574675+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:35.574675+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8630,"log_dups_size":0,"ondisk_log_size":8630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35315712,"num_objects":8622,"num_object_clones":0,"num_object_copies":17244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":21212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4602},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":98,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-15T21:23:34.887834+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:34.887834+0000","last_peered":"2024-09-15T21:23:34.887834+0000","last_clean":"2024-09-15T21:23:34.887834+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:34.887834+0000","last_undegraded":"2024-09-15T21:23:34.887834+0000","last_fullsized":"2024-09-15T21:23:34.887834+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8874","reported_seq":8992,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:38.335510+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:38.335510+0000","last_peered":"2024-09-15T21:23:38.335510+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:38.335510+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":8874,"log_dups_size":0,"ondisk_log_size":8874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36327907,"num_objects":8870,"num_object_clones":0,"num_object_copies":17740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8870,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6733,"num_write_kb":26932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4780}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8901","reported_seq":17359,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:38.439916+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:38.439916+0000","last_peered":"2024-09-15T21:23:38.439916+0000","last_clean":"2024-09-15T21:23:38.439916+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:38.439916+0000","last_undegraded":"2024-09-15T21:23:38.439916+0000","last_fullsized":"2024-09-15T21:23:38.439916+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":8901,"log_dups_size":0,"ondisk_log_size":8901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36433920,"num_objects":8895,"num_object_clones":0,"num_object_copies":17790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6594,"num_write_kb":26376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'8779","reported_seq":8910,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.593135+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:35.593135+0000","last_peered":"2024-09-15T21:23:35.593135+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:35.593135+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8779,"log_dups_size":0,"ondisk_log_size":8779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35950592,"num_objects":8777,"num_object_clones":0,"num_object_copies":17554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5723,"num_write_kb":22892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4683},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'8720","reported_seq":8850,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.680309+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:35.680309+0000","last_peered":"2024-09-15T21:23:35.680309+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:35.680309+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8720,"log_dups_size":0,"ondisk_log_size":8720,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35684352,"num_objects":8712,"num_object_clones":0,"num_object_copies":17424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5629,"num_write_kb":22516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4573},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'8631","reported_seq":8843,"reported_epoch":44,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:35.791081+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:35.791081+0000","last_peered":"2024-09-15T21:23:35.791081+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:35.791081+0000","last_undegraded":"2024-09-15T21:23:35.791081+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8631,"log_dups_size":0,"ondisk_log_size":8631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35336192,"num_objects":8627,"num_object_clones":0,"num_object_copies":17254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4043,"num_objects_unfound":0,"num_objects_dirty":8627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5115,"num_write_kb":20460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39,"num_bytes_recovered":159744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4584},{"shards":"0,2","objects":4043}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'8854","reported_seq":8990,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:35.721883+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:35.721883+0000","last_peered":"2024-09-15T21:23:35.721883+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:35.721883+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8854,"log_dups_size":0,"ondisk_log_size":8854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36217798,"num_objects":8844,"num_object_clones":0,"num_object_copies":17688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8844,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5558,"num_write_kb":22232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4697},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":577245918,"num_objects":140938,"num_object_clones":0,"num_object_copies":281876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":4043,"num_objects_unfound":0,"num_objects_dirty":140938,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":100803,"num_write_kb":403212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4204,"num_bytes_recovered":17211392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1172119552,"data_stored":1171971419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141010,"ondisk_log_size":141010,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681784,"kb_used_data":415460,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93690056,"statfs":{"total":96636764160,"available":95938617344,"internally_reserved":0,"allocated":425431040,"data_stored":425159465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":272690857},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,3,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":792992,"kb_used_data":486476,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93578848,"statfs":{"total":96636764160,"available":95824740352,"internally_reserved":0,"allocated":498151424,"data_stored":497880563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,1,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463164,"kb_used_data":242660,"kb_used_omap":4,"kb_used_meta":220475,"kb_avail":93908676,"statfs":{"total":96636764160,"available":96162484224,"internally_reserved":0,"allocated":248483840,"data_stored":248168445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":225767087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":498630656,"data_stored":498594526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":248213504,"data_stored":248137631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":425275392,"data_stored":425239262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:41.759 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 190 20 143183 143163 2.94287 1.71875 0.0755013 0.0265397 2024-09-15T21:23:42.759 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 191 20 143663 143643 2.93728 1.875 0.0174725 0.0265898 2024-09-15T21:23:43.759 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 192 20 144148 144128 2.93185 1.89453 0.0482561 0.0266391 2024-09-15T21:23:44.603 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:44.759 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 193 20 144622 144602 2.92625 1.85156 0.0555347 0.0266877 2024-09-15T21:23:45.146 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:45.148 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:45.171 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-15T21:23:44.746436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":586621681,"num_objects":143229,"num_object_clones":0,"num_object_copies":286458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":143229,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":103094,"num_write_kb":412370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4292,"num_bytes_recovered":17567763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143302,"ondisk_log_size":143302,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1952296,"kb_used_data":1163128,"kb_used_omap":12,"kb_used_meta":789107,"kb_avail":281163224,"statfs":{"total":289910292480,"available":287911141376,"internally_reserved":0,"allocated":1191043072,"data_stored":1190180450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":808045571},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,1,7,6,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18440192,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4084,"num_objects_misplaced":3900,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4502,"num_write_kb":18008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":745472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.876667"},"pg_stats":[{"pgid":"2.f","version":"45'9143","reported_seq":9218,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.821780+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:40.821780+0000","last_peered":"2024-09-15T21:23:40.821780+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:40.821780+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9143,"log_dups_size":0,"ondisk_log_size":9143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37449728,"num_objects":9143,"num_object_clones":0,"num_object_copies":18286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9143,"num_write_kb":36572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5011},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9015","reported_seq":9120,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.487804+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:43.487804+0000","last_peered":"2024-09-15T21:23:43.487804+0000","last_clean":"2024-09-15T21:23:43.487804+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:43.487804+0000","last_undegraded":"2024-09-15T21:23:43.487804+0000","last_fullsized":"2024-09-15T21:23:43.487804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9015,"log_dups_size":0,"ondisk_log_size":9015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36901830,"num_objects":9011,"num_object_clones":0,"num_object_copies":18022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9011,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7296,"num_write_kb":29184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9201","reported_seq":9326,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.440353+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:43.440353+0000","last_peered":"2024-09-15T21:23:43.440353+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:43.440353+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9201,"log_dups_size":0,"ondisk_log_size":9201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37670912,"num_objects":9197,"num_object_clones":0,"num_object_copies":18394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7301,"num_write_kb":29204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4930}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8854","reported_seq":8924,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.378043+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:43.378043+0000","last_peered":"2024-09-15T21:23:43.378043+0000","last_clean":"2024-09-15T21:23:43.378043+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:43.378043+0000","last_undegraded":"2024-09-15T21:23:43.378043+0000","last_fullsized":"2024-09-15T21:23:43.378043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8854,"log_dups_size":0,"ondisk_log_size":8854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36265984,"num_objects":8854,"num_object_clones":0,"num_object_copies":17708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8854,"num_write_kb":35416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8820","reported_seq":8951,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:40.823963+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:40.823963+0000","last_peered":"2024-09-15T21:23:40.823963+0000","last_clean":"2024-09-15T21:23:40.823963+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:40.823963+0000","last_undegraded":"2024-09-15T21:23:40.823963+0000","last_fullsized":"2024-09-15T21:23:40.823963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8820,"log_dups_size":0,"ondisk_log_size":8820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36098531,"num_objects":8814,"num_object_clones":0,"num_object_copies":17628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5275,"num_write_kb":21100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8882","reported_seq":9016,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.818144+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:40.818144+0000","last_peered":"2024-09-15T21:23:40.818144+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:40.818144+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8882,"log_dups_size":0,"ondisk_log_size":8882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36360675,"num_objects":8878,"num_object_clones":0,"num_object_copies":17756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5558,"num_write_kb":22232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4802},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8968","reported_seq":9104,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.742484+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:40.742484+0000","last_peered":"2024-09-15T21:23:40.742484+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:40.742484+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8968,"log_dups_size":0,"ondisk_log_size":8968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36709318,"num_objects":8964,"num_object_clones":0,"num_object_copies":17928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5279,"num_write_kb":21116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4803},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9051","reported_seq":9179,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:40.816960+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:40.816960+0000","last_peered":"2024-09-15T21:23:40.816960+0000","last_clean":"2024-09-15T21:23:40.816960+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:40.816960+0000","last_undegraded":"2024-09-15T21:23:40.816960+0000","last_fullsized":"2024-09-15T21:23:40.816960+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9051,"log_dups_size":0,"ondisk_log_size":9051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5823,"num_write_kb":23292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8977","reported_seq":9103,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.454657+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:43.454657+0000","last_peered":"2024-09-15T21:23:43.454657+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:43.454657+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8977,"log_dups_size":0,"ondisk_log_size":8977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36741603,"num_objects":8971,"num_object_clones":0,"num_object_copies":17942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8971,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6909,"num_write_kb":27636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4884}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8776","reported_seq":8906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.612453+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:40.612453+0000","last_peered":"2024-09-15T21:23:40.612453+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:40.612453+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8776,"log_dups_size":0,"ondisk_log_size":8776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5449,"num_write_kb":21796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4748},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9024","reported_seq":9137,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.474374+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:43.474374+0000","last_peered":"2024-09-15T21:23:43.474374+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:43.474374+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9024,"log_dups_size":0,"ondisk_log_size":9024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36942307,"num_objects":9020,"num_object_clones":0,"num_object_copies":18040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6883,"num_write_kb":27532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4930}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9025","reported_seq":17483,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.485756+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:43.485756+0000","last_peered":"2024-09-15T21:23:43.485756+0000","last_clean":"2024-09-15T21:23:43.485756+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:43.485756+0000","last_undegraded":"2024-09-15T21:23:43.485756+0000","last_fullsized":"2024-09-15T21:23:43.485756+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9025,"log_dups_size":0,"ondisk_log_size":9025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36941824,"num_objects":9019,"num_object_clones":0,"num_object_copies":18038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6718,"num_write_kb":26872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8912","reported_seq":9045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.693291+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:40.693291+0000","last_peered":"2024-09-15T21:23:40.693291+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:40.693291+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8912,"log_dups_size":0,"ondisk_log_size":8912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36495360,"num_objects":8910,"num_object_clones":0,"num_object_copies":17820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5856,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4816},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8870","reported_seq":9002,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.789856+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:40.789856+0000","last_peered":"2024-09-15T21:23:40.789856+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:40.789856+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8870,"log_dups_size":0,"ondisk_log_size":8870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36298752,"num_objects":8862,"num_object_clones":0,"num_object_copies":17724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5779,"num_write_kb":23116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4723},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8784","reported_seq":9172,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:40.839014+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:40.839014+0000","last_peered":"2024-09-15T21:23:40.839014+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:40.839014+0000","last_undegraded":"2024-09-15T21:23:40.839014+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8784,"log_dups_size":0,"ondisk_log_size":8784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962880,"num_objects":8780,"num_object_clones":0,"num_object_copies":17560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":8780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5268,"num_write_kb":21072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":516096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4824},{"shards":"0,2","objects":3956}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8997","reported_seq":9135,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.809949+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:40.809949+0000","last_peered":"2024-09-15T21:23:40.809949+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:40.809949+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8997,"log_dups_size":0,"ondisk_log_size":8997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36803526,"num_objects":8987,"num_object_clones":0,"num_object_copies":17974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8987,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5701,"num_write_kb":22804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4840},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":586621662,"num_objects":143227,"num_object_clones":0,"num_object_copies":286454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":143227,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":103092,"num_write_kb":412368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4291,"num_bytes_recovered":17567744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1191333888,"data_stored":1191185755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143299,"ondisk_log_size":143299,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":690496,"kb_used_data":420016,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93681344,"statfs":{"total":96636764160,"available":95929696256,"internally_reserved":0,"allocated":430096384,"data_stored":429824809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":276950697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182449,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803928,"kb_used_data":493256,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93567912,"statfs":{"total":96636764160,"available":95813541888,"internally_reserved":0,"allocated":505094144,"data_stored":504818492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,0,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457872,"kb_used_data":249856,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93913968,"statfs":{"total":96636764160,"available":96167903232,"internally_reserved":0,"allocated":255852544,"data_stored":255537149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":212987567},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":505839616,"data_stored":505803486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255557632,"data_stored":255481759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":429936640,"data_stored":429900510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:45.173 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:45.759 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 194 20 145039 145019 2.91956 1.62891 0.0287776 0.0267515 2024-09-15T21:23:45.774 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:45.774 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:45.804 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-15T21:23:44.746436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":586621681,"num_objects":143229,"num_object_clones":0,"num_object_copies":286458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":143229,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":103094,"num_write_kb":412370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4292,"num_bytes_recovered":17567763,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143302,"ondisk_log_size":143302,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1952296,"kb_used_data":1163128,"kb_used_omap":12,"kb_used_meta":789107,"kb_avail":281163224,"statfs":{"total":289910292480,"available":287911141376,"internally_reserved":0,"allocated":1191043072,"data_stored":1190180450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":808045571},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,2,1,7,6,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18440192,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-4084,"num_objects_misplaced":3900,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4502,"num_write_kb":18008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":745472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.876667"},"pg_stats":[{"pgid":"2.f","version":"45'9143","reported_seq":9218,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.821780+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:40.821780+0000","last_peered":"2024-09-15T21:23:40.821780+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:40.821780+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9143,"log_dups_size":0,"ondisk_log_size":9143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37449728,"num_objects":9143,"num_object_clones":0,"num_object_copies":18286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9143,"num_write_kb":36572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5011},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9015","reported_seq":9120,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.487804+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:43.487804+0000","last_peered":"2024-09-15T21:23:43.487804+0000","last_clean":"2024-09-15T21:23:43.487804+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:43.487804+0000","last_undegraded":"2024-09-15T21:23:43.487804+0000","last_fullsized":"2024-09-15T21:23:43.487804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9015,"log_dups_size":0,"ondisk_log_size":9015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36901830,"num_objects":9011,"num_object_clones":0,"num_object_copies":18022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9011,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7296,"num_write_kb":29184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9201","reported_seq":9326,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.440353+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:43.440353+0000","last_peered":"2024-09-15T21:23:43.440353+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:43.440353+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9201,"log_dups_size":0,"ondisk_log_size":9201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37670912,"num_objects":9197,"num_object_clones":0,"num_object_copies":18394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7301,"num_write_kb":29204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4930}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8854","reported_seq":8924,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.378043+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:43.378043+0000","last_peered":"2024-09-15T21:23:43.378043+0000","last_clean":"2024-09-15T21:23:43.378043+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:43.378043+0000","last_undegraded":"2024-09-15T21:23:43.378043+0000","last_fullsized":"2024-09-15T21:23:43.378043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8854,"log_dups_size":0,"ondisk_log_size":8854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36265984,"num_objects":8854,"num_object_clones":0,"num_object_copies":17708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8854,"num_write_kb":35416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8820","reported_seq":8951,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:40.823963+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:40.823963+0000","last_peered":"2024-09-15T21:23:40.823963+0000","last_clean":"2024-09-15T21:23:40.823963+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:40.823963+0000","last_undegraded":"2024-09-15T21:23:40.823963+0000","last_fullsized":"2024-09-15T21:23:40.823963+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":8820,"log_dups_size":0,"ondisk_log_size":8820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36098531,"num_objects":8814,"num_object_clones":0,"num_object_copies":17628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5275,"num_write_kb":21100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8882","reported_seq":9016,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.818144+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:40.818144+0000","last_peered":"2024-09-15T21:23:40.818144+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:40.818144+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":8882,"log_dups_size":0,"ondisk_log_size":8882,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36360675,"num_objects":8878,"num_object_clones":0,"num_object_copies":17756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8878,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5558,"num_write_kb":22232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4802},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8968","reported_seq":9104,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.742484+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:40.742484+0000","last_peered":"2024-09-15T21:23:40.742484+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:40.742484+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":8968,"log_dups_size":0,"ondisk_log_size":8968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36709318,"num_objects":8964,"num_object_clones":0,"num_object_copies":17928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5279,"num_write_kb":21116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4803},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9051","reported_seq":9179,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:40.816960+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:40.816960+0000","last_peered":"2024-09-15T21:23:40.816960+0000","last_clean":"2024-09-15T21:23:40.816960+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:40.816960+0000","last_undegraded":"2024-09-15T21:23:40.816960+0000","last_fullsized":"2024-09-15T21:23:40.816960+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9051,"log_dups_size":0,"ondisk_log_size":9051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37064704,"num_objects":9049,"num_object_clones":0,"num_object_copies":18098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5823,"num_write_kb":23292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8977","reported_seq":9103,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.454657+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:43.454657+0000","last_peered":"2024-09-15T21:23:43.454657+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:43.454657+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8977,"log_dups_size":0,"ondisk_log_size":8977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36741603,"num_objects":8971,"num_object_clones":0,"num_object_copies":17942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8971,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6909,"num_write_kb":27636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4884}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8776","reported_seq":8906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.612453+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:40.612453+0000","last_peered":"2024-09-15T21:23:40.612453+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:40.612453+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8776,"log_dups_size":0,"ondisk_log_size":8776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35913728,"num_objects":8768,"num_object_clones":0,"num_object_copies":17536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5449,"num_write_kb":21796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4748},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9024","reported_seq":9137,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.474374+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:43.474374+0000","last_peered":"2024-09-15T21:23:43.474374+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:43.474374+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9024,"log_dups_size":0,"ondisk_log_size":9024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36942307,"num_objects":9020,"num_object_clones":0,"num_object_copies":18040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6883,"num_write_kb":27532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4930}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9025","reported_seq":17483,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.485756+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:43.485756+0000","last_peered":"2024-09-15T21:23:43.485756+0000","last_clean":"2024-09-15T21:23:43.485756+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:43.485756+0000","last_undegraded":"2024-09-15T21:23:43.485756+0000","last_fullsized":"2024-09-15T21:23:43.485756+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9025,"log_dups_size":0,"ondisk_log_size":9025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36941824,"num_objects":9019,"num_object_clones":0,"num_object_copies":18038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6718,"num_write_kb":26872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8912","reported_seq":9045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.693291+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:40.693291+0000","last_peered":"2024-09-15T21:23:40.693291+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:40.693291+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":8912,"log_dups_size":0,"ondisk_log_size":8912,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36495360,"num_objects":8910,"num_object_clones":0,"num_object_copies":17820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5856,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4816},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8870","reported_seq":9002,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.789856+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:40.789856+0000","last_peered":"2024-09-15T21:23:40.789856+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:40.789856+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":8870,"log_dups_size":0,"ondisk_log_size":8870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36298752,"num_objects":8862,"num_object_clones":0,"num_object_copies":17724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5779,"num_write_kb":23116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4723},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8784","reported_seq":9172,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:40.839014+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:40.839014+0000","last_peered":"2024-09-15T21:23:40.839014+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:40.839014+0000","last_undegraded":"2024-09-15T21:23:40.839014+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8784,"log_dups_size":0,"ondisk_log_size":8784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962880,"num_objects":8780,"num_object_clones":0,"num_object_copies":17560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":8780,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5268,"num_write_kb":21072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":516096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4824},{"shards":"0,2","objects":3956}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8997","reported_seq":9135,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:40.809949+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:40.809949+0000","last_peered":"2024-09-15T21:23:40.809949+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:40.809949+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":8997,"log_dups_size":0,"ondisk_log_size":8997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36803526,"num_objects":8987,"num_object_clones":0,"num_object_copies":17974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8987,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5701,"num_write_kb":22804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4840},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":586621662,"num_objects":143227,"num_object_clones":0,"num_object_copies":286454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3956,"num_objects_unfound":0,"num_objects_dirty":143227,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":103092,"num_write_kb":412368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4291,"num_bytes_recovered":17567744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1191333888,"data_stored":1191185755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143299,"ondisk_log_size":143299,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":690496,"kb_used_data":420016,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93681344,"statfs":{"total":96636764160,"available":95929696256,"internally_reserved":0,"allocated":430096384,"data_stored":429824809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":276950697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182449,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":803928,"kb_used_data":493256,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93567912,"statfs":{"total":96636764160,"available":95813541888,"internally_reserved":0,"allocated":505094144,"data_stored":504818492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,0,3,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457872,"kb_used_data":249856,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93913968,"statfs":{"total":96636764160,"available":96167903232,"internally_reserved":0,"allocated":255852544,"data_stored":255537149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":212987567},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":505839616,"data_stored":505803486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255557632,"data_stored":255481759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":429936640,"data_stored":429900510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:45.805 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:45.806 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'9143", 'reported_seq': 9218, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:40.821780+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:23:40.821780+0000', 'last_peered': '2024-09-15T21:23:40.821780+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:23:40.821780+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 9143, 'log_dups_size': 0, 'ondisk_log_size': 9143, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37449728, 'num_objects': 9143, 'num_object_clones': 0, 'num_object_copies': 18286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9143, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9143, 'num_write_kb': 36572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5011}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.806 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:23:45.806 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'9201", 'reported_seq': 9326, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:43.440353+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:23:43.440353+0000', 'last_peered': '2024-09-15T21:23:43.440353+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:23:43.440353+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 9201, 'log_dups_size': 0, 'ondisk_log_size': 9201, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37670912, 'num_objects': 9197, 'num_object_clones': 0, 'num_object_copies': 18394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9197, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7301, 'num_write_kb': 29204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 4930}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:45.806 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:45.806 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'8882", 'reported_seq': 9016, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:40.818144+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:23:40.818144+0000', 'last_peered': '2024-09-15T21:23:40.818144+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:23:40.818144+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 8882, 'log_dups_size': 0, 'ondisk_log_size': 8882, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36360675, 'num_objects': 8878, 'num_object_clones': 0, 'num_object_copies': 17756, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8878, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5558, 'num_write_kb': 22232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4802}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.806 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:45.806 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'8968", 'reported_seq': 9104, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:40.742484+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:23:40.742484+0000', 'last_peered': '2024-09-15T21:23:40.742484+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:23:40.742484+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 8968, 'log_dups_size': 0, 'ondisk_log_size': 8968, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36709318, 'num_objects': 8964, 'num_object_clones': 0, 'num_object_copies': 17928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8964, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5279, 'num_write_kb': 21116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4803}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.806 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:45.806 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'8977", 'reported_seq': 9103, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:43.454657+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:23:43.454657+0000', 'last_peered': '2024-09-15T21:23:43.454657+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:23:43.454657+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 8977, 'log_dups_size': 0, 'ondisk_log_size': 8977, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36741603, 'num_objects': 8971, 'num_object_clones': 0, 'num_object_copies': 17942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8971, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6909, 'num_write_kb': 27636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 4884}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:45.807 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:45.807 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'8776", 'reported_seq': 8906, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:40.612453+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:23:40.612453+0000', 'last_peered': '2024-09-15T21:23:40.612453+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:23:40.612453+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 8776, 'log_dups_size': 0, 'ondisk_log_size': 8776, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35913728, 'num_objects': 8768, 'num_object_clones': 0, 'num_object_copies': 17536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8768, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5449, 'num_write_kb': 21796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4748}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.807 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:45.807 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'9024", 'reported_seq': 9137, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:43.474374+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:23:43.474374+0000', 'last_peered': '2024-09-15T21:23:43.474374+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:23:43.474374+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 9024, 'log_dups_size': 0, 'ondisk_log_size': 9024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36942307, 'num_objects': 9020, 'num_object_clones': 0, 'num_object_copies': 18040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9020, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6883, 'num_write_kb': 27532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 4930}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:45.807 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:45.807 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'8912", 'reported_seq': 9045, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:40.693291+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:23:40.693291+0000', 'last_peered': '2024-09-15T21:23:40.693291+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:23:40.693291+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 8912, 'log_dups_size': 0, 'ondisk_log_size': 8912, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36495360, 'num_objects': 8910, 'num_object_clones': 0, 'num_object_copies': 17820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8910, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5856, 'num_write_kb': 23424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4816}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.807 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:45.807 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'8870", 'reported_seq': 9002, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:40.789856+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:23:40.789856+0000', 'last_peered': '2024-09-15T21:23:40.789856+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:23:40.789856+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 8870, 'log_dups_size': 0, 'ondisk_log_size': 8870, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36298752, 'num_objects': 8862, 'num_object_clones': 0, 'num_object_copies': 17724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8862, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5779, 'num_write_kb': 23116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4723}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.807 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:45.807 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'8784", 'reported_seq': 9172, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:23:40.839014+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:23:40.839014+0000', 'last_peered': '2024-09-15T21:23:40.839014+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:23:40.839014+0000', 'last_undegraded': '2024-09-15T21:23:40.839014+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 8784, 'log_dups_size': 0, 'ondisk_log_size': 8784, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35962880, 'num_objects': 8780, 'num_object_clones': 0, 'num_object_copies': 17560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3956, 'num_objects_unfound': 0, 'num_objects_dirty': 8780, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5268, 'num_write_kb': 21072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 126, 'num_bytes_recovered': 516096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4824}, {'shards': '0,2', 'objects': 3956}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.807 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:23:45.808 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'8997", 'reported_seq': 9135, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:40.809949+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:23:40.809949+0000', 'last_peered': '2024-09-15T21:23:40.809949+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:23:40.809949+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 8997, 'log_dups_size': 0, 'ondisk_log_size': 8997, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36803526, 'num_objects': 8987, 'num_object_clones': 0, 'num_object_copies': 17974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8987, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5701, 'num_write_kb': 22804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4840}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:45.808 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:46.509 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:46.509 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/286458 objects degraded (14.391%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":278,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":143229,"data_bytes":586621681,"bytes_used":1999151104,"bytes_avail":287911141376,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":286458,"degraded_ratio":0.14391289473500479,"misplaced_objects":3956,"misplaced_total":286458,"misplaced_ratio":0.013810052433515558,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":75478,"recovering_keys_per_sec":0,"num_objects_recovered":182,"num_bytes_recovered":745472,"num_keys_recovered":0,"write_bytes_sec":1867046,"read_op_per_sec":0,"write_op_per_sec":455},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:46.539 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:46.539 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:46.759 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 195 20 145472 145452 2.91326 1.69141 0.0347775 0.0268099 2024-09-15T21:23:47.094 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:47.097 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:47.117 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":177,"stamp":"2024-09-15T21:23:46.746894+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592659185,"num_objects":144703,"num_object_clones":0,"num_object_copies":289406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3876,"num_objects_unfound":0,"num_objects_dirty":144703,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":104568,"num_write_kb":418266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":17895443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144776,"ondisk_log_size":144776,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1946960,"kb_used_data":1170272,"kb_used_omap":12,"kb_used_meta":776627,"kb_avail":281168560,"statfs":{"total":289910292480,"available":287916605440,"internally_reserved":0,"allocated":1198358528,"data_stored":1197495906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":795266051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,11,9,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18739200,"num_objects":4575,"num_object_clones":0,"num_object_copies":9150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-223,"num_objects_unfound":0,"num_objects_dirty":4575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4575,"num_write_kb":18300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.871353"},"pg_stats":[{"pgid":"2.f","version":"45'9306","reported_seq":9380,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.938279+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:45.938279+0000","last_peered":"2024-09-15T21:23:45.938279+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:45.938279+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9306,"log_dups_size":0,"ondisk_log_size":9306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38117376,"num_objects":9306,"num_object_clones":0,"num_object_copies":18612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9306,"num_write_kb":37224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5174},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9015","reported_seq":9120,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.487804+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:43.487804+0000","last_peered":"2024-09-15T21:23:43.487804+0000","last_clean":"2024-09-15T21:23:43.487804+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:43.487804+0000","last_undegraded":"2024-09-15T21:23:43.487804+0000","last_fullsized":"2024-09-15T21:23:43.487804+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9015,"log_dups_size":0,"ondisk_log_size":9015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36901830,"num_objects":9011,"num_object_clones":0,"num_object_copies":18022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9011,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7296,"num_write_kb":29184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9201","reported_seq":9326,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.440353+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:43.440353+0000","last_peered":"2024-09-15T21:23:43.440353+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:43.440353+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9201,"log_dups_size":0,"ondisk_log_size":9201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37670912,"num_objects":9197,"num_object_clones":0,"num_object_copies":18394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7301,"num_write_kb":29204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":4930}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8854","reported_seq":8924,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.378043+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:43.378043+0000","last_peered":"2024-09-15T21:23:43.378043+0000","last_clean":"2024-09-15T21:23:43.378043+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:43.378043+0000","last_undegraded":"2024-09-15T21:23:43.378043+0000","last_fullsized":"2024-09-15T21:23:43.378043+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":8854,"log_dups_size":0,"ondisk_log_size":8854,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36265984,"num_objects":8854,"num_object_clones":0,"num_object_copies":17708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8854,"num_write_kb":35416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9000","reported_seq":9131,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:45.924718+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:45.924718+0000","last_peered":"2024-09-15T21:23:45.924718+0000","last_clean":"2024-09-15T21:23:45.924718+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:45.924718+0000","last_undegraded":"2024-09-15T21:23:45.924718+0000","last_fullsized":"2024-09-15T21:23:45.924718+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9000,"log_dups_size":0,"ondisk_log_size":9000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36835811,"num_objects":8994,"num_object_clones":0,"num_object_copies":17988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5455,"num_write_kb":21820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9012","reported_seq":9148,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.875427+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:45.875427+0000","last_peered":"2024-09-15T21:23:45.875427+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:45.875427+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9012,"log_dups_size":0,"ondisk_log_size":9012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36893155,"num_objects":9008,"num_object_clones":0,"num_object_copies":18016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9008,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5688,"num_write_kb":22752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4932},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9113","reported_seq":9248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.837457+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:45.837457+0000","last_peered":"2024-09-15T21:23:45.837457+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:45.837457+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9113,"log_dups_size":0,"ondisk_log_size":9113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37303238,"num_objects":9109,"num_object_clones":0,"num_object_copies":18218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5424,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4948},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9174","reported_seq":9302,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:45.777771+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:45.777771+0000","last_peered":"2024-09-15T21:23:45.777771+0000","last_clean":"2024-09-15T21:23:45.777771+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:45.777771+0000","last_undegraded":"2024-09-15T21:23:45.777771+0000","last_fullsized":"2024-09-15T21:23:45.777771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9174,"log_dups_size":0,"ondisk_log_size":9174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37568512,"num_objects":9172,"num_object_clones":0,"num_object_copies":18344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5946,"num_write_kb":23784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8977","reported_seq":9103,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.454657+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:43.454657+0000","last_peered":"2024-09-15T21:23:43.454657+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:43.454657+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":8977,"log_dups_size":0,"ondisk_log_size":8977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36741603,"num_objects":8971,"num_object_clones":0,"num_object_copies":17942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8971,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6909,"num_write_kb":27636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":4884}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8959","reported_seq":9089,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.803849+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:45.803849+0000","last_peered":"2024-09-15T21:23:45.803849+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:45.803849+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8959,"log_dups_size":0,"ondisk_log_size":8959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663296,"num_objects":8951,"num_object_clones":0,"num_object_copies":17902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5632,"num_write_kb":22528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4931},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9024","reported_seq":9137,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:43.474374+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:43.474374+0000","last_peered":"2024-09-15T21:23:43.474374+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:43.474374+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9024,"log_dups_size":0,"ondisk_log_size":9024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36942307,"num_objects":9020,"num_object_clones":0,"num_object_copies":18040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6883,"num_write_kb":27532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":4930}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9025","reported_seq":17483,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:43.485756+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:43.485756+0000","last_peered":"2024-09-15T21:23:43.485756+0000","last_clean":"2024-09-15T21:23:43.485756+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:43.485756+0000","last_undegraded":"2024-09-15T21:23:43.485756+0000","last_fullsized":"2024-09-15T21:23:43.485756+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9025,"log_dups_size":0,"ondisk_log_size":9025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36941824,"num_objects":9019,"num_object_clones":0,"num_object_copies":18038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6718,"num_write_kb":26872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9047","reported_seq":9180,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.836969+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:45.836969+0000","last_peered":"2024-09-15T21:23:45.836969+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:45.836969+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9047,"log_dups_size":0,"ondisk_log_size":9047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5991,"num_write_kb":23964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4951},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9009","reported_seq":9141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.792121+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:45.792121+0000","last_peered":"2024-09-15T21:23:45.792121+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:45.792121+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9009,"log_dups_size":0,"ondisk_log_size":9009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36868096,"num_objects":9001,"num_object_clones":0,"num_object_copies":18002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5918,"num_write_kb":23672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4862},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8933","reported_seq":9477,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:45.893964+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:45.893964+0000","last_peered":"2024-09-15T21:23:45.893964+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:45.893964+0000","last_undegraded":"2024-09-15T21:23:45.893964+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8933,"log_dups_size":0,"ondisk_log_size":8933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36573184,"num_objects":8929,"num_object_clones":0,"num_object_copies":17858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3876,"num_objects_unfound":0,"num_objects_dirty":8929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5417,"num_write_kb":21668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5053},{"shards":"0,2","objects":3876}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9124","reported_seq":9262,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.809765+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:45.809765+0000","last_peered":"2024-09-15T21:23:45.809765+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:45.809765+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9124,"log_dups_size":0,"ondisk_log_size":9124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37323718,"num_objects":9114,"num_object_clones":0,"num_object_copies":18228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9114,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5828,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4967},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":592659166,"num_objects":144701,"num_object_clones":0,"num_object_copies":289402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3876,"num_objects_unfound":0,"num_objects_dirty":144701,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":104566,"num_write_kb":418264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4371,"num_bytes_recovered":17895424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1198489600,"data_stored":1198341467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":144773,"ondisk_log_size":144773,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":690496,"kb_used_data":420016,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93681344,"statfs":{"total":96636764160,"available":95929696256,"internally_reserved":0,"allocated":430096384,"data_stored":429824809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":276950697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798592,"kb_used_data":500400,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93573248,"statfs":{"total":96636764160,"available":95819005952,"internally_reserved":0,"allocated":512409600,"data_stored":512133948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305327787},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457872,"kb_used_data":249856,"kb_used_omap":4,"kb_used_meta":207995,"kb_avail":93913968,"statfs":{"total":96636764160,"available":96167903232,"internally_reserved":0,"allocated":255852544,"data_stored":255537149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":212987567},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":512995328,"data_stored":512959198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":255557632,"data_stored":255481759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":429936640,"data_stored":429900510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:47.759 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 196 20 145937 145917 2.90767 1.81641 0.0676683 0.0268616 2024-09-15T21:23:48.762 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 197 20 146417 146397 2.90239 1.875 0.0191563 0.0269092 2024-09-15T21:23:49.762 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 198 20 146900 146880 2.89726 1.88672 0.0942848 0.0269576 2024-09-15T21:23:50.119 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:50.690 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:50.691 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:50.713 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-15T21:23:48.747351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596251377,"num_objects":145580,"num_object_clones":0,"num_object_copies":291160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3876,"num_objects_unfound":0,"num_objects_dirty":145580,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":105445,"num_write_kb":421774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":17895443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145653,"ondisk_log_size":145653,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1967036,"kb_used_data":1182028,"kb_used_omap":12,"kb_used_meta":784947,"kb_avail":281148484,"statfs":{"total":289910292480,"available":287896047616,"internally_reserved":0,"allocated":1210396672,"data_stored":1209534050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":803785731},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,9,13,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22331392,"num_objects":5452,"num_object_clones":0,"num_object_copies":10904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-223,"num_objects_unfound":0,"num_objects_dirty":5452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5452,"num_write_kb":21808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006704"},"pg_stats":[{"pgid":"2.f","version":"45'9306","reported_seq":9380,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.938279+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:45.938279+0000","last_peered":"2024-09-15T21:23:45.938279+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:45.938279+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9306,"log_dups_size":0,"ondisk_log_size":9306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38117376,"num_objects":9306,"num_object_clones":0,"num_object_copies":18612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9306,"num_write_kb":37224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5174},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9157","reported_seq":9262,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.485514+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:48.485514+0000","last_peered":"2024-09-15T21:23:48.485514+0000","last_clean":"2024-09-15T21:23:48.485514+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:48.485514+0000","last_undegraded":"2024-09-15T21:23:48.485514+0000","last_fullsized":"2024-09-15T21:23:48.485514+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9157,"log_dups_size":0,"ondisk_log_size":9157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37483462,"num_objects":9153,"num_object_clones":0,"num_object_copies":18306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9153,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7438,"num_write_kb":29752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9368","reported_seq":9493,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.453300+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:48.453300+0000","last_peered":"2024-09-15T21:23:48.453300+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:48.453300+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9368,"log_dups_size":0,"ondisk_log_size":9368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38354944,"num_objects":9364,"num_object_clones":0,"num_object_copies":18728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7468,"num_write_kb":29872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5097}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9004","reported_seq":9074,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.365994+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:48.365994+0000","last_peered":"2024-09-15T21:23:48.365994+0000","last_clean":"2024-09-15T21:23:48.365994+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:48.365994+0000","last_undegraded":"2024-09-15T21:23:48.365994+0000","last_fullsized":"2024-09-15T21:23:48.365994+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9004,"log_dups_size":0,"ondisk_log_size":9004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9004,"num_write_kb":36016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9000","reported_seq":9131,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:45.924718+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:45.924718+0000","last_peered":"2024-09-15T21:23:45.924718+0000","last_clean":"2024-09-15T21:23:45.924718+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:45.924718+0000","last_undegraded":"2024-09-15T21:23:45.924718+0000","last_fullsized":"2024-09-15T21:23:45.924718+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9000,"log_dups_size":0,"ondisk_log_size":9000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36835811,"num_objects":8994,"num_object_clones":0,"num_object_copies":17988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5455,"num_write_kb":21820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9012","reported_seq":9148,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.875427+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:45.875427+0000","last_peered":"2024-09-15T21:23:45.875427+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:45.875427+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9012,"log_dups_size":0,"ondisk_log_size":9012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36893155,"num_objects":9008,"num_object_clones":0,"num_object_copies":18016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9008,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5688,"num_write_kb":22752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4932},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9113","reported_seq":9248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.837457+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:45.837457+0000","last_peered":"2024-09-15T21:23:45.837457+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:45.837457+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9113,"log_dups_size":0,"ondisk_log_size":9113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37303238,"num_objects":9109,"num_object_clones":0,"num_object_copies":18218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5424,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4948},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9174","reported_seq":9302,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:45.777771+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:45.777771+0000","last_peered":"2024-09-15T21:23:45.777771+0000","last_clean":"2024-09-15T21:23:45.777771+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:45.777771+0000","last_undegraded":"2024-09-15T21:23:45.777771+0000","last_fullsized":"2024-09-15T21:23:45.777771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9174,"log_dups_size":0,"ondisk_log_size":9174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37568512,"num_objects":9172,"num_object_clones":0,"num_object_copies":18344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5946,"num_write_kb":23784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9117","reported_seq":9243,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.464523+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:48.464523+0000","last_peered":"2024-09-15T21:23:48.464523+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:48.464523+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9117,"log_dups_size":0,"ondisk_log_size":9117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37315043,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7049,"num_write_kb":28196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5024}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8959","reported_seq":9089,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.803849+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:45.803849+0000","last_peered":"2024-09-15T21:23:45.803849+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:45.803849+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8959,"log_dups_size":0,"ondisk_log_size":8959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663296,"num_objects":8951,"num_object_clones":0,"num_object_copies":17902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5632,"num_write_kb":22528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4931},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9160","reported_seq":9277,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.501081+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:48.501081+0000","last_peered":"2024-09-15T21:23:48.501081+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:48.501081+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9160,"log_dups_size":0,"ondisk_log_size":9160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37499363,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7019,"num_write_kb":28076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5066}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9167","reported_seq":17625,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.511962+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:48.511962+0000","last_peered":"2024-09-15T21:23:48.511962+0000","last_clean":"2024-09-15T21:23:48.511962+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:48.511962+0000","last_undegraded":"2024-09-15T21:23:48.511962+0000","last_fullsized":"2024-09-15T21:23:48.511962+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9167,"log_dups_size":0,"ondisk_log_size":9167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37523456,"num_objects":9161,"num_object_clones":0,"num_object_copies":18322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6860,"num_write_kb":27440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9047","reported_seq":9180,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.836969+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:45.836969+0000","last_peered":"2024-09-15T21:23:45.836969+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:45.836969+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9047,"log_dups_size":0,"ondisk_log_size":9047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5991,"num_write_kb":23964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4951},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9009","reported_seq":9141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.792121+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:45.792121+0000","last_peered":"2024-09-15T21:23:45.792121+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:45.792121+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9009,"log_dups_size":0,"ondisk_log_size":9009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36868096,"num_objects":9001,"num_object_clones":0,"num_object_copies":18002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5918,"num_write_kb":23672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4862},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8933","reported_seq":9477,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:45.893964+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:45.893964+0000","last_peered":"2024-09-15T21:23:45.893964+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:45.893964+0000","last_undegraded":"2024-09-15T21:23:45.893964+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8933,"log_dups_size":0,"ondisk_log_size":8933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36573184,"num_objects":8929,"num_object_clones":0,"num_object_copies":17858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3876,"num_objects_unfound":0,"num_objects_dirty":8929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5417,"num_write_kb":21668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5053},{"shards":"0,2","objects":3876}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9124","reported_seq":9262,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.809765+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:45.809765+0000","last_peered":"2024-09-15T21:23:45.809765+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:45.809765+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9124,"log_dups_size":0,"ondisk_log_size":9124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37323718,"num_objects":9114,"num_object_clones":0,"num_object_copies":18228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9114,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5828,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4967},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":596251358,"num_objects":145578,"num_object_clones":0,"num_object_copies":291156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3876,"num_objects_unfound":0,"num_objects_dirty":145578,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":105443,"num_write_kb":421772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4371,"num_bytes_recovered":17895424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1210777600,"data_stored":1210629467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145650,"ondisk_log_size":145650,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699340,"kb_used_data":424700,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93672500,"statfs":{"total":96636764160,"available":95920640000,"internally_reserved":0,"allocated":434892800,"data_stored":434621225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798592,"kb_used_data":500400,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93573248,"statfs":{"total":96636764160,"available":95819005952,"internally_reserved":0,"allocated":512409600,"data_stored":512133948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305327787},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469104,"kb_used_data":256928,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93902736,"statfs":{"total":96636764160,"available":96156401664,"internally_reserved":0,"allocated":263094272,"data_stored":262778877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,1,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":512995328,"data_stored":512959198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":263098368,"data_stored":263022495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434683904,"data_stored":434647774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:50.715 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:50.762 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 199 20 147340 147320 2.89133 1.71875 0.0343842 0.0270131 2024-09-15T21:23:51.281 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:51.283 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:51.310 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-15T21:23:50.747858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596251377,"num_objects":145580,"num_object_clones":0,"num_object_copies":291160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3876,"num_objects_unfound":0,"num_objects_dirty":145580,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":105445,"num_write_kb":421774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":17895443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145653,"ondisk_log_size":145653,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1967036,"kb_used_data":1182028,"kb_used_omap":12,"kb_used_meta":784947,"kb_avail":281148484,"statfs":{"total":289910292480,"available":287896047616,"internally_reserved":0,"allocated":1210396672,"data_stored":1209534050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":803785731},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,9,13,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19005440,"num_objects":4640,"num_object_clones":0,"num_object_copies":9280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-167,"num_objects_unfound":0,"num_objects_dirty":4640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4640,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":684032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006690"},"pg_stats":[{"pgid":"2.f","version":"45'9306","reported_seq":9380,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.938279+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:45.938279+0000","last_peered":"2024-09-15T21:23:45.938279+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:45.938279+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9306,"log_dups_size":0,"ondisk_log_size":9306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38117376,"num_objects":9306,"num_object_clones":0,"num_object_copies":18612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9306,"num_write_kb":37224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5174},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9157","reported_seq":9262,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.485514+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:48.485514+0000","last_peered":"2024-09-15T21:23:48.485514+0000","last_clean":"2024-09-15T21:23:48.485514+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:48.485514+0000","last_undegraded":"2024-09-15T21:23:48.485514+0000","last_fullsized":"2024-09-15T21:23:48.485514+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9157,"log_dups_size":0,"ondisk_log_size":9157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37483462,"num_objects":9153,"num_object_clones":0,"num_object_copies":18306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9153,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7438,"num_write_kb":29752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9368","reported_seq":9493,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.453300+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:48.453300+0000","last_peered":"2024-09-15T21:23:48.453300+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:48.453300+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9368,"log_dups_size":0,"ondisk_log_size":9368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38354944,"num_objects":9364,"num_object_clones":0,"num_object_copies":18728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7468,"num_write_kb":29872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5097}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9004","reported_seq":9074,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.365994+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:48.365994+0000","last_peered":"2024-09-15T21:23:48.365994+0000","last_clean":"2024-09-15T21:23:48.365994+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:48.365994+0000","last_undegraded":"2024-09-15T21:23:48.365994+0000","last_fullsized":"2024-09-15T21:23:48.365994+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9004,"log_dups_size":0,"ondisk_log_size":9004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9004,"num_write_kb":36016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9000","reported_seq":9131,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:45.924718+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:45.924718+0000","last_peered":"2024-09-15T21:23:45.924718+0000","last_clean":"2024-09-15T21:23:45.924718+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:45.924718+0000","last_undegraded":"2024-09-15T21:23:45.924718+0000","last_fullsized":"2024-09-15T21:23:45.924718+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9000,"log_dups_size":0,"ondisk_log_size":9000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36835811,"num_objects":8994,"num_object_clones":0,"num_object_copies":17988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5455,"num_write_kb":21820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9012","reported_seq":9148,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.875427+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:45.875427+0000","last_peered":"2024-09-15T21:23:45.875427+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:45.875427+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9012,"log_dups_size":0,"ondisk_log_size":9012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36893155,"num_objects":9008,"num_object_clones":0,"num_object_copies":18016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9008,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5688,"num_write_kb":22752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4932},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9113","reported_seq":9248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.837457+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:45.837457+0000","last_peered":"2024-09-15T21:23:45.837457+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:45.837457+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9113,"log_dups_size":0,"ondisk_log_size":9113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37303238,"num_objects":9109,"num_object_clones":0,"num_object_copies":18218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5424,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4948},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9174","reported_seq":9302,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:45.777771+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:45.777771+0000","last_peered":"2024-09-15T21:23:45.777771+0000","last_clean":"2024-09-15T21:23:45.777771+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:45.777771+0000","last_undegraded":"2024-09-15T21:23:45.777771+0000","last_fullsized":"2024-09-15T21:23:45.777771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9174,"log_dups_size":0,"ondisk_log_size":9174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37568512,"num_objects":9172,"num_object_clones":0,"num_object_copies":18344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5946,"num_write_kb":23784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9117","reported_seq":9243,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.464523+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:48.464523+0000","last_peered":"2024-09-15T21:23:48.464523+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:48.464523+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9117,"log_dups_size":0,"ondisk_log_size":9117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37315043,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7049,"num_write_kb":28196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5024}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8959","reported_seq":9089,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.803849+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:45.803849+0000","last_peered":"2024-09-15T21:23:45.803849+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:45.803849+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8959,"log_dups_size":0,"ondisk_log_size":8959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663296,"num_objects":8951,"num_object_clones":0,"num_object_copies":17902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5632,"num_write_kb":22528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4931},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9160","reported_seq":9277,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.501081+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:48.501081+0000","last_peered":"2024-09-15T21:23:48.501081+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:48.501081+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9160,"log_dups_size":0,"ondisk_log_size":9160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37499363,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7019,"num_write_kb":28076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5066}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9167","reported_seq":17625,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.511962+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:48.511962+0000","last_peered":"2024-09-15T21:23:48.511962+0000","last_clean":"2024-09-15T21:23:48.511962+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:48.511962+0000","last_undegraded":"2024-09-15T21:23:48.511962+0000","last_fullsized":"2024-09-15T21:23:48.511962+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9167,"log_dups_size":0,"ondisk_log_size":9167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37523456,"num_objects":9161,"num_object_clones":0,"num_object_copies":18322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6860,"num_write_kb":27440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9047","reported_seq":9180,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.836969+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:45.836969+0000","last_peered":"2024-09-15T21:23:45.836969+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:45.836969+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9047,"log_dups_size":0,"ondisk_log_size":9047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5991,"num_write_kb":23964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4951},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9009","reported_seq":9141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.792121+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:45.792121+0000","last_peered":"2024-09-15T21:23:45.792121+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:45.792121+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9009,"log_dups_size":0,"ondisk_log_size":9009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36868096,"num_objects":9001,"num_object_clones":0,"num_object_copies":18002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5918,"num_write_kb":23672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4862},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8933","reported_seq":9477,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:45.893964+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:45.893964+0000","last_peered":"2024-09-15T21:23:45.893964+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:45.893964+0000","last_undegraded":"2024-09-15T21:23:45.893964+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8933,"log_dups_size":0,"ondisk_log_size":8933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36573184,"num_objects":8929,"num_object_clones":0,"num_object_copies":17858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3876,"num_objects_unfound":0,"num_objects_dirty":8929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5417,"num_write_kb":21668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5053},{"shards":"0,2","objects":3876}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9124","reported_seq":9262,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.809765+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:45.809765+0000","last_peered":"2024-09-15T21:23:45.809765+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:45.809765+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9124,"log_dups_size":0,"ondisk_log_size":9124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37323718,"num_objects":9114,"num_object_clones":0,"num_object_copies":18228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9114,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5828,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4967},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":596251358,"num_objects":145578,"num_object_clones":0,"num_object_copies":291156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3876,"num_objects_unfound":0,"num_objects_dirty":145578,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":105443,"num_write_kb":421772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4371,"num_bytes_recovered":17895424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1210777600,"data_stored":1210629467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145650,"ondisk_log_size":145650,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699340,"kb_used_data":424700,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93672500,"statfs":{"total":96636764160,"available":95920640000,"internally_reserved":0,"allocated":434892800,"data_stored":434621225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798592,"kb_used_data":500400,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93573248,"statfs":{"total":96636764160,"available":95819005952,"internally_reserved":0,"allocated":512409600,"data_stored":512133948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305327787},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469104,"kb_used_data":256928,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93902736,"statfs":{"total":96636764160,"available":96156401664,"internally_reserved":0,"allocated":263094272,"data_stored":262778877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,1,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":512995328,"data_stored":512959198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":263098368,"data_stored":263022495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434683904,"data_stored":434647774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:51.311 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:51.311 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'9306", 'reported_seq': 9380, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:45.938279+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:23:45.938279+0000', 'last_peered': '2024-09-15T21:23:45.938279+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:23:45.938279+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 9306, 'log_dups_size': 0, 'ondisk_log_size': 9306, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38117376, 'num_objects': 9306, 'num_object_clones': 0, 'num_object_copies': 18612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9306, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9306, 'num_write_kb': 37224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5174}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.312 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:23:51.312 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'9368", 'reported_seq': 9493, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:48.453300+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:23:48.453300+0000', 'last_peered': '2024-09-15T21:23:48.453300+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:23:48.453300+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 9368, 'log_dups_size': 0, 'ondisk_log_size': 9368, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38354944, 'num_objects': 9364, 'num_object_clones': 0, 'num_object_copies': 18728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9364, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7468, 'num_write_kb': 29872, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 5097}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:51.312 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:51.312 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'9012", 'reported_seq': 9148, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:45.875427+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:23:45.875427+0000', 'last_peered': '2024-09-15T21:23:45.875427+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:23:45.875427+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 9012, 'log_dups_size': 0, 'ondisk_log_size': 9012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36893155, 'num_objects': 9008, 'num_object_clones': 0, 'num_object_copies': 18016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9008, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5688, 'num_write_kb': 22752, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4932}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.312 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:51.312 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'9113", 'reported_seq': 9248, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:45.837457+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:23:45.837457+0000', 'last_peered': '2024-09-15T21:23:45.837457+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:23:45.837457+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 9113, 'log_dups_size': 0, 'ondisk_log_size': 9113, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37303238, 'num_objects': 9109, 'num_object_clones': 0, 'num_object_copies': 18218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9109, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5424, 'num_write_kb': 21696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4948}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.312 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:51.312 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'9117", 'reported_seq': 9243, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:48.464523+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:23:48.464523+0000', 'last_peered': '2024-09-15T21:23:48.464523+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:23:48.464523+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 9117, 'log_dups_size': 0, 'ondisk_log_size': 9117, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37315043, 'num_objects': 9111, 'num_object_clones': 0, 'num_object_copies': 18222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9111, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7049, 'num_write_kb': 28196, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 5024}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:51.312 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:51.313 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'8959", 'reported_seq': 9089, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:45.803849+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:23:45.803849+0000', 'last_peered': '2024-09-15T21:23:45.803849+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:23:45.803849+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 8959, 'log_dups_size': 0, 'ondisk_log_size': 8959, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36663296, 'num_objects': 8951, 'num_object_clones': 0, 'num_object_copies': 17902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5632, 'num_write_kb': 22528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4931}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.313 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:51.313 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'9160", 'reported_seq': 9277, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:48.501081+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:23:48.501081+0000', 'last_peered': '2024-09-15T21:23:48.501081+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:23:48.501081+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 9160, 'log_dups_size': 0, 'ondisk_log_size': 9160, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37499363, 'num_objects': 9156, 'num_object_clones': 0, 'num_object_copies': 18312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9156, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7019, 'num_write_kb': 28076, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 5066}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:51.313 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:51.313 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'9047", 'reported_seq': 9180, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:45.836969+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:23:45.836969+0000', 'last_peered': '2024-09-15T21:23:45.836969+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:23:45.836969+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 9047, 'log_dups_size': 0, 'ondisk_log_size': 9047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37048320, 'num_objects': 9045, 'num_object_clones': 0, 'num_object_copies': 18090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5991, 'num_write_kb': 23964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4951}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.313 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:51.313 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'9009", 'reported_seq': 9141, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:45.792121+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:23:45.792121+0000', 'last_peered': '2024-09-15T21:23:45.792121+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:23:45.792121+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 9009, 'log_dups_size': 0, 'ondisk_log_size': 9009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36868096, 'num_objects': 9001, 'num_object_clones': 0, 'num_object_copies': 18002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9001, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5918, 'num_write_kb': 23672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4862}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.313 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:51.313 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'8933", 'reported_seq': 9477, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:23:45.893964+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:23:45.893964+0000', 'last_peered': '2024-09-15T21:23:45.893964+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:23:45.893964+0000', 'last_undegraded': '2024-09-15T21:23:45.893964+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 8933, 'log_dups_size': 0, 'ondisk_log_size': 8933, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36573184, 'num_objects': 8929, 'num_object_clones': 0, 'num_object_copies': 17858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3876, 'num_objects_unfound': 0, 'num_objects_dirty': 8929, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5417, 'num_write_kb': 21668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 206, 'num_bytes_recovered': 843776, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5053}, {'shards': '0,2', 'objects': 3876}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.313 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:23:51.313 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'9124", 'reported_seq': 9262, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:45.809765+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:23:45.809765+0000', 'last_peered': '2024-09-15T21:23:45.809765+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:23:45.809765+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 9124, 'log_dups_size': 0, 'ondisk_log_size': 9124, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37323718, 'num_objects': 9114, 'num_object_clones': 0, 'num_object_copies': 18228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9114, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5828, 'num_write_kb': 23312, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4967}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:51.314 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:51.764 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:2024-09-15T21:23:51.760301+0000 min lat: 0.00235499 max lat: 2.60418 avg lat: 0.0270719 2024-09-15T21:23:51.764 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:23:51.764 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 200 20 147766 147746 2.8852 1.66406 0.0124288 0.0270719 2024-09-15T21:23:51.999 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:51.999 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/291160 objects degraded (14.159%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":283,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":145580,"data_bytes":596251377,"bytes_used":2014244864,"bytes_avail":287896047616,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":291160,"degraded_ratio":0.14158881714521226,"misplaced_objects":3876,"misplaced_total":291160,"misplaced_ratio":0.01331226816870449,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":56970,"recovering_keys_per_sec":0,"num_objects_recovered":167,"num_bytes_recovered":684032,"num_keys_recovered":0,"write_bytes_sec":1582904,"read_op_per_sec":0,"write_op_per_sec":386},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:52.024 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:52.024 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:52.632 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:52.634 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:52.664 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-15T21:23:50.747858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596251377,"num_objects":145580,"num_object_clones":0,"num_object_copies":291160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3876,"num_objects_unfound":0,"num_objects_dirty":145580,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":105445,"num_write_kb":421774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4372,"num_bytes_recovered":17895443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145653,"ondisk_log_size":145653,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1967036,"kb_used_data":1182028,"kb_used_omap":12,"kb_used_meta":784947,"kb_avail":281148484,"statfs":{"total":289910292480,"available":287896047616,"internally_reserved":0,"allocated":1210396672,"data_stored":1209534050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":803785731},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,9,13,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19005440,"num_objects":4640,"num_object_clones":0,"num_object_copies":9280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-167,"num_objects_unfound":0,"num_objects_dirty":4640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4640,"num_write_kb":18560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":684032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006690"},"pg_stats":[{"pgid":"2.f","version":"45'9306","reported_seq":9380,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.938279+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:45.938279+0000","last_peered":"2024-09-15T21:23:45.938279+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:45.938279+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9306,"log_dups_size":0,"ondisk_log_size":9306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38117376,"num_objects":9306,"num_object_clones":0,"num_object_copies":18612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9306,"num_write_kb":37224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5174},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9157","reported_seq":9262,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.485514+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:48.485514+0000","last_peered":"2024-09-15T21:23:48.485514+0000","last_clean":"2024-09-15T21:23:48.485514+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:48.485514+0000","last_undegraded":"2024-09-15T21:23:48.485514+0000","last_fullsized":"2024-09-15T21:23:48.485514+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9157,"log_dups_size":0,"ondisk_log_size":9157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37483462,"num_objects":9153,"num_object_clones":0,"num_object_copies":18306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9153,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7438,"num_write_kb":29752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9368","reported_seq":9493,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.453300+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:48.453300+0000","last_peered":"2024-09-15T21:23:48.453300+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:48.453300+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9368,"log_dups_size":0,"ondisk_log_size":9368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38354944,"num_objects":9364,"num_object_clones":0,"num_object_copies":18728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7468,"num_write_kb":29872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5097}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9004","reported_seq":9074,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.365994+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:48.365994+0000","last_peered":"2024-09-15T21:23:48.365994+0000","last_clean":"2024-09-15T21:23:48.365994+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:48.365994+0000","last_undegraded":"2024-09-15T21:23:48.365994+0000","last_fullsized":"2024-09-15T21:23:48.365994+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9004,"log_dups_size":0,"ondisk_log_size":9004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880384,"num_objects":9004,"num_object_clones":0,"num_object_copies":18008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9004,"num_write_kb":36016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9000","reported_seq":9131,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:45.924718+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:45.924718+0000","last_peered":"2024-09-15T21:23:45.924718+0000","last_clean":"2024-09-15T21:23:45.924718+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:45.924718+0000","last_undegraded":"2024-09-15T21:23:45.924718+0000","last_fullsized":"2024-09-15T21:23:45.924718+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9000,"log_dups_size":0,"ondisk_log_size":9000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36835811,"num_objects":8994,"num_object_clones":0,"num_object_copies":17988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8994,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5455,"num_write_kb":21820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9012","reported_seq":9148,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.875427+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:45.875427+0000","last_peered":"2024-09-15T21:23:45.875427+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:45.875427+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9012,"log_dups_size":0,"ondisk_log_size":9012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36893155,"num_objects":9008,"num_object_clones":0,"num_object_copies":18016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9008,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5688,"num_write_kb":22752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4932},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9113","reported_seq":9248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.837457+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:45.837457+0000","last_peered":"2024-09-15T21:23:45.837457+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:45.837457+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9113,"log_dups_size":0,"ondisk_log_size":9113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37303238,"num_objects":9109,"num_object_clones":0,"num_object_copies":18218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5424,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4948},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9174","reported_seq":9302,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:45.777771+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:45.777771+0000","last_peered":"2024-09-15T21:23:45.777771+0000","last_clean":"2024-09-15T21:23:45.777771+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:45.777771+0000","last_undegraded":"2024-09-15T21:23:45.777771+0000","last_fullsized":"2024-09-15T21:23:45.777771+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9174,"log_dups_size":0,"ondisk_log_size":9174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37568512,"num_objects":9172,"num_object_clones":0,"num_object_copies":18344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9172,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5946,"num_write_kb":23784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9117","reported_seq":9243,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.464523+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:48.464523+0000","last_peered":"2024-09-15T21:23:48.464523+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:48.464523+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9117,"log_dups_size":0,"ondisk_log_size":9117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37315043,"num_objects":9111,"num_object_clones":0,"num_object_copies":18222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7049,"num_write_kb":28196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5024}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8959","reported_seq":9089,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.803849+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:45.803849+0000","last_peered":"2024-09-15T21:23:45.803849+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:45.803849+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":8959,"log_dups_size":0,"ondisk_log_size":8959,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36663296,"num_objects":8951,"num_object_clones":0,"num_object_copies":17902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5632,"num_write_kb":22528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4931},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9160","reported_seq":9277,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:48.501081+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:48.501081+0000","last_peered":"2024-09-15T21:23:48.501081+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:48.501081+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9160,"log_dups_size":0,"ondisk_log_size":9160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37499363,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7019,"num_write_kb":28076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5066}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9167","reported_seq":17625,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:48.511962+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:48.511962+0000","last_peered":"2024-09-15T21:23:48.511962+0000","last_clean":"2024-09-15T21:23:48.511962+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:48.511962+0000","last_undegraded":"2024-09-15T21:23:48.511962+0000","last_fullsized":"2024-09-15T21:23:48.511962+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9167,"log_dups_size":0,"ondisk_log_size":9167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37523456,"num_objects":9161,"num_object_clones":0,"num_object_copies":18322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6860,"num_write_kb":27440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9047","reported_seq":9180,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.836969+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:45.836969+0000","last_peered":"2024-09-15T21:23:45.836969+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:45.836969+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9047,"log_dups_size":0,"ondisk_log_size":9047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37048320,"num_objects":9045,"num_object_clones":0,"num_object_copies":18090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5991,"num_write_kb":23964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4951},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9009","reported_seq":9141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.792121+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:45.792121+0000","last_peered":"2024-09-15T21:23:45.792121+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:45.792121+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9009,"log_dups_size":0,"ondisk_log_size":9009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36868096,"num_objects":9001,"num_object_clones":0,"num_object_copies":18002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5918,"num_write_kb":23672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4862},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8933","reported_seq":9477,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:45.893964+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:45.893964+0000","last_peered":"2024-09-15T21:23:45.893964+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:45.893964+0000","last_undegraded":"2024-09-15T21:23:45.893964+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":8933,"log_dups_size":0,"ondisk_log_size":8933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36573184,"num_objects":8929,"num_object_clones":0,"num_object_copies":17858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3876,"num_objects_unfound":0,"num_objects_dirty":8929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5417,"num_write_kb":21668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5053},{"shards":"0,2","objects":3876}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9124","reported_seq":9262,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:45.809765+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:45.809765+0000","last_peered":"2024-09-15T21:23:45.809765+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:45.809765+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9124,"log_dups_size":0,"ondisk_log_size":9124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37323718,"num_objects":9114,"num_object_clones":0,"num_object_copies":18228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9114,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5828,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4967},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":596251358,"num_objects":145578,"num_object_clones":0,"num_object_copies":291156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3876,"num_objects_unfound":0,"num_objects_dirty":145578,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":105443,"num_write_kb":421772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4371,"num_bytes_recovered":17895424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1210777600,"data_stored":1210629467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145650,"ondisk_log_size":145650,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699340,"kb_used_data":424700,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93672500,"statfs":{"total":96636764160,"available":95920640000,"internally_reserved":0,"allocated":434892800,"data_stored":434621225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":798592,"kb_used_data":500400,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93573248,"statfs":{"total":96636764160,"available":95819005952,"internally_reserved":0,"allocated":512409600,"data_stored":512133948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305327787},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469104,"kb_used_data":256928,"kb_used_omap":4,"kb_used_meta":212155,"kb_avail":93902736,"statfs":{"total":96636764160,"available":96156401664,"internally_reserved":0,"allocated":263094272,"data_stored":262778877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":217247407},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,1,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":512995328,"data_stored":512959198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":263098368,"data_stored":263022495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434683904,"data_stored":434647774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:52.762 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 201 20 148184 148164 2.87897 1.63281 0.044246 0.02713 2024-09-15T21:23:53.762 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 202 20 148646 148626 2.87365 1.80469 0.0213578 0.0271776 2024-09-15T21:23:54.762 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 203 20 149125 149105 2.86871 1.87109 0.0268462 0.0272259 2024-09-15T21:23:55.667 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:55.762 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 204 20 149590 149570 2.86355 1.81641 0.0377129 0.0272743 2024-09-15T21:23:56.218 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:56.221 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:56.241 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-15T21:23:54.748798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605455089,"num_objects":147827,"num_object_clones":0,"num_object_copies":295654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3798,"num_objects_unfound":0,"num_objects_dirty":147827,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":107692,"num_write_kb":430762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4450,"num_bytes_recovered":18214931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147900,"ondisk_log_size":147900,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1981260,"kb_used_data":1200412,"kb_used_omap":12,"kb_used_meta":780787,"kb_avail":281134260,"statfs":{"total":289910292480,"available":287881482240,"internally_reserved":0,"allocated":1229221888,"data_stored":1228359266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":799525891},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,6,9,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22216704,"num_objects":5424,"num_object_clones":0,"num_object_copies":10848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-158,"num_objects_unfound":0,"num_objects_dirty":5424,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5424,"num_write_kb":21696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":158,"num_bytes_recovered":647168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002867"},"pg_stats":[{"pgid":"2.f","version":"45'9448","reported_seq":9522,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:50.911481+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:50.911481+0000","last_peered":"2024-09-15T21:23:50.911481+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:50.911481+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9448,"log_dups_size":0,"ondisk_log_size":9448,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38699008,"num_objects":9448,"num_object_clones":0,"num_object_copies":18896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9448,"num_write_kb":37792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5316},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9293","reported_seq":9398,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.458185+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:53.458185+0000","last_peered":"2024-09-15T21:23:53.458185+0000","last_clean":"2024-09-15T21:23:53.458185+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:53.458185+0000","last_undegraded":"2024-09-15T21:23:53.458185+0000","last_fullsized":"2024-09-15T21:23:53.458185+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38040518,"num_objects":9289,"num_object_clones":0,"num_object_copies":18578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9289,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7574,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9501","reported_seq":9626,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.417966+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:53.417966+0000","last_peered":"2024-09-15T21:23:53.417966+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:53.417966+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9501,"log_dups_size":0,"ondisk_log_size":9501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38899712,"num_objects":9497,"num_object_clones":0,"num_object_copies":18994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7601,"num_write_kb":30404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5230}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9142","reported_seq":9212,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.468362+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:53.468362+0000","last_peered":"2024-09-15T21:23:53.468362+0000","last_clean":"2024-09-15T21:23:53.468362+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:53.468362+0000","last_undegraded":"2024-09-15T21:23:53.468362+0000","last_fullsized":"2024-09-15T21:23:53.468362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9142,"log_dups_size":0,"ondisk_log_size":9142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37445632,"num_objects":9142,"num_object_clones":0,"num_object_copies":18284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9142,"num_write_kb":36568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9169","reported_seq":9301,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:50.929587+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:50.929587+0000","last_peered":"2024-09-15T21:23:50.929587+0000","last_clean":"2024-09-15T21:23:50.929587+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:50.929587+0000","last_undegraded":"2024-09-15T21:23:50.929587+0000","last_fullsized":"2024-09-15T21:23:50.929587+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9169,"log_dups_size":0,"ondisk_log_size":9169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37528035,"num_objects":9163,"num_object_clones":0,"num_object_copies":18326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5624,"num_write_kb":22496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9162","reported_seq":9298,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:50.980316+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:50.980316+0000","last_peered":"2024-09-15T21:23:50.980316+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:50.980316+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9162,"log_dups_size":0,"ondisk_log_size":9162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37507555,"num_objects":9158,"num_object_clones":0,"num_object_copies":18316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9158,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5838,"num_write_kb":23352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5082},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9251","reported_seq":9387,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:50.968339+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:50.968339+0000","last_peered":"2024-09-15T21:23:50.968339+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:50.968339+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9251,"log_dups_size":0,"ondisk_log_size":9251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37868486,"num_objects":9247,"num_object_clones":0,"num_object_copies":18494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5562,"num_write_kb":22248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5086},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9315","reported_seq":9443,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:50.815430+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:50.815430+0000","last_peered":"2024-09-15T21:23:50.815430+0000","last_clean":"2024-09-15T21:23:50.815430+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:50.815430+0000","last_undegraded":"2024-09-15T21:23:50.815430+0000","last_fullsized":"2024-09-15T21:23:50.815430+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9315,"log_dups_size":0,"ondisk_log_size":9315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38146048,"num_objects":9313,"num_object_clones":0,"num_object_copies":18626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6087,"num_write_kb":24348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9256","reported_seq":9381,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.517813+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:53.517813+0000","last_peered":"2024-09-15T21:23:53.517813+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:53.517813+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9256,"log_dups_size":0,"ondisk_log_size":9256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37884387,"num_objects":9250,"num_object_clones":0,"num_object_copies":18500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7188,"num_write_kb":28752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5163}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9091","reported_seq":9221,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:50.858037+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:50.858037+0000","last_peered":"2024-09-15T21:23:50.858037+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:50.858037+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9091,"log_dups_size":0,"ondisk_log_size":9091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37203968,"num_objects":9083,"num_object_clones":0,"num_object_copies":18166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5764,"num_write_kb":23056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5063},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9298","reported_seq":9416,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.397883+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:53.397883+0000","last_peered":"2024-09-15T21:23:53.397883+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:53.397883+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9298,"log_dups_size":0,"ondisk_log_size":9298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38064611,"num_objects":9294,"num_object_clones":0,"num_object_copies":18588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9294,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7157,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5204}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9273","reported_seq":17731,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.447728+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:53.447728+0000","last_peered":"2024-09-15T21:23:53.447728+0000","last_clean":"2024-09-15T21:23:53.447728+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:53.447728+0000","last_undegraded":"2024-09-15T21:23:53.447728+0000","last_fullsized":"2024-09-15T21:23:53.447728+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9273,"log_dups_size":0,"ondisk_log_size":9273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37957632,"num_objects":9267,"num_object_clones":0,"num_object_copies":18534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6966,"num_write_kb":27864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9193","reported_seq":9322,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:50.925263+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:50.925263+0000","last_peered":"2024-09-15T21:23:50.925263+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:50.925263+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9193,"log_dups_size":0,"ondisk_log_size":9193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37646336,"num_objects":9191,"num_object_clones":0,"num_object_copies":18382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6137,"num_write_kb":24548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5097},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9153","reported_seq":9284,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:51.006315+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:51.006315+0000","last_peered":"2024-09-15T21:23:51.006315+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:51.006315+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9153,"log_dups_size":0,"ondisk_log_size":9153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37457920,"num_objects":9145,"num_object_clones":0,"num_object_copies":18290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6062,"num_write_kb":24248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5006},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9080","reported_seq":9783,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:51.002336+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:51.002336+0000","last_peered":"2024-09-15T21:23:51.002336+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:51.002336+0000","last_undegraded":"2024-09-15T21:23:51.002336+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9080,"log_dups_size":0,"ondisk_log_size":9080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37175296,"num_objects":9076,"num_object_clones":0,"num_object_copies":18152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3798,"num_objects_unfound":0,"num_objects_dirty":9076,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5564,"num_write_kb":22256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1163264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5278},{"shards":"0,2","objects":3798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9272","reported_seq":9410,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:50.887254+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:50.887254+0000","last_peered":"2024-09-15T21:23:50.887254+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:50.887254+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9272,"log_dups_size":0,"ondisk_log_size":9272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37929926,"num_objects":9262,"num_object_clones":0,"num_object_copies":18524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9262,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5976,"num_write_kb":23904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5115},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":605455070,"num_objects":147825,"num_object_clones":0,"num_object_copies":295650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3798,"num_objects_unfound":0,"num_objects_dirty":147825,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":107690,"num_write_kb":430760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4449,"num_bytes_recovered":18214912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1229697024,"data_stored":1229548891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147897,"ondisk_log_size":147897,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691196,"kb_used_data":429036,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93680644,"statfs":{"total":96636764160,"available":95928979456,"internally_reserved":0,"allocated":439332864,"data_stored":439061289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182451,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":809816,"kb_used_data":507464,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93562024,"statfs":{"total":96636764160,"available":95807512576,"internally_reserved":0,"allocated":519643136,"data_stored":519367484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309587627},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480248,"kb_used_data":263912,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93891592,"statfs":{"total":96636764160,"available":96144990208,"internally_reserved":0,"allocated":270245888,"data_stored":269930493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221507247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":520220672,"data_stored":520184542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":270352384,"data_stored":270276511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":439123968,"data_stored":439087838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:56.242 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:56.763 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 205 19 150009 149990 2.85758 1.64062 0.0343954 0.0273321 2024-09-15T21:23:56.793 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:56.796 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:56.818 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-15T21:23:56.749258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":611394289,"num_objects":149277,"num_object_clones":0,"num_object_copies":298554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":149277,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":109142,"num_write_kb":436562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4522,"num_bytes_recovered":18509843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149350,"ondisk_log_size":149350,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1992032,"kb_used_data":1207024,"kb_used_omap":12,"kb_used_meta":784947,"kb_avail":281123488,"statfs":{"total":289910292480,"available":287870451712,"internally_reserved":0,"allocated":1235992576,"data_stored":1235129954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":803785731},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,5,9,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24772608,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-230,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6048,"num_write_kb":24192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":942080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002821"},"pg_stats":[{"pgid":"2.f","version":"45'9599","reported_seq":9674,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.971843+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:55.971843+0000","last_peered":"2024-09-15T21:23:55.971843+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:55.971843+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9599,"log_dups_size":0,"ondisk_log_size":9599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39317504,"num_objects":9599,"num_object_clones":0,"num_object_copies":19198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":38396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5467},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9293","reported_seq":9398,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.458185+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:53.458185+0000","last_peered":"2024-09-15T21:23:53.458185+0000","last_clean":"2024-09-15T21:23:53.458185+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:53.458185+0000","last_undegraded":"2024-09-15T21:23:53.458185+0000","last_fullsized":"2024-09-15T21:23:53.458185+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38040518,"num_objects":9289,"num_object_clones":0,"num_object_copies":18578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9289,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7574,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9501","reported_seq":9626,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.417966+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:53.417966+0000","last_peered":"2024-09-15T21:23:53.417966+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:53.417966+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9501,"log_dups_size":0,"ondisk_log_size":9501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38899712,"num_objects":9497,"num_object_clones":0,"num_object_copies":18994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7601,"num_write_kb":30404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5230}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9142","reported_seq":9212,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.468362+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:53.468362+0000","last_peered":"2024-09-15T21:23:53.468362+0000","last_clean":"2024-09-15T21:23:53.468362+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:53.468362+0000","last_undegraded":"2024-09-15T21:23:53.468362+0000","last_fullsized":"2024-09-15T21:23:53.468362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9142,"log_dups_size":0,"ondisk_log_size":9142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37445632,"num_objects":9142,"num_object_clones":0,"num_object_copies":18284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9142,"num_write_kb":36568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9302","reported_seq":9434,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:56.036822+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:56.036822+0000","last_peered":"2024-09-15T21:23:56.036822+0000","last_clean":"2024-09-15T21:23:56.036822+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:56.036822+0000","last_undegraded":"2024-09-15T21:23:56.036822+0000","last_fullsized":"2024-09-15T21:23:56.036822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9302,"log_dups_size":0,"ondisk_log_size":9302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072803,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5757,"num_write_kb":23028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9312","reported_seq":9448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.997027+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:55.997027+0000","last_peered":"2024-09-15T21:23:55.997027+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:55.997027+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9312,"log_dups_size":0,"ondisk_log_size":9312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38121955,"num_objects":9308,"num_object_clones":0,"num_object_copies":18616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5988,"num_write_kb":23952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5232},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9390","reported_seq":9523,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:56.003121+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:56.003121+0000","last_peered":"2024-09-15T21:23:56.003121+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:56.003121+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9390,"log_dups_size":0,"ondisk_log_size":9390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38437830,"num_objects":9386,"num_object_clones":0,"num_object_copies":18772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5701,"num_write_kb":22804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5225},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9444","reported_seq":9572,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:55.923159+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:55.923159+0000","last_peered":"2024-09-15T21:23:55.923159+0000","last_clean":"2024-09-15T21:23:55.923159+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:55.923159+0000","last_undegraded":"2024-09-15T21:23:55.923159+0000","last_fullsized":"2024-09-15T21:23:55.923159+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38674432,"num_objects":9442,"num_object_clones":0,"num_object_copies":18884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6216,"num_write_kb":24864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9256","reported_seq":9381,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.517813+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:53.517813+0000","last_peered":"2024-09-15T21:23:53.517813+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:53.517813+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9256,"log_dups_size":0,"ondisk_log_size":9256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37884387,"num_objects":9250,"num_object_clones":0,"num_object_copies":18500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7188,"num_write_kb":28752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5163}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9239","reported_seq":9369,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:56.042095+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:56.042095+0000","last_peered":"2024-09-15T21:23:56.042095+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:56.042095+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9239,"log_dups_size":0,"ondisk_log_size":9239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37810176,"num_objects":9231,"num_object_clones":0,"num_object_copies":18462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5912,"num_write_kb":23648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5211},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9298","reported_seq":9416,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.397883+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:53.397883+0000","last_peered":"2024-09-15T21:23:53.397883+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:53.397883+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9298,"log_dups_size":0,"ondisk_log_size":9298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38064611,"num_objects":9294,"num_object_clones":0,"num_object_copies":18588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9294,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7157,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5204}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9273","reported_seq":17731,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.447728+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:53.447728+0000","last_peered":"2024-09-15T21:23:53.447728+0000","last_clean":"2024-09-15T21:23:53.447728+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:53.447728+0000","last_undegraded":"2024-09-15T21:23:53.447728+0000","last_fullsized":"2024-09-15T21:23:53.447728+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9273,"log_dups_size":0,"ondisk_log_size":9273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37957632,"num_objects":9267,"num_object_clones":0,"num_object_copies":18534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6966,"num_write_kb":27864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9341","reported_seq":9474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.981001+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:55.981001+0000","last_peered":"2024-09-15T21:23:55.981001+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:55.981001+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9341,"log_dups_size":0,"ondisk_log_size":9341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38252544,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6285,"num_write_kb":25140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5245},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9304","reported_seq":9436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.947146+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:55.947146+0000","last_peered":"2024-09-15T21:23:55.947146+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:55.947146+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9304,"log_dups_size":0,"ondisk_log_size":9304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38076416,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6213,"num_write_kb":24852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5157},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9222","reported_seq":10069,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:56.037755+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:56.037755+0000","last_peered":"2024-09-15T21:23:56.037755+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:56.037755+0000","last_undegraded":"2024-09-15T21:23:56.037755+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9222,"log_dups_size":0,"ondisk_log_size":9222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37756928,"num_objects":9218,"num_object_clones":0,"num_object_copies":18436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":9218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5706,"num_write_kb":22824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":1458176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5492},{"shards":"0,2","objects":3726}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9431","reported_seq":9567,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:56.028520+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:56.028520+0000","last_peered":"2024-09-15T21:23:56.028520+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:56.028520+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9431,"log_dups_size":0,"ondisk_log_size":9431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38581190,"num_objects":9421,"num_object_clones":0,"num_object_copies":18842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9421,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6135,"num_write_kb":24540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5274},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":611394270,"num_objects":149275,"num_object_clones":0,"num_object_copies":298550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":149275,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":109140,"num_write_kb":436560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4521,"num_bytes_recovered":18509824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1236705280,"data_stored":1236557147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149347,"ondisk_log_size":149347,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691196,"kb_used_data":429036,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93680644,"statfs":{"total":96636764160,"available":95928979456,"internally_reserved":0,"allocated":439332864,"data_stored":439061289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820588,"kb_used_data":514076,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93551252,"statfs":{"total":96636764160,"available":95796482048,"internally_reserved":0,"allocated":526413824,"data_stored":526138172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480248,"kb_used_data":263912,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93891592,"statfs":{"total":96636764160,"available":96144990208,"internally_reserved":0,"allocated":270245888,"data_stored":269930493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221507247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527228928,"data_stored":527192798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":270352384,"data_stored":270276511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":439123968,"data_stored":439087838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:56.819 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:23:56.819 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'9599", 'reported_seq': 9674, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:55.971843+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:23:55.971843+0000', 'last_peered': '2024-09-15T21:23:55.971843+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:23:55.971843+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 9599, 'log_dups_size': 0, 'ondisk_log_size': 9599, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39317504, 'num_objects': 9599, 'num_object_clones': 0, 'num_object_copies': 19198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9599, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9599, 'num_write_kb': 38396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5467}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:56.819 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:23:56.819 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'9501", 'reported_seq': 9626, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:53.417966+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:23:53.417966+0000', 'last_peered': '2024-09-15T21:23:53.417966+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:23:53.417966+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 9501, 'log_dups_size': 0, 'ondisk_log_size': 9501, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38899712, 'num_objects': 9497, 'num_object_clones': 0, 'num_object_copies': 18994, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9497, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7601, 'num_write_kb': 30404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 5230}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:56.819 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:23:56.819 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'9312", 'reported_seq': 9448, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:55.997027+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:23:55.997027+0000', 'last_peered': '2024-09-15T21:23:55.997027+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:23:55.997027+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 9312, 'log_dups_size': 0, 'ondisk_log_size': 9312, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38121955, 'num_objects': 9308, 'num_object_clones': 0, 'num_object_copies': 18616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9308, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5988, 'num_write_kb': 23952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5232}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:56.820 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:23:56.820 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'9390", 'reported_seq': 9523, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:56.003121+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:23:56.003121+0000', 'last_peered': '2024-09-15T21:23:56.003121+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:23:56.003121+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 9390, 'log_dups_size': 0, 'ondisk_log_size': 9390, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38437830, 'num_objects': 9386, 'num_object_clones': 0, 'num_object_copies': 18772, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9386, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5701, 'num_write_kb': 22804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5225}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:56.820 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:23:56.820 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'9256", 'reported_seq': 9381, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:53.517813+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:23:53.517813+0000', 'last_peered': '2024-09-15T21:23:53.517813+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:23:53.517813+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 9256, 'log_dups_size': 0, 'ondisk_log_size': 9256, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37884387, 'num_objects': 9250, 'num_object_clones': 0, 'num_object_copies': 18500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9250, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7188, 'num_write_kb': 28752, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 5163}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:56.820 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:23:56.820 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'9239", 'reported_seq': 9369, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:56.042095+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:23:56.042095+0000', 'last_peered': '2024-09-15T21:23:56.042095+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:23:56.042095+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 9239, 'log_dups_size': 0, 'ondisk_log_size': 9239, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37810176, 'num_objects': 9231, 'num_object_clones': 0, 'num_object_copies': 18462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9231, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5912, 'num_write_kb': 23648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5211}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:56.820 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:23:56.820 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'9298", 'reported_seq': 9416, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:53.397883+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:23:53.397883+0000', 'last_peered': '2024-09-15T21:23:53.397883+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:23:53.397883+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 9298, 'log_dups_size': 0, 'ondisk_log_size': 9298, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38064611, 'num_objects': 9294, 'num_object_clones': 0, 'num_object_copies': 18588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9294, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7157, 'num_write_kb': 28628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 5204}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:23:56.820 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:23:56.820 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'9341", 'reported_seq': 9474, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:55.981001+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:23:55.981001+0000', 'last_peered': '2024-09-15T21:23:55.981001+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:23:55.981001+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 9341, 'log_dups_size': 0, 'ondisk_log_size': 9341, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38252544, 'num_objects': 9339, 'num_object_clones': 0, 'num_object_copies': 18678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9339, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6285, 'num_write_kb': 25140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5245}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:56.821 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:23:56.821 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'9304", 'reported_seq': 9436, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:55.947146+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:23:55.947146+0000', 'last_peered': '2024-09-15T21:23:55.947146+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:23:55.947146+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 9304, 'log_dups_size': 0, 'ondisk_log_size': 9304, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38076416, 'num_objects': 9296, 'num_object_clones': 0, 'num_object_copies': 18592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6213, 'num_write_kb': 24852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5157}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:56.821 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:23:56.821 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'9222", 'reported_seq': 10069, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:23:56.037755+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:23:56.037755+0000', 'last_peered': '2024-09-15T21:23:56.037755+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:23:56.037755+0000', 'last_undegraded': '2024-09-15T21:23:56.037755+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 9222, 'log_dups_size': 0, 'ondisk_log_size': 9222, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37756928, 'num_objects': 9218, 'num_object_clones': 0, 'num_object_copies': 18436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3726, 'num_objects_unfound': 0, 'num_objects_dirty': 9218, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5706, 'num_write_kb': 22824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 356, 'num_bytes_recovered': 1458176, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5492}, {'shards': '0,2', 'objects': 3726}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:56.821 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:23:56.821 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'9431", 'reported_seq': 9567, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:56.028520+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:23:56.028520+0000', 'last_peered': '2024-09-15T21:23:56.028520+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:23:56.028520+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 9431, 'log_dups_size': 0, 'ondisk_log_size': 9431, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38581190, 'num_objects': 9421, 'num_object_clones': 0, 'num_object_copies': 18842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9421, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6135, 'num_write_kb': 24540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5274}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:23:56.821 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:23:57.515 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:57.516 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/298554 objects degraded (13.808%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":289,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":149277,"data_bytes":611394289,"bytes_used":2039840768,"bytes_avail":287870451712,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":298554,"degraded_ratio":0.13808222298143719,"misplaced_objects":3726,"misplaced_total":298554,"misplaced_ratio":0.01248015434393778,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":78488,"recovering_keys_per_sec":0,"num_objects_recovered":230,"num_bytes_recovered":942080,"num_keys_recovered":0,"write_bytes_sec":2063898,"read_op_per_sec":0,"write_op_per_sec":503},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:23:57.543 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:23:57.543 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:23:57.762 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 206 20 150428 150408 2.85164 1.63281 0.0685071 0.0273895 2024-09-15T21:23:58.102 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:23:58.105 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:23:58.129 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-15T21:23:56.749258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":611394289,"num_objects":149277,"num_object_clones":0,"num_object_copies":298554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":149277,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":109142,"num_write_kb":436562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4522,"num_bytes_recovered":18509843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149350,"ondisk_log_size":149350,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1992032,"kb_used_data":1207024,"kb_used_omap":12,"kb_used_meta":784947,"kb_avail":281123488,"statfs":{"total":289910292480,"available":287870451712,"internally_reserved":0,"allocated":1235992576,"data_stored":1235129954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":803785731},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,5,9,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24772608,"num_objects":6048,"num_object_clones":0,"num_object_copies":12096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-230,"num_objects_unfound":0,"num_objects_dirty":6048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6048,"num_write_kb":24192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":942080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002821"},"pg_stats":[{"pgid":"2.f","version":"45'9599","reported_seq":9674,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.971843+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:55.971843+0000","last_peered":"2024-09-15T21:23:55.971843+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:55.971843+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9599,"log_dups_size":0,"ondisk_log_size":9599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39317504,"num_objects":9599,"num_object_clones":0,"num_object_copies":19198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":38396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5467},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9293","reported_seq":9398,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.458185+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:53.458185+0000","last_peered":"2024-09-15T21:23:53.458185+0000","last_clean":"2024-09-15T21:23:53.458185+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:53.458185+0000","last_undegraded":"2024-09-15T21:23:53.458185+0000","last_fullsized":"2024-09-15T21:23:53.458185+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9293,"log_dups_size":0,"ondisk_log_size":9293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38040518,"num_objects":9289,"num_object_clones":0,"num_object_copies":18578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9289,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7574,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9501","reported_seq":9626,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.417966+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:53.417966+0000","last_peered":"2024-09-15T21:23:53.417966+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:53.417966+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9501,"log_dups_size":0,"ondisk_log_size":9501,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38899712,"num_objects":9497,"num_object_clones":0,"num_object_copies":18994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7601,"num_write_kb":30404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5230}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9142","reported_seq":9212,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.468362+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:53.468362+0000","last_peered":"2024-09-15T21:23:53.468362+0000","last_clean":"2024-09-15T21:23:53.468362+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:53.468362+0000","last_undegraded":"2024-09-15T21:23:53.468362+0000","last_fullsized":"2024-09-15T21:23:53.468362+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9142,"log_dups_size":0,"ondisk_log_size":9142,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37445632,"num_objects":9142,"num_object_clones":0,"num_object_copies":18284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9142,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9142,"num_write_kb":36568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9302","reported_seq":9434,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:56.036822+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:56.036822+0000","last_peered":"2024-09-15T21:23:56.036822+0000","last_clean":"2024-09-15T21:23:56.036822+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:56.036822+0000","last_undegraded":"2024-09-15T21:23:56.036822+0000","last_fullsized":"2024-09-15T21:23:56.036822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9302,"log_dups_size":0,"ondisk_log_size":9302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072803,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5757,"num_write_kb":23028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9312","reported_seq":9448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.997027+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:55.997027+0000","last_peered":"2024-09-15T21:23:55.997027+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:55.997027+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9312,"log_dups_size":0,"ondisk_log_size":9312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38121955,"num_objects":9308,"num_object_clones":0,"num_object_copies":18616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5988,"num_write_kb":23952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5232},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9390","reported_seq":9523,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:56.003121+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:56.003121+0000","last_peered":"2024-09-15T21:23:56.003121+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:56.003121+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9390,"log_dups_size":0,"ondisk_log_size":9390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38437830,"num_objects":9386,"num_object_clones":0,"num_object_copies":18772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5701,"num_write_kb":22804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5225},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9444","reported_seq":9572,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:55.923159+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:55.923159+0000","last_peered":"2024-09-15T21:23:55.923159+0000","last_clean":"2024-09-15T21:23:55.923159+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:55.923159+0000","last_undegraded":"2024-09-15T21:23:55.923159+0000","last_fullsized":"2024-09-15T21:23:55.923159+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38674432,"num_objects":9442,"num_object_clones":0,"num_object_copies":18884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6216,"num_write_kb":24864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9256","reported_seq":9381,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.517813+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:53.517813+0000","last_peered":"2024-09-15T21:23:53.517813+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:53.517813+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9256,"log_dups_size":0,"ondisk_log_size":9256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37884387,"num_objects":9250,"num_object_clones":0,"num_object_copies":18500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7188,"num_write_kb":28752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5163}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9239","reported_seq":9369,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:56.042095+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:56.042095+0000","last_peered":"2024-09-15T21:23:56.042095+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:56.042095+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9239,"log_dups_size":0,"ondisk_log_size":9239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37810176,"num_objects":9231,"num_object_clones":0,"num_object_copies":18462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5912,"num_write_kb":23648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5211},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9298","reported_seq":9416,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:53.397883+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:53.397883+0000","last_peered":"2024-09-15T21:23:53.397883+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:53.397883+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9298,"log_dups_size":0,"ondisk_log_size":9298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38064611,"num_objects":9294,"num_object_clones":0,"num_object_copies":18588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9294,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7157,"num_write_kb":28628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5204}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9273","reported_seq":17731,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:53.447728+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:53.447728+0000","last_peered":"2024-09-15T21:23:53.447728+0000","last_clean":"2024-09-15T21:23:53.447728+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:53.447728+0000","last_undegraded":"2024-09-15T21:23:53.447728+0000","last_fullsized":"2024-09-15T21:23:53.447728+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9273,"log_dups_size":0,"ondisk_log_size":9273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37957632,"num_objects":9267,"num_object_clones":0,"num_object_copies":18534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6966,"num_write_kb":27864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9341","reported_seq":9474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.981001+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:55.981001+0000","last_peered":"2024-09-15T21:23:55.981001+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:55.981001+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9341,"log_dups_size":0,"ondisk_log_size":9341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38252544,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6285,"num_write_kb":25140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5245},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9304","reported_seq":9436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.947146+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:55.947146+0000","last_peered":"2024-09-15T21:23:55.947146+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:55.947146+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9304,"log_dups_size":0,"ondisk_log_size":9304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38076416,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6213,"num_write_kb":24852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5157},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9222","reported_seq":10069,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:56.037755+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:56.037755+0000","last_peered":"2024-09-15T21:23:56.037755+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:56.037755+0000","last_undegraded":"2024-09-15T21:23:56.037755+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9222,"log_dups_size":0,"ondisk_log_size":9222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37756928,"num_objects":9218,"num_object_clones":0,"num_object_copies":18436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":9218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5706,"num_write_kb":22824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":1458176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5492},{"shards":"0,2","objects":3726}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9431","reported_seq":9567,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:56.028520+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:56.028520+0000","last_peered":"2024-09-15T21:23:56.028520+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:56.028520+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9431,"log_dups_size":0,"ondisk_log_size":9431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38581190,"num_objects":9421,"num_object_clones":0,"num_object_copies":18842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9421,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6135,"num_write_kb":24540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5274},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":611394270,"num_objects":149275,"num_object_clones":0,"num_object_copies":298550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":149275,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":109140,"num_write_kb":436560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4521,"num_bytes_recovered":18509824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1236705280,"data_stored":1236557147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149347,"ondisk_log_size":149347,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":691196,"kb_used_data":429036,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93680644,"statfs":{"total":96636764160,"available":95928979456,"internally_reserved":0,"allocated":439332864,"data_stored":439061289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820588,"kb_used_data":514076,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93551252,"statfs":{"total":96636764160,"available":95796482048,"internally_reserved":0,"allocated":526413824,"data_stored":526138172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480248,"kb_used_data":263912,"kb_used_omap":4,"kb_used_meta":216315,"kb_avail":93891592,"statfs":{"total":96636764160,"available":96144990208,"internally_reserved":0,"allocated":270245888,"data_stored":269930493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221507247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527228928,"data_stored":527192798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":270352384,"data_stored":270276511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":439123968,"data_stored":439087838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:23:58.762 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 207 20 150877 150857 2.84633 1.75391 0.0394599 0.0274391 2024-09-15T21:23:59.763 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 208 20 151339 151319 2.84133 1.80469 0.0998892 0.0274889 2024-09-15T21:24:00.763 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 209 20 151807 151787 2.83648 1.82812 0.0502488 0.0275357 2024-09-15T21:24:01.131 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:01.697 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:01.725 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-15T21:24:00.750224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614871793,"num_objects":150126,"num_object_clones":0,"num_object_copies":300252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":150126,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":109991,"num_write_kb":439958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4522,"num_bytes_recovered":18509843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150199,"ondisk_log_size":150199,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2030032,"kb_used_data":1218656,"kb_used_omap":12,"kb_used_meta":811315,"kb_avail":281085488,"statfs":{"total":289910292480,"available":287831539712,"internally_reserved":0,"allocated":1247903744,"data_stored":1247041122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":830786563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,4,5,14,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18620416,"num_objects":4546,"num_object_clones":0,"num_object_copies":9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-150,"num_objects_unfound":0,"num_objects_dirty":4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4546,"num_write_kb":18184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":614400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002872"},"pg_stats":[{"pgid":"2.f","version":"45'9599","reported_seq":9674,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.971843+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:55.971843+0000","last_peered":"2024-09-15T21:23:55.971843+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:55.971843+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9599,"log_dups_size":0,"ondisk_log_size":9599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39317504,"num_objects":9599,"num_object_clones":0,"num_object_copies":19198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":38396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5467},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9442","reported_seq":9547,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.437372+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:58.437372+0000","last_peered":"2024-09-15T21:23:58.437372+0000","last_clean":"2024-09-15T21:23:58.437372+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:58.437372+0000","last_undegraded":"2024-09-15T21:23:58.437372+0000","last_fullsized":"2024-09-15T21:23:58.437372+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9442,"log_dups_size":0,"ondisk_log_size":9442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38650822,"num_objects":9438,"num_object_clones":0,"num_object_copies":18876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9438,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7723,"num_write_kb":30892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9654","reported_seq":9779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.467912+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:58.467912+0000","last_peered":"2024-09-15T21:23:58.467912+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:58.467912+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9654,"log_dups_size":0,"ondisk_log_size":9654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39526400,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7754,"num_write_kb":31016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5383}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9280","reported_seq":9350,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.507662+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:58.507662+0000","last_peered":"2024-09-15T21:23:58.507662+0000","last_clean":"2024-09-15T21:23:58.507662+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:58.507662+0000","last_undegraded":"2024-09-15T21:23:58.507662+0000","last_fullsized":"2024-09-15T21:23:58.507662+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9280,"log_dups_size":0,"ondisk_log_size":9280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38010880,"num_objects":9280,"num_object_clones":0,"num_object_copies":18560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9280,"num_write_kb":37120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9302","reported_seq":9434,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:56.036822+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:56.036822+0000","last_peered":"2024-09-15T21:23:56.036822+0000","last_clean":"2024-09-15T21:23:56.036822+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:56.036822+0000","last_undegraded":"2024-09-15T21:23:56.036822+0000","last_fullsized":"2024-09-15T21:23:56.036822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9302,"log_dups_size":0,"ondisk_log_size":9302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072803,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5757,"num_write_kb":23028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9312","reported_seq":9448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.997027+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:55.997027+0000","last_peered":"2024-09-15T21:23:55.997027+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:55.997027+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9312,"log_dups_size":0,"ondisk_log_size":9312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38121955,"num_objects":9308,"num_object_clones":0,"num_object_copies":18616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5988,"num_write_kb":23952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5232},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9390","reported_seq":9523,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:56.003121+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:56.003121+0000","last_peered":"2024-09-15T21:23:56.003121+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:56.003121+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9390,"log_dups_size":0,"ondisk_log_size":9390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38437830,"num_objects":9386,"num_object_clones":0,"num_object_copies":18772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5701,"num_write_kb":22804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5225},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9444","reported_seq":9572,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:55.923159+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:55.923159+0000","last_peered":"2024-09-15T21:23:55.923159+0000","last_clean":"2024-09-15T21:23:55.923159+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:55.923159+0000","last_undegraded":"2024-09-15T21:23:55.923159+0000","last_fullsized":"2024-09-15T21:23:55.923159+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38674432,"num_objects":9442,"num_object_clones":0,"num_object_copies":18884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6216,"num_write_kb":24864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9398","reported_seq":9524,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.520338+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:58.520338+0000","last_peered":"2024-09-15T21:23:58.520338+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:58.520338+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9398,"log_dups_size":0,"ondisk_log_size":9398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38466019,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7330,"num_write_kb":29320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5305}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9239","reported_seq":9369,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:56.042095+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:56.042095+0000","last_peered":"2024-09-15T21:23:56.042095+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:56.042095+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9239,"log_dups_size":0,"ondisk_log_size":9239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37810176,"num_objects":9231,"num_object_clones":0,"num_object_copies":18462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5912,"num_write_kb":23648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5211},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9428","reported_seq":9545,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.499229+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:58.499229+0000","last_peered":"2024-09-15T21:23:58.499229+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:58.499229+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9428,"log_dups_size":0,"ondisk_log_size":9428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38597091,"num_objects":9424,"num_object_clones":0,"num_object_copies":18848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7287,"num_write_kb":29148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5334}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9410","reported_seq":17868,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.423398+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:58.423398+0000","last_peered":"2024-09-15T21:23:58.423398+0000","last_clean":"2024-09-15T21:23:58.423398+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:58.423398+0000","last_undegraded":"2024-09-15T21:23:58.423398+0000","last_fullsized":"2024-09-15T21:23:58.423398+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9410,"log_dups_size":0,"ondisk_log_size":9410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38518784,"num_objects":9404,"num_object_clones":0,"num_object_copies":18808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7103,"num_write_kb":28412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9341","reported_seq":9474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.981001+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:55.981001+0000","last_peered":"2024-09-15T21:23:55.981001+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:55.981001+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9341,"log_dups_size":0,"ondisk_log_size":9341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38252544,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6285,"num_write_kb":25140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5245},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9304","reported_seq":9436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.947146+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:55.947146+0000","last_peered":"2024-09-15T21:23:55.947146+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:55.947146+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9304,"log_dups_size":0,"ondisk_log_size":9304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38076416,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6213,"num_write_kb":24852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5157},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9222","reported_seq":10069,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:56.037755+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:56.037755+0000","last_peered":"2024-09-15T21:23:56.037755+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:56.037755+0000","last_undegraded":"2024-09-15T21:23:56.037755+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9222,"log_dups_size":0,"ondisk_log_size":9222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37756928,"num_objects":9218,"num_object_clones":0,"num_object_copies":18436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":9218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5706,"num_write_kb":22824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":1458176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5492},{"shards":"0,2","objects":3726}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9431","reported_seq":9567,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:56.028520+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:56.028520+0000","last_peered":"2024-09-15T21:23:56.028520+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:56.028520+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9431,"log_dups_size":0,"ondisk_log_size":9431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38581190,"num_objects":9421,"num_object_clones":0,"num_object_copies":18842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9421,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6135,"num_write_kb":24540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5274},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":614871774,"num_objects":150124,"num_object_clones":0,"num_object_copies":300248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":150124,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":109989,"num_write_kb":439956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4521,"num_bytes_recovered":18509824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1248718848,"data_stored":1248570715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150196,"ondisk_log_size":150196,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699896,"kb_used_data":433576,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93671944,"statfs":{"total":96636764160,"available":95920070656,"internally_reserved":0,"allocated":443981824,"data_stored":443710249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":272690857},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820588,"kb_used_data":514076,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93551252,"statfs":{"total":96636764160,"available":95796482048,"internally_reserved":0,"allocated":526413824,"data_stored":526138172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509548,"kb_used_data":271004,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93862292,"statfs":{"total":96636764160,"available":96114987008,"internally_reserved":0,"allocated":277508096,"data_stored":277192701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527228928,"data_stored":527192798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":277815296,"data_stored":277739423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":443674624,"data_stored":443638494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:01.727 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:01.763 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 210 20 152242 152222 2.83106 1.69922 0.0381037 0.0275878 2024-09-15T21:24:02.290 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:02.291 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:02.312 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-15T21:24:00.750224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":614871793,"num_objects":150126,"num_object_clones":0,"num_object_copies":300252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":150126,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":109991,"num_write_kb":439958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4522,"num_bytes_recovered":18509843,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150199,"ondisk_log_size":150199,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2030032,"kb_used_data":1218656,"kb_used_omap":12,"kb_used_meta":811315,"kb_avail":281085488,"statfs":{"total":289910292480,"available":287831539712,"internally_reserved":0,"allocated":1247903744,"data_stored":1247041122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":830786563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,4,5,14,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18620416,"num_objects":4546,"num_object_clones":0,"num_object_copies":9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-150,"num_objects_unfound":0,"num_objects_dirty":4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4546,"num_write_kb":18184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":150,"num_bytes_recovered":614400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002872"},"pg_stats":[{"pgid":"2.f","version":"45'9599","reported_seq":9674,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.971843+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:23:55.971843+0000","last_peered":"2024-09-15T21:23:55.971843+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:23:55.971843+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9599,"log_dups_size":0,"ondisk_log_size":9599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39317504,"num_objects":9599,"num_object_clones":0,"num_object_copies":19198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9599,"num_write_kb":38396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5467},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9442","reported_seq":9547,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.437372+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:58.437372+0000","last_peered":"2024-09-15T21:23:58.437372+0000","last_clean":"2024-09-15T21:23:58.437372+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:58.437372+0000","last_undegraded":"2024-09-15T21:23:58.437372+0000","last_fullsized":"2024-09-15T21:23:58.437372+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9442,"log_dups_size":0,"ondisk_log_size":9442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38650822,"num_objects":9438,"num_object_clones":0,"num_object_copies":18876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9438,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7723,"num_write_kb":30892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9654","reported_seq":9779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.467912+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:58.467912+0000","last_peered":"2024-09-15T21:23:58.467912+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:58.467912+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9654,"log_dups_size":0,"ondisk_log_size":9654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39526400,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7754,"num_write_kb":31016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5383}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9280","reported_seq":9350,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.507662+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:58.507662+0000","last_peered":"2024-09-15T21:23:58.507662+0000","last_clean":"2024-09-15T21:23:58.507662+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:58.507662+0000","last_undegraded":"2024-09-15T21:23:58.507662+0000","last_fullsized":"2024-09-15T21:23:58.507662+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9280,"log_dups_size":0,"ondisk_log_size":9280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38010880,"num_objects":9280,"num_object_clones":0,"num_object_copies":18560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9280,"num_write_kb":37120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9302","reported_seq":9434,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:56.036822+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:23:56.036822+0000","last_peered":"2024-09-15T21:23:56.036822+0000","last_clean":"2024-09-15T21:23:56.036822+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:23:56.036822+0000","last_undegraded":"2024-09-15T21:23:56.036822+0000","last_fullsized":"2024-09-15T21:23:56.036822+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9302,"log_dups_size":0,"ondisk_log_size":9302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072803,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5757,"num_write_kb":23028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9312","reported_seq":9448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.997027+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:23:55.997027+0000","last_peered":"2024-09-15T21:23:55.997027+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:23:55.997027+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9312,"log_dups_size":0,"ondisk_log_size":9312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38121955,"num_objects":9308,"num_object_clones":0,"num_object_copies":18616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5988,"num_write_kb":23952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5232},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9390","reported_seq":9523,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:56.003121+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:23:56.003121+0000","last_peered":"2024-09-15T21:23:56.003121+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:23:56.003121+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9390,"log_dups_size":0,"ondisk_log_size":9390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38437830,"num_objects":9386,"num_object_clones":0,"num_object_copies":18772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5701,"num_write_kb":22804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5225},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9444","reported_seq":9572,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:55.923159+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:23:55.923159+0000","last_peered":"2024-09-15T21:23:55.923159+0000","last_clean":"2024-09-15T21:23:55.923159+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:23:55.923159+0000","last_undegraded":"2024-09-15T21:23:55.923159+0000","last_fullsized":"2024-09-15T21:23:55.923159+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9444,"log_dups_size":0,"ondisk_log_size":9444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38674432,"num_objects":9442,"num_object_clones":0,"num_object_copies":18884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6216,"num_write_kb":24864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9398","reported_seq":9524,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.520338+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:58.520338+0000","last_peered":"2024-09-15T21:23:58.520338+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:58.520338+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9398,"log_dups_size":0,"ondisk_log_size":9398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38466019,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7330,"num_write_kb":29320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5305}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9239","reported_seq":9369,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:56.042095+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:23:56.042095+0000","last_peered":"2024-09-15T21:23:56.042095+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:23:56.042095+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9239,"log_dups_size":0,"ondisk_log_size":9239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37810176,"num_objects":9231,"num_object_clones":0,"num_object_copies":18462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5912,"num_write_kb":23648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5211},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9428","reported_seq":9545,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.499229+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:58.499229+0000","last_peered":"2024-09-15T21:23:58.499229+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:58.499229+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9428,"log_dups_size":0,"ondisk_log_size":9428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38597091,"num_objects":9424,"num_object_clones":0,"num_object_copies":18848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7287,"num_write_kb":29148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5334}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9410","reported_seq":17868,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.423398+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:58.423398+0000","last_peered":"2024-09-15T21:23:58.423398+0000","last_clean":"2024-09-15T21:23:58.423398+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:58.423398+0000","last_undegraded":"2024-09-15T21:23:58.423398+0000","last_fullsized":"2024-09-15T21:23:58.423398+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9410,"log_dups_size":0,"ondisk_log_size":9410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38518784,"num_objects":9404,"num_object_clones":0,"num_object_copies":18808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7103,"num_write_kb":28412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9341","reported_seq":9474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.981001+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:23:55.981001+0000","last_peered":"2024-09-15T21:23:55.981001+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:23:55.981001+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9341,"log_dups_size":0,"ondisk_log_size":9341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38252544,"num_objects":9339,"num_object_clones":0,"num_object_copies":18678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6285,"num_write_kb":25140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5245},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9304","reported_seq":9436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:55.947146+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:23:55.947146+0000","last_peered":"2024-09-15T21:23:55.947146+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:23:55.947146+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9304,"log_dups_size":0,"ondisk_log_size":9304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38076416,"num_objects":9296,"num_object_clones":0,"num_object_copies":18592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6213,"num_write_kb":24852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5157},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9222","reported_seq":10069,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:23:56.037755+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:23:56.037755+0000","last_peered":"2024-09-15T21:23:56.037755+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:23:56.037755+0000","last_undegraded":"2024-09-15T21:23:56.037755+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9222,"log_dups_size":0,"ondisk_log_size":9222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37756928,"num_objects":9218,"num_object_clones":0,"num_object_copies":18436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":9218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5706,"num_write_kb":22824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":1458176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5492},{"shards":"0,2","objects":3726}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9431","reported_seq":9567,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:56.028520+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:23:56.028520+0000","last_peered":"2024-09-15T21:23:56.028520+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:23:56.028520+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9431,"log_dups_size":0,"ondisk_log_size":9431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38581190,"num_objects":9421,"num_object_clones":0,"num_object_copies":18842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9421,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6135,"num_write_kb":24540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5274},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":614871774,"num_objects":150124,"num_object_clones":0,"num_object_copies":300248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3726,"num_objects_unfound":0,"num_objects_dirty":150124,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":109989,"num_write_kb":439956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4521,"num_bytes_recovered":18509824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1248718848,"data_stored":1248570715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150196,"ondisk_log_size":150196,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699896,"kb_used_data":433576,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93671944,"statfs":{"total":96636764160,"available":95920070656,"internally_reserved":0,"allocated":443981824,"data_stored":443710249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":272690857},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820588,"kb_used_data":514076,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93551252,"statfs":{"total":96636764160,"available":95796482048,"internally_reserved":0,"allocated":526413824,"data_stored":526138172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509548,"kb_used_data":271004,"kb_used_omap":4,"kb_used_meta":238523,"kb_avail":93862292,"statfs":{"total":96636764160,"available":96114987008,"internally_reserved":0,"allocated":277508096,"data_stored":277192701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244248239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527228928,"data_stored":527192798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":277815296,"data_stored":277739423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":443674624,"data_stored":443638494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:02.314 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:02.314 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'9599", 'reported_seq': 9674, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:55.971843+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:23:55.971843+0000', 'last_peered': '2024-09-15T21:23:55.971843+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:23:55.971843+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 9599, 'log_dups_size': 0, 'ondisk_log_size': 9599, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39317504, 'num_objects': 9599, 'num_object_clones': 0, 'num_object_copies': 19198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9599, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9599, 'num_write_kb': 38396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5467}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:02.314 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:24:02.314 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'9654", 'reported_seq': 9779, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:58.467912+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:23:58.467912+0000', 'last_peered': '2024-09-15T21:23:58.467912+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:23:58.467912+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 9654, 'log_dups_size': 0, 'ondisk_log_size': 9654, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39526400, 'num_objects': 9650, 'num_object_clones': 0, 'num_object_copies': 19300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7754, 'num_write_kb': 31016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 5383}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:02.314 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:02.314 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'9312", 'reported_seq': 9448, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:55.997027+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:23:55.997027+0000', 'last_peered': '2024-09-15T21:23:55.997027+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:23:55.997027+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 9312, 'log_dups_size': 0, 'ondisk_log_size': 9312, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38121955, 'num_objects': 9308, 'num_object_clones': 0, 'num_object_copies': 18616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9308, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 5988, 'num_write_kb': 23952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5232}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:02.314 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:02.314 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'9390", 'reported_seq': 9523, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:56.003121+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:23:56.003121+0000', 'last_peered': '2024-09-15T21:23:56.003121+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:23:56.003121+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 9390, 'log_dups_size': 0, 'ondisk_log_size': 9390, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38437830, 'num_objects': 9386, 'num_object_clones': 0, 'num_object_copies': 18772, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9386, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5701, 'num_write_kb': 22804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5225}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:02.314 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:24:02.315 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'9398", 'reported_seq': 9524, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:58.520338+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:23:58.520338+0000', 'last_peered': '2024-09-15T21:23:58.520338+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:23:58.520338+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 9398, 'log_dups_size': 0, 'ondisk_log_size': 9398, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38466019, 'num_objects': 9392, 'num_object_clones': 0, 'num_object_copies': 18784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9392, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7330, 'num_write_kb': 29320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 5305}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:02.315 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:02.315 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'9239", 'reported_seq': 9369, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:56.042095+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:23:56.042095+0000', 'last_peered': '2024-09-15T21:23:56.042095+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:23:56.042095+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 9239, 'log_dups_size': 0, 'ondisk_log_size': 9239, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37810176, 'num_objects': 9231, 'num_object_clones': 0, 'num_object_copies': 18462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9231, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5912, 'num_write_kb': 23648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5211}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:02.315 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:02.315 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'9428", 'reported_seq': 9545, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:58.499229+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:23:58.499229+0000', 'last_peered': '2024-09-15T21:23:58.499229+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:23:58.499229+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 9428, 'log_dups_size': 0, 'ondisk_log_size': 9428, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38597091, 'num_objects': 9424, 'num_object_clones': 0, 'num_object_copies': 18848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9424, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7287, 'num_write_kb': 29148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 5334}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:02.315 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:02.315 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'9341", 'reported_seq': 9474, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:55.981001+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:23:55.981001+0000', 'last_peered': '2024-09-15T21:23:55.981001+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:23:55.981001+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 9341, 'log_dups_size': 0, 'ondisk_log_size': 9341, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38252544, 'num_objects': 9339, 'num_object_clones': 0, 'num_object_copies': 18678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9339, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6285, 'num_write_kb': 25140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5245}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:02.315 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:02.315 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'9304", 'reported_seq': 9436, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:55.947146+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:23:55.947146+0000', 'last_peered': '2024-09-15T21:23:55.947146+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:23:55.947146+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 9304, 'log_dups_size': 0, 'ondisk_log_size': 9304, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38076416, 'num_objects': 9296, 'num_object_clones': 0, 'num_object_copies': 18592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6213, 'num_write_kb': 24852, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5157}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:02.315 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:02.315 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'9222", 'reported_seq': 10069, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:23:56.037755+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:23:56.037755+0000', 'last_peered': '2024-09-15T21:23:56.037755+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:23:56.037755+0000', 'last_undegraded': '2024-09-15T21:23:56.037755+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 9222, 'log_dups_size': 0, 'ondisk_log_size': 9222, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37756928, 'num_objects': 9218, 'num_object_clones': 0, 'num_object_copies': 18436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3726, 'num_objects_unfound': 0, 'num_objects_dirty': 9218, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5706, 'num_write_kb': 22824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 356, 'num_bytes_recovered': 1458176, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5492}, {'shards': '0,2', 'objects': 3726}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:02.316 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:24:02.316 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'9431", 'reported_seq': 9567, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:23:56.028520+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:23:56.028520+0000', 'last_peered': '2024-09-15T21:23:56.028520+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:23:56.028520+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 9431, 'log_dups_size': 0, 'ondisk_log_size': 9431, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38581190, 'num_objects': 9421, 'num_object_clones': 0, 'num_object_copies': 18842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9421, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6135, 'num_write_kb': 24540, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5274}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:02.316 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:02.763 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 211 20 152663 152643 2.82544 1.64453 0.0905658 0.0276429 2024-09-15T21:24:03.064 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:03.064 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/300252 objects degraded (13.730%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":294,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":150126,"data_bytes":614871793,"bytes_used":2078752768,"bytes_avail":287831539712,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":300252,"degraded_ratio":0.13730133354648763,"misplaced_objects":3726,"misplaced_total":300252,"misplaced_ratio":0.012409575956196794,"recovering_objects_per_sec":12,"recovering_bytes_per_sec":51187,"recovering_keys_per_sec":0,"num_objects_recovered":150,"num_bytes_recovered":614400,"num_keys_recovered":0,"write_bytes_sec":1551329,"read_op_per_sec":0,"write_op_per_sec":378},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:03.092 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:03.092 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:03.660 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:03.662 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:03.689 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-15T21:24:02.750647+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620671729,"num_objects":151542,"num_object_clones":0,"num_object_copies":303084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3642,"num_objects_unfound":0,"num_objects_dirty":151542,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":111407,"num_write_kb":445622,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4606,"num_bytes_recovered":18853907,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151615,"ondisk_log_size":151615,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2039648,"kb_used_data":1232432,"kb_used_omap":12,"kb_used_meta":807155,"kb_avail":281075872,"statfs":{"total":289910292480,"available":287821692928,"internally_reserved":0,"allocated":1262010368,"data_stored":1261147746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":826526723},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,6,10,13,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24420352,"num_objects":5962,"num_object_clones":0,"num_object_copies":11924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-234,"num_objects_unfound":0,"num_objects_dirty":5962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5962,"num_write_kb":23848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":958464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002789"},"pg_stats":[{"pgid":"2.f","version":"45'9739","reported_seq":9813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:01.075278+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:01.075278+0000","last_peered":"2024-09-15T21:24:01.075278+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:01.075278+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9739,"log_dups_size":0,"ondisk_log_size":9739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39890944,"num_objects":9739,"num_object_clones":0,"num_object_copies":19478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9739,"num_write_kb":38956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5607},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9442","reported_seq":9547,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.437372+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:23:58.437372+0000","last_peered":"2024-09-15T21:23:58.437372+0000","last_clean":"2024-09-15T21:23:58.437372+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:23:58.437372+0000","last_undegraded":"2024-09-15T21:23:58.437372+0000","last_fullsized":"2024-09-15T21:23:58.437372+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9442,"log_dups_size":0,"ondisk_log_size":9442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38650822,"num_objects":9438,"num_object_clones":0,"num_object_copies":18876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9438,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7723,"num_write_kb":30892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9654","reported_seq":9779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.467912+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:23:58.467912+0000","last_peered":"2024-09-15T21:23:58.467912+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:23:58.467912+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9654,"log_dups_size":0,"ondisk_log_size":9654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39526400,"num_objects":9650,"num_object_clones":0,"num_object_copies":19300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7754,"num_write_kb":31016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5383}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9280","reported_seq":9350,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.507662+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:23:58.507662+0000","last_peered":"2024-09-15T21:23:58.507662+0000","last_clean":"2024-09-15T21:23:58.507662+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:23:58.507662+0000","last_undegraded":"2024-09-15T21:23:58.507662+0000","last_fullsized":"2024-09-15T21:23:58.507662+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9280,"log_dups_size":0,"ondisk_log_size":9280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38010880,"num_objects":9280,"num_object_clones":0,"num_object_copies":18560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9280,"num_write_kb":37120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9433","reported_seq":9565,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:01.043497+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:01.043497+0000","last_peered":"2024-09-15T21:24:01.043497+0000","last_clean":"2024-09-15T21:24:01.043497+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:01.043497+0000","last_undegraded":"2024-09-15T21:24:01.043497+0000","last_fullsized":"2024-09-15T21:24:01.043497+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9433,"log_dups_size":0,"ondisk_log_size":9433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38609379,"num_objects":9427,"num_object_clones":0,"num_object_copies":18854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5888,"num_write_kb":23552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9468","reported_seq":9602,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:01.051333+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:01.051333+0000","last_peered":"2024-09-15T21:24:01.051333+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:01.051333+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9468,"log_dups_size":0,"ondisk_log_size":9468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38760931,"num_objects":9464,"num_object_clones":0,"num_object_copies":18928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6144,"num_write_kb":24576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5388},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9535","reported_seq":9671,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:00.966935+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:00.966935+0000","last_peered":"2024-09-15T21:24:00.966935+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:00.966935+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9535,"log_dups_size":0,"ondisk_log_size":9535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39031750,"num_objects":9531,"num_object_clones":0,"num_object_copies":19062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5846,"num_write_kb":23384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5370},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9577","reported_seq":9705,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:00.989483+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:00.989483+0000","last_peered":"2024-09-15T21:24:00.989483+0000","last_clean":"2024-09-15T21:24:00.989483+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:00.989483+0000","last_undegraded":"2024-09-15T21:24:00.989483+0000","last_fullsized":"2024-09-15T21:24:00.989483+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9577,"log_dups_size":0,"ondisk_log_size":9577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39219200,"num_objects":9575,"num_object_clones":0,"num_object_copies":19150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6349,"num_write_kb":25396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9398","reported_seq":9524,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.520338+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:23:58.520338+0000","last_peered":"2024-09-15T21:23:58.520338+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:23:58.520338+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9398,"log_dups_size":0,"ondisk_log_size":9398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38466019,"num_objects":9392,"num_object_clones":0,"num_object_copies":18784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7330,"num_write_kb":29320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5305}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9360","reported_seq":9490,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:01.012213+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:01.012213+0000","last_peered":"2024-09-15T21:24:01.012213+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:01.012213+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9360,"log_dups_size":0,"ondisk_log_size":9360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38305792,"num_objects":9352,"num_object_clones":0,"num_object_copies":18704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6033,"num_write_kb":24132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5332},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9428","reported_seq":9545,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:23:58.499229+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:23:58.499229+0000","last_peered":"2024-09-15T21:23:58.499229+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:23:58.499229+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9428,"log_dups_size":0,"ondisk_log_size":9428,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38597091,"num_objects":9424,"num_object_clones":0,"num_object_copies":18848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9424,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7287,"num_write_kb":29148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5334}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9410","reported_seq":17868,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:58.423398+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:23:58.423398+0000","last_peered":"2024-09-15T21:23:58.423398+0000","last_clean":"2024-09-15T21:23:58.423398+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:23:58.423398+0000","last_undegraded":"2024-09-15T21:23:58.423398+0000","last_fullsized":"2024-09-15T21:23:58.423398+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9410,"log_dups_size":0,"ondisk_log_size":9410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38518784,"num_objects":9404,"num_object_clones":0,"num_object_copies":18808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7103,"num_write_kb":28412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9482","reported_seq":9615,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:01.065978+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:01.065978+0000","last_peered":"2024-09-15T21:24:01.065978+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:01.065978+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9482,"log_dups_size":0,"ondisk_log_size":9482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38830080,"num_objects":9480,"num_object_clones":0,"num_object_copies":18960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6426,"num_write_kb":25704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5386},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9476","reported_seq":9608,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:00.974571+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:00.974571+0000","last_peered":"2024-09-15T21:24:00.974571+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:00.974571+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9476,"log_dups_size":0,"ondisk_log_size":9476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38780928,"num_objects":9468,"num_object_clones":0,"num_object_copies":18936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6385,"num_write_kb":25540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5329},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9357","reported_seq":10373,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:01.095316+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:01.095316+0000","last_peered":"2024-09-15T21:24:01.095316+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:01.095316+0000","last_undegraded":"2024-09-15T21:24:01.095316+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9357,"log_dups_size":0,"ondisk_log_size":9357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38309888,"num_objects":9353,"num_object_clones":0,"num_object_copies":18706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3642,"num_objects_unfound":0,"num_objects_dirty":9353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5841,"num_write_kb":23364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5711},{"shards":"0,2","objects":3642}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9573","reported_seq":9711,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:00.955419+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:00.955419+0000","last_peered":"2024-09-15T21:24:00.955419+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:00.955419+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9573,"log_dups_size":0,"ondisk_log_size":9573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39162822,"num_objects":9563,"num_object_clones":0,"num_object_copies":19126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9563,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6277,"num_write_kb":25108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5416},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":620671710,"num_objects":151540,"num_object_clones":0,"num_object_copies":303080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3642,"num_objects_unfound":0,"num_objects_dirty":151540,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":111405,"num_write_kb":445620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4605,"num_bytes_recovered":18853888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1263153152,"data_stored":1263005019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":151612,"ondisk_log_size":151612,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699896,"kb_used_data":433576,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93671944,"statfs":{"total":96636764160,"available":95920070656,"internally_reserved":0,"allocated":443981824,"data_stored":443710249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":272690857},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":831328,"kb_used_data":520656,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93540512,"statfs":{"total":96636764160,"available":95785484288,"internally_reserved":0,"allocated":533151744,"data_stored":532876092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,6,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":278200,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":284876800,"data_stored":284561405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":235728559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":534179840,"data_stored":534143710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":285298688,"data_stored":285222815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":443674624,"data_stored":443638494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:03.763 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 212 20 153091 153071 2.82 1.67188 0.025725 0.0276978 2024-09-15T21:24:04.763 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 213 20 153553 153533 2.81523 1.80469 0.0455548 0.0277439 2024-09-15T21:24:05.763 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 214 20 154015 153995 2.81051 1.80469 0.0247637 0.02779 2024-09-15T21:24:06.692 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:06.763 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 215 20 154490 154470 2.80607 1.85547 0.0584514 0.027835 2024-09-15T21:24:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:07.257 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:07.278 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-15T21:24:06.751586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":629834481,"num_objects":153779,"num_object_clones":0,"num_object_copies":307558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":153779,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":113644,"num_write_kb":454570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4685,"num_bytes_recovered":19177491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153852,"ondisk_log_size":153852,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2065916,"kb_used_data":1244812,"kb_used_omap":12,"kb_used_meta":821043,"kb_avail":281049604,"statfs":{"total":289910292480,"available":287794794496,"internally_reserved":0,"allocated":1274687488,"data_stored":1273824866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":840748035},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,12,3,10,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24379392,"num_objects":5952,"num_object_clones":0,"num_object_copies":11904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-235,"num_objects_unfound":0,"num_objects_dirty":5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5952,"num_write_kb":23808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":962560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002787"},"pg_stats":[{"pgid":"2.f","version":"45'9872","reported_seq":9946,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.099863+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:06.099863+0000","last_peered":"2024-09-15T21:24:06.099863+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:06.099863+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9872,"log_dups_size":0,"ondisk_log_size":9872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40435712,"num_objects":9872,"num_object_clones":0,"num_object_copies":19744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9872,"num_write_kb":39488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5740},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9583","reported_seq":9688,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.355946+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:03.355946+0000","last_peered":"2024-09-15T21:24:03.355946+0000","last_clean":"2024-09-15T21:24:03.355946+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:03.355946+0000","last_undegraded":"2024-09-15T21:24:03.355946+0000","last_fullsized":"2024-09-15T21:24:03.355946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9583,"log_dups_size":0,"ondisk_log_size":9583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39228358,"num_objects":9579,"num_object_clones":0,"num_object_copies":19158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9579,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7864,"num_write_kb":31456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9801","reported_seq":9926,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.395482+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:03.395482+0000","last_peered":"2024-09-15T21:24:03.395482+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:03.395482+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9801,"log_dups_size":0,"ondisk_log_size":9801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40128512,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7901,"num_write_kb":31604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5530}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9401","reported_seq":9471,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.460901+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:03.460901+0000","last_peered":"2024-09-15T21:24:03.460901+0000","last_clean":"2024-09-15T21:24:03.460901+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:03.460901+0000","last_undegraded":"2024-09-15T21:24:03.460901+0000","last_fullsized":"2024-09-15T21:24:03.460901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9401,"log_dups_size":0,"ondisk_log_size":9401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38506496,"num_objects":9401,"num_object_clones":0,"num_object_copies":18802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9401,"num_write_kb":37604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9587","reported_seq":9719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:06.124620+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:06.124620+0000","last_peered":"2024-09-15T21:24:06.124620+0000","last_clean":"2024-09-15T21:24:06.124620+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:06.124620+0000","last_undegraded":"2024-09-15T21:24:06.124620+0000","last_fullsized":"2024-09-15T21:24:06.124620+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9587,"log_dups_size":0,"ondisk_log_size":9587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39240163,"num_objects":9581,"num_object_clones":0,"num_object_copies":19162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6042,"num_write_kb":24168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9613","reported_seq":9749,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.021232+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:06.021232+0000","last_peered":"2024-09-15T21:24:06.021232+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:06.021232+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9613,"log_dups_size":0,"ondisk_log_size":9613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39354851,"num_objects":9609,"num_object_clones":0,"num_object_copies":19218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6289,"num_write_kb":25156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5533},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9677","reported_seq":9813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:05.996357+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:05.996357+0000","last_peered":"2024-09-15T21:24:05.996357+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:05.996357+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39613382,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5988,"num_write_kb":23952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5512},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9721","reported_seq":9849,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:06.122087+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:06.122087+0000","last_peered":"2024-09-15T21:24:06.122087+0000","last_clean":"2024-09-15T21:24:06.122087+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:06.122087+0000","last_undegraded":"2024-09-15T21:24:06.122087+0000","last_fullsized":"2024-09-15T21:24:06.122087+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9721,"log_dups_size":0,"ondisk_log_size":9721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39809024,"num_objects":9719,"num_object_clones":0,"num_object_copies":19438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6493,"num_write_kb":25972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9520","reported_seq":9646,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.549456+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:03.549456+0000","last_peered":"2024-09-15T21:24:03.549456+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:03.549456+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9520,"log_dups_size":0,"ondisk_log_size":9520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38965731,"num_objects":9514,"num_object_clones":0,"num_object_copies":19028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9514,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7452,"num_write_kb":29808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5427}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9502","reported_seq":9632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.043090+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:06.043090+0000","last_peered":"2024-09-15T21:24:06.043090+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:06.043090+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9502,"log_dups_size":0,"ondisk_log_size":9502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38887424,"num_objects":9494,"num_object_clones":0,"num_object_copies":18988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6175,"num_write_kb":24700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5474},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9574","reported_seq":9692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.483305+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:03.483305+0000","last_peered":"2024-09-15T21:24:03.483305+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:03.483305+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9574,"log_dups_size":0,"ondisk_log_size":9574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39195107,"num_objects":9570,"num_object_clones":0,"num_object_copies":19140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9570,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7433,"num_write_kb":29732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5480}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9555","reported_seq":18013,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.487845+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:03.487845+0000","last_peered":"2024-09-15T21:24:03.487845+0000","last_clean":"2024-09-15T21:24:03.487845+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:03.487845+0000","last_undegraded":"2024-09-15T21:24:03.487845+0000","last_fullsized":"2024-09-15T21:24:03.487845+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9555,"log_dups_size":0,"ondisk_log_size":9555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7248,"num_write_kb":28992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9642","reported_seq":9775,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.049535+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:06.049535+0000","last_peered":"2024-09-15T21:24:06.049535+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:06.049535+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9642,"log_dups_size":0,"ondisk_log_size":9642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39485440,"num_objects":9640,"num_object_clones":0,"num_object_copies":19280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6586,"num_write_kb":26344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5546},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9599","reported_seq":9730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.104552+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:06.104552+0000","last_peered":"2024-09-15T21:24:06.104552+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:06.104552+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9599,"log_dups_size":0,"ondisk_log_size":9599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6508,"num_write_kb":26032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5452},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9489","reported_seq":10660,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:06.137215+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:06.137215+0000","last_peered":"2024-09-15T21:24:06.137215+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:06.137215+0000","last_undegraded":"2024-09-15T21:24:06.137215+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9489,"log_dups_size":0,"ondisk_log_size":9489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38850560,"num_objects":9485,"num_object_clones":0,"num_object_copies":18970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":9485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5973,"num_write_kb":23892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":2125824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5922},{"shards":"0,2","objects":3563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9713","reported_seq":9851,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.067697+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:06.067697+0000","last_peered":"2024-09-15T21:24:06.067697+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:06.067697+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9713,"log_dups_size":0,"ondisk_log_size":9713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39736262,"num_objects":9703,"num_object_clones":0,"num_object_copies":19406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9703,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6417,"num_write_kb":25668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5556},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":629834462,"num_objects":153777,"num_object_clones":0,"num_object_copies":307554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":153777,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":113642,"num_write_kb":454568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4684,"num_bytes_recovered":19177472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1274564608,"data_stored":1274416475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153849,"ondisk_log_size":153849,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708348,"kb_used_data":437868,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93663492,"statfs":{"total":96636764160,"available":95911415808,"internally_reserved":0,"allocated":448376832,"data_stored":448105257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":276950697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":849144,"kb_used_data":528744,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93522696,"statfs":{"total":96636764160,"available":95767240704,"internally_reserved":0,"allocated":541433856,"data_stored":541158204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328068779},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,6,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":278200,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":284876800,"data_stored":284561405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":235728559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541167616,"data_stored":541131486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":285298688,"data_stored":285222815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448098304,"data_stored":448062174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:07.280 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:07.763 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 216 20 154890 154870 2.80031 1.5625 0.0293472 0.0278913 2024-09-15T21:24:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:07.847 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:07.873 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-15T21:24:06.751586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":629834481,"num_objects":153779,"num_object_clones":0,"num_object_copies":307558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":153779,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":113644,"num_write_kb":454570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4685,"num_bytes_recovered":19177491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153852,"ondisk_log_size":153852,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2065916,"kb_used_data":1244812,"kb_used_omap":12,"kb_used_meta":821043,"kb_avail":281049604,"statfs":{"total":289910292480,"available":287794794496,"internally_reserved":0,"allocated":1274687488,"data_stored":1273824866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":840748035},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,12,3,10,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24379392,"num_objects":5952,"num_object_clones":0,"num_object_copies":11904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-235,"num_objects_unfound":0,"num_objects_dirty":5952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5952,"num_write_kb":23808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":235,"num_bytes_recovered":962560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002787"},"pg_stats":[{"pgid":"2.f","version":"45'9872","reported_seq":9946,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.099863+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:06.099863+0000","last_peered":"2024-09-15T21:24:06.099863+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:06.099863+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9872,"log_dups_size":0,"ondisk_log_size":9872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40435712,"num_objects":9872,"num_object_clones":0,"num_object_copies":19744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9872,"num_write_kb":39488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5740},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9583","reported_seq":9688,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.355946+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:03.355946+0000","last_peered":"2024-09-15T21:24:03.355946+0000","last_clean":"2024-09-15T21:24:03.355946+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:03.355946+0000","last_undegraded":"2024-09-15T21:24:03.355946+0000","last_fullsized":"2024-09-15T21:24:03.355946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9583,"log_dups_size":0,"ondisk_log_size":9583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39228358,"num_objects":9579,"num_object_clones":0,"num_object_copies":19158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9579,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":7864,"num_write_kb":31456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9801","reported_seq":9926,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.395482+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:03.395482+0000","last_peered":"2024-09-15T21:24:03.395482+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:03.395482+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9801,"log_dups_size":0,"ondisk_log_size":9801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40128512,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7901,"num_write_kb":31604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5530}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9401","reported_seq":9471,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.460901+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:03.460901+0000","last_peered":"2024-09-15T21:24:03.460901+0000","last_clean":"2024-09-15T21:24:03.460901+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:03.460901+0000","last_undegraded":"2024-09-15T21:24:03.460901+0000","last_fullsized":"2024-09-15T21:24:03.460901+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9401,"log_dups_size":0,"ondisk_log_size":9401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38506496,"num_objects":9401,"num_object_clones":0,"num_object_copies":18802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9401,"num_write_kb":37604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9587","reported_seq":9719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:06.124620+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:06.124620+0000","last_peered":"2024-09-15T21:24:06.124620+0000","last_clean":"2024-09-15T21:24:06.124620+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:06.124620+0000","last_undegraded":"2024-09-15T21:24:06.124620+0000","last_fullsized":"2024-09-15T21:24:06.124620+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9587,"log_dups_size":0,"ondisk_log_size":9587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39240163,"num_objects":9581,"num_object_clones":0,"num_object_copies":19162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6042,"num_write_kb":24168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9613","reported_seq":9749,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.021232+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:06.021232+0000","last_peered":"2024-09-15T21:24:06.021232+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:06.021232+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9613,"log_dups_size":0,"ondisk_log_size":9613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39354851,"num_objects":9609,"num_object_clones":0,"num_object_copies":19218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6289,"num_write_kb":25156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5533},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9677","reported_seq":9813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:05.996357+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:05.996357+0000","last_peered":"2024-09-15T21:24:05.996357+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:05.996357+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39613382,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5988,"num_write_kb":23952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5512},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9721","reported_seq":9849,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:06.122087+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:06.122087+0000","last_peered":"2024-09-15T21:24:06.122087+0000","last_clean":"2024-09-15T21:24:06.122087+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:06.122087+0000","last_undegraded":"2024-09-15T21:24:06.122087+0000","last_fullsized":"2024-09-15T21:24:06.122087+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9721,"log_dups_size":0,"ondisk_log_size":9721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39809024,"num_objects":9719,"num_object_clones":0,"num_object_copies":19438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6493,"num_write_kb":25972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9520","reported_seq":9646,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.549456+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:03.549456+0000","last_peered":"2024-09-15T21:24:03.549456+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:03.549456+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9520,"log_dups_size":0,"ondisk_log_size":9520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38965731,"num_objects":9514,"num_object_clones":0,"num_object_copies":19028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9514,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7452,"num_write_kb":29808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5427}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9502","reported_seq":9632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.043090+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:06.043090+0000","last_peered":"2024-09-15T21:24:06.043090+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:06.043090+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9502,"log_dups_size":0,"ondisk_log_size":9502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38887424,"num_objects":9494,"num_object_clones":0,"num_object_copies":18988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6175,"num_write_kb":24700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5474},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9574","reported_seq":9692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:03.483305+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:03.483305+0000","last_peered":"2024-09-15T21:24:03.483305+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:03.483305+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9574,"log_dups_size":0,"ondisk_log_size":9574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39195107,"num_objects":9570,"num_object_clones":0,"num_object_copies":19140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9570,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7433,"num_write_kb":29732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5480}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9555","reported_seq":18013,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:03.487845+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:03.487845+0000","last_peered":"2024-09-15T21:24:03.487845+0000","last_clean":"2024-09-15T21:24:03.487845+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:03.487845+0000","last_undegraded":"2024-09-15T21:24:03.487845+0000","last_fullsized":"2024-09-15T21:24:03.487845+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9555,"log_dups_size":0,"ondisk_log_size":9555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39112704,"num_objects":9549,"num_object_clones":0,"num_object_copies":19098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7248,"num_write_kb":28992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9642","reported_seq":9775,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.049535+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:06.049535+0000","last_peered":"2024-09-15T21:24:06.049535+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:06.049535+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9642,"log_dups_size":0,"ondisk_log_size":9642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39485440,"num_objects":9640,"num_object_clones":0,"num_object_copies":19280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6586,"num_write_kb":26344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5546},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9599","reported_seq":9730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.104552+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:06.104552+0000","last_peered":"2024-09-15T21:24:06.104552+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:06.104552+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9599,"log_dups_size":0,"ondisk_log_size":9599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6508,"num_write_kb":26032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5452},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9489","reported_seq":10660,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:06.137215+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:06.137215+0000","last_peered":"2024-09-15T21:24:06.137215+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:06.137215+0000","last_undegraded":"2024-09-15T21:24:06.137215+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9489,"log_dups_size":0,"ondisk_log_size":9489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38850560,"num_objects":9485,"num_object_clones":0,"num_object_copies":18970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":9485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5973,"num_write_kb":23892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":2125824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5922},{"shards":"0,2","objects":3563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9713","reported_seq":9851,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.067697+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:06.067697+0000","last_peered":"2024-09-15T21:24:06.067697+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:06.067697+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9713,"log_dups_size":0,"ondisk_log_size":9713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39736262,"num_objects":9703,"num_object_clones":0,"num_object_copies":19406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9703,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6417,"num_write_kb":25668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5556},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":629834462,"num_objects":153777,"num_object_clones":0,"num_object_copies":307554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":153777,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":113642,"num_write_kb":454568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4684,"num_bytes_recovered":19177472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1274564608,"data_stored":1274416475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":153849,"ondisk_log_size":153849,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708348,"kb_used_data":437868,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93663492,"statfs":{"total":96636764160,"available":95911415808,"internally_reserved":0,"allocated":448376832,"data_stored":448105257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":276950697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":849144,"kb_used_data":528744,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93522696,"statfs":{"total":96636764160,"available":95767240704,"internally_reserved":0,"allocated":541433856,"data_stored":541158204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328068779},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,6,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508424,"kb_used_data":278200,"kb_used_omap":4,"kb_used_meta":230203,"kb_avail":93863416,"statfs":{"total":96636764160,"available":96116137984,"internally_reserved":0,"allocated":284876800,"data_stored":284561405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":235728559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,0,3,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541167616,"data_stored":541131486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":285298688,"data_stored":285222815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448098304,"data_stored":448062174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:07.875 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:07.876 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'9872", 'reported_seq': 9946, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:06.099863+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:24:06.099863+0000', 'last_peered': '2024-09-15T21:24:06.099863+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:24:06.099863+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 9872, 'log_dups_size': 0, 'ondisk_log_size': 9872, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40435712, 'num_objects': 9872, 'num_object_clones': 0, 'num_object_copies': 19744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9872, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9872, 'num_write_kb': 39488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5740}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:07.876 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:24:07.876 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'9801", 'reported_seq': 9926, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:03.395482+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:24:03.395482+0000', 'last_peered': '2024-09-15T21:24:03.395482+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:24:03.395482+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 9801, 'log_dups_size': 0, 'ondisk_log_size': 9801, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40128512, 'num_objects': 9797, 'num_object_clones': 0, 'num_object_copies': 19594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9797, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7901, 'num_write_kb': 31604, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 5530}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:07.876 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:07.876 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'9613", 'reported_seq': 9749, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:06.021232+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:24:06.021232+0000', 'last_peered': '2024-09-15T21:24:06.021232+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:24:06.021232+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 9613, 'log_dups_size': 0, 'ondisk_log_size': 9613, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39354851, 'num_objects': 9609, 'num_object_clones': 0, 'num_object_copies': 19218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9609, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6289, 'num_write_kb': 25156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5533}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:07.876 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:07.876 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'9677", 'reported_seq': 9813, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:05.996357+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:24:05.996357+0000', 'last_peered': '2024-09-15T21:24:05.996357+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:24:05.996357+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 9677, 'log_dups_size': 0, 'ondisk_log_size': 9677, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39613382, 'num_objects': 9673, 'num_object_clones': 0, 'num_object_copies': 19346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9673, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5988, 'num_write_kb': 23952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5512}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:07.876 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:24:07.876 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'9520", 'reported_seq': 9646, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:03.549456+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:24:03.549456+0000', 'last_peered': '2024-09-15T21:24:03.549456+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:24:03.549456+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 9520, 'log_dups_size': 0, 'ondisk_log_size': 9520, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38965731, 'num_objects': 9514, 'num_object_clones': 0, 'num_object_copies': 19028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9514, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7452, 'num_write_kb': 29808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 5427}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:07.877 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:07.877 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'9502", 'reported_seq': 9632, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:06.043090+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:24:06.043090+0000', 'last_peered': '2024-09-15T21:24:06.043090+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:24:06.043090+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 9502, 'log_dups_size': 0, 'ondisk_log_size': 9502, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38887424, 'num_objects': 9494, 'num_object_clones': 0, 'num_object_copies': 18988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9494, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6175, 'num_write_kb': 24700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5474}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:07.877 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:07.877 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'9574", 'reported_seq': 9692, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:03.483305+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:24:03.483305+0000', 'last_peered': '2024-09-15T21:24:03.483305+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:24:03.483305+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 9574, 'log_dups_size': 0, 'ondisk_log_size': 9574, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39195107, 'num_objects': 9570, 'num_object_clones': 0, 'num_object_copies': 19140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9570, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7433, 'num_write_kb': 29732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 5480}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:07.877 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:07.877 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'9642", 'reported_seq': 9775, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:06.049535+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:24:06.049535+0000', 'last_peered': '2024-09-15T21:24:06.049535+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:24:06.049535+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 9642, 'log_dups_size': 0, 'ondisk_log_size': 9642, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39485440, 'num_objects': 9640, 'num_object_clones': 0, 'num_object_copies': 19280, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9640, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6586, 'num_write_kb': 26344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5546}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:07.877 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:07.877 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'9599", 'reported_seq': 9730, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:06.104552+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:24:06.104552+0000', 'last_peered': '2024-09-15T21:24:06.104552+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:24:06.104552+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 9599, 'log_dups_size': 0, 'ondisk_log_size': 9599, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39284736, 'num_objects': 9591, 'num_object_clones': 0, 'num_object_copies': 19182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6508, 'num_write_kb': 26032, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5452}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:07.877 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:07.877 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'9489", 'reported_seq': 10660, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:24:06.137215+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:24:06.137215+0000', 'last_peered': '2024-09-15T21:24:06.137215+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:24:06.137215+0000', 'last_undegraded': '2024-09-15T21:24:06.137215+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 9489, 'log_dups_size': 0, 'ondisk_log_size': 9489, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38850560, 'num_objects': 9485, 'num_object_clones': 0, 'num_object_copies': 18970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3563, 'num_objects_unfound': 0, 'num_objects_dirty': 9485, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5973, 'num_write_kb': 23892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 519, 'num_bytes_recovered': 2125824, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5922}, {'shards': '0,2', 'objects': 3563}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:07.878 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:24:07.878 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'9713", 'reported_seq': 9851, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:06.067697+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:24:06.067697+0000', 'last_peered': '2024-09-15T21:24:06.067697+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:24:06.067697+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 9713, 'log_dups_size': 0, 'ondisk_log_size': 9713, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39736262, 'num_objects': 9703, 'num_object_clones': 0, 'num_object_copies': 19406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9703, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6417, 'num_write_kb': 25668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5556}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:07.878 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:08.564 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:08.564 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/307558 objects degraded (13.404%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":300,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":153779,"data_bytes":629834481,"bytes_used":2115497984,"bytes_avail":287794794496,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":307558,"degraded_ratio":0.13403975835452175,"misplaced_objects":3563,"misplaced_total":307558,"misplaced_ratio":0.0115848067681543,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":80194,"recovering_keys_per_sec":0,"num_objects_recovered":235,"num_bytes_recovered":962560,"num_keys_recovered":0,"write_bytes_sec":2031144,"read_op_per_sec":0,"write_op_per_sec":495},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:08.591 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:08.591 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:08.763 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 217 20 155300 155280 2.79478 1.60156 0.0425438 0.0279468 2024-09-15T21:24:09.140 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:09.143 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:09.177 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-15T21:24:08.752334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633074417,"num_objects":154570,"num_object_clones":0,"num_object_copies":309140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":154570,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":114435,"num_write_kb":457734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4685,"num_bytes_recovered":19177491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154643,"ondisk_log_size":154643,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2103452,"kb_used_data":1255980,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":281012068,"statfs":{"total":289910292480,"available":287756357632,"internally_reserved":0,"allocated":1286123520,"data_stored":1285260898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":867748867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,6,11,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21680128,"num_objects":5293,"num_object_clones":0,"num_object_copies":10586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-163,"num_objects_unfound":0,"num_objects_dirty":5293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5293,"num_write_kb":21172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":667648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003076"},"pg_stats":[{"pgid":"2.f","version":"45'9872","reported_seq":9946,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.099863+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:06.099863+0000","last_peered":"2024-09-15T21:24:06.099863+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:06.099863+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9872,"log_dups_size":0,"ondisk_log_size":9872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40435712,"num_objects":9872,"num_object_clones":0,"num_object_copies":19744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9872,"num_write_kb":39488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5740},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9734","reported_seq":9839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.484338+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:08.484338+0000","last_peered":"2024-09-15T21:24:08.484338+0000","last_clean":"2024-09-15T21:24:08.484338+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:08.484338+0000","last_undegraded":"2024-09-15T21:24:08.484338+0000","last_fullsized":"2024-09-15T21:24:08.484338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9734,"log_dups_size":0,"ondisk_log_size":9734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39846854,"num_objects":9730,"num_object_clones":0,"num_object_copies":19460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9730,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8015,"num_write_kb":32060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9933","reported_seq":10058,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.469682+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:08.469682+0000","last_peered":"2024-09-15T21:24:08.469682+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:08.469682+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9933,"log_dups_size":0,"ondisk_log_size":9933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40669184,"num_objects":9929,"num_object_clones":0,"num_object_copies":19858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8033,"num_write_kb":32132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5662}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9522","reported_seq":9592,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.467691+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:08.467691+0000","last_peered":"2024-09-15T21:24:08.467691+0000","last_clean":"2024-09-15T21:24:08.467691+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:08.467691+0000","last_undegraded":"2024-09-15T21:24:08.467691+0000","last_fullsized":"2024-09-15T21:24:08.467691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9522,"log_dups_size":0,"ondisk_log_size":9522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39002112,"num_objects":9522,"num_object_clones":0,"num_object_copies":19044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9522,"num_write_kb":38088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9587","reported_seq":9719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:06.124620+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:06.124620+0000","last_peered":"2024-09-15T21:24:06.124620+0000","last_clean":"2024-09-15T21:24:06.124620+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:06.124620+0000","last_undegraded":"2024-09-15T21:24:06.124620+0000","last_fullsized":"2024-09-15T21:24:06.124620+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9587,"log_dups_size":0,"ondisk_log_size":9587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39240163,"num_objects":9581,"num_object_clones":0,"num_object_copies":19162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6042,"num_write_kb":24168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9613","reported_seq":9749,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.021232+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:06.021232+0000","last_peered":"2024-09-15T21:24:06.021232+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:06.021232+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9613,"log_dups_size":0,"ondisk_log_size":9613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39354851,"num_objects":9609,"num_object_clones":0,"num_object_copies":19218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6289,"num_write_kb":25156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5533},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9677","reported_seq":9813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:05.996357+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:05.996357+0000","last_peered":"2024-09-15T21:24:05.996357+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:05.996357+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39613382,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5988,"num_write_kb":23952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5512},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9721","reported_seq":9849,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:06.122087+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:06.122087+0000","last_peered":"2024-09-15T21:24:06.122087+0000","last_clean":"2024-09-15T21:24:06.122087+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:06.122087+0000","last_undegraded":"2024-09-15T21:24:06.122087+0000","last_fullsized":"2024-09-15T21:24:06.122087+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9721,"log_dups_size":0,"ondisk_log_size":9721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39809024,"num_objects":9719,"num_object_clones":0,"num_object_copies":19438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6493,"num_write_kb":25972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9655","reported_seq":9781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.526362+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:08.526362+0000","last_peered":"2024-09-15T21:24:08.526362+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:08.526362+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9655,"log_dups_size":0,"ondisk_log_size":9655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39518691,"num_objects":9649,"num_object_clones":0,"num_object_copies":19298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7587,"num_write_kb":30348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5562}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9502","reported_seq":9632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.043090+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:06.043090+0000","last_peered":"2024-09-15T21:24:06.043090+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:06.043090+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9502,"log_dups_size":0,"ondisk_log_size":9502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38887424,"num_objects":9494,"num_object_clones":0,"num_object_copies":18988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6175,"num_write_kb":24700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5474},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9699","reported_seq":9816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.483151+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:08.483151+0000","last_peered":"2024-09-15T21:24:08.483151+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:08.483151+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9699,"log_dups_size":0,"ondisk_log_size":9699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39707107,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7558,"num_write_kb":30232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9682","reported_seq":18140,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.521606+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:08.521606+0000","last_peered":"2024-09-15T21:24:08.521606+0000","last_clean":"2024-09-15T21:24:08.521606+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:08.521606+0000","last_undegraded":"2024-09-15T21:24:08.521606+0000","last_fullsized":"2024-09-15T21:24:08.521606+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9682,"log_dups_size":0,"ondisk_log_size":9682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39632896,"num_objects":9676,"num_object_clones":0,"num_object_copies":19352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7375,"num_write_kb":29500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9642","reported_seq":9775,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.049535+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:06.049535+0000","last_peered":"2024-09-15T21:24:06.049535+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:06.049535+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9642,"log_dups_size":0,"ondisk_log_size":9642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39485440,"num_objects":9640,"num_object_clones":0,"num_object_copies":19280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6586,"num_write_kb":26344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5546},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9599","reported_seq":9730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.104552+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:06.104552+0000","last_peered":"2024-09-15T21:24:06.104552+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:06.104552+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9599,"log_dups_size":0,"ondisk_log_size":9599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6508,"num_write_kb":26032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5452},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9489","reported_seq":10660,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:06.137215+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:06.137215+0000","last_peered":"2024-09-15T21:24:06.137215+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:06.137215+0000","last_undegraded":"2024-09-15T21:24:06.137215+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9489,"log_dups_size":0,"ondisk_log_size":9489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38850560,"num_objects":9485,"num_object_clones":0,"num_object_copies":18970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":9485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5973,"num_write_kb":23892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":2125824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5922},{"shards":"0,2","objects":3563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9713","reported_seq":9851,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.067697+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:06.067697+0000","last_peered":"2024-09-15T21:24:06.067697+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:06.067697+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9713,"log_dups_size":0,"ondisk_log_size":9713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39736262,"num_objects":9703,"num_object_clones":0,"num_object_copies":19406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9703,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6417,"num_write_kb":25668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5556},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":633074398,"num_objects":154568,"num_object_clones":0,"num_object_copies":309136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":154568,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":114433,"num_write_kb":457732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4684,"num_bytes_recovered":19177472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1286090752,"data_stored":1285942619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154640,"ondisk_log_size":154640,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716724,"kb_used_data":442084,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93655116,"statfs":{"total":96636764160,"available":95902838784,"internally_reserved":0,"allocated":452694016,"data_stored":452422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":849144,"kb_used_data":528744,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93522696,"statfs":{"total":96636764160,"available":95767240704,"internally_reserved":0,"allocated":541433856,"data_stored":541158204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328068779},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,6,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537584,"kb_used_data":285152,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93834256,"statfs":{"total":96636764160,"available":96086278144,"internally_reserved":0,"allocated":291995648,"data_stored":291680253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":258469551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541167616,"data_stored":541131486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":292429824,"data_stored":292353951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":452493312,"data_stored":452457182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:09.763 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 218 20 155742 155722 2.78988 1.72656 0.0205851 0.0279966 2024-09-15T21:24:10.764 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 219 20 156204 156184 2.78538 1.80469 0.0273107 0.0280416 2024-09-15T21:24:11.763 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:2024-09-15T21:24:11.761955+0000 min lat: 0.00235499 max lat: 2.60418 avg lat: 0.0280858 2024-09-15T21:24:11.764 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-15T21:24:11.764 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 220 20 156667 156647 2.78094 1.80859 0.0536894 0.0280858 2024-09-15T21:24:12.180 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:12.746 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:12.747 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:12.768 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 221 20 157104 157084 2.77604 1.70703 0.0271633 0.0281342 2024-09-15T21:24:12.770 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-15T21:24:10.752849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":633074417,"num_objects":154570,"num_object_clones":0,"num_object_copies":309140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":154570,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":114435,"num_write_kb":457734,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4685,"num_bytes_recovered":19177491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154643,"ondisk_log_size":154643,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2103452,"kb_used_data":1255980,"kb_used_omap":12,"kb_used_meta":847411,"kb_avail":281012068,"statfs":{"total":289910292480,"available":287756357632,"internally_reserved":0,"allocated":1286123520,"data_stored":1285260898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":867748867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,6,11,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18202624,"num_objects":4444,"num_object_clones":0,"num_object_copies":8888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-163,"num_objects_unfound":0,"num_objects_dirty":4444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4444,"num_write_kb":17776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":667648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003093"},"pg_stats":[{"pgid":"2.f","version":"45'9872","reported_seq":9946,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.099863+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:06.099863+0000","last_peered":"2024-09-15T21:24:06.099863+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:06.099863+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9872,"log_dups_size":0,"ondisk_log_size":9872,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40435712,"num_objects":9872,"num_object_clones":0,"num_object_copies":19744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9872,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9872,"num_write_kb":39488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5740},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9734","reported_seq":9839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.484338+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:08.484338+0000","last_peered":"2024-09-15T21:24:08.484338+0000","last_clean":"2024-09-15T21:24:08.484338+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:08.484338+0000","last_undegraded":"2024-09-15T21:24:08.484338+0000","last_fullsized":"2024-09-15T21:24:08.484338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9734,"log_dups_size":0,"ondisk_log_size":9734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39846854,"num_objects":9730,"num_object_clones":0,"num_object_copies":19460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9730,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8015,"num_write_kb":32060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9933","reported_seq":10058,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.469682+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:08.469682+0000","last_peered":"2024-09-15T21:24:08.469682+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:08.469682+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9933,"log_dups_size":0,"ondisk_log_size":9933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40669184,"num_objects":9929,"num_object_clones":0,"num_object_copies":19858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8033,"num_write_kb":32132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5662}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9522","reported_seq":9592,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.467691+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:08.467691+0000","last_peered":"2024-09-15T21:24:08.467691+0000","last_clean":"2024-09-15T21:24:08.467691+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:08.467691+0000","last_undegraded":"2024-09-15T21:24:08.467691+0000","last_fullsized":"2024-09-15T21:24:08.467691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9522,"log_dups_size":0,"ondisk_log_size":9522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39002112,"num_objects":9522,"num_object_clones":0,"num_object_copies":19044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9522,"num_write_kb":38088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9587","reported_seq":9719,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:06.124620+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:06.124620+0000","last_peered":"2024-09-15T21:24:06.124620+0000","last_clean":"2024-09-15T21:24:06.124620+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:06.124620+0000","last_undegraded":"2024-09-15T21:24:06.124620+0000","last_fullsized":"2024-09-15T21:24:06.124620+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9587,"log_dups_size":0,"ondisk_log_size":9587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39240163,"num_objects":9581,"num_object_clones":0,"num_object_copies":19162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6042,"num_write_kb":24168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9613","reported_seq":9749,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.021232+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:06.021232+0000","last_peered":"2024-09-15T21:24:06.021232+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:06.021232+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9613,"log_dups_size":0,"ondisk_log_size":9613,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39354851,"num_objects":9609,"num_object_clones":0,"num_object_copies":19218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6289,"num_write_kb":25156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5533},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9677","reported_seq":9813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:05.996357+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:05.996357+0000","last_peered":"2024-09-15T21:24:05.996357+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:05.996357+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9677,"log_dups_size":0,"ondisk_log_size":9677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39613382,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5988,"num_write_kb":23952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5512},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9721","reported_seq":9849,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:06.122087+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:06.122087+0000","last_peered":"2024-09-15T21:24:06.122087+0000","last_clean":"2024-09-15T21:24:06.122087+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:06.122087+0000","last_undegraded":"2024-09-15T21:24:06.122087+0000","last_fullsized":"2024-09-15T21:24:06.122087+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9721,"log_dups_size":0,"ondisk_log_size":9721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39809024,"num_objects":9719,"num_object_clones":0,"num_object_copies":19438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6493,"num_write_kb":25972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9655","reported_seq":9781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.526362+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:08.526362+0000","last_peered":"2024-09-15T21:24:08.526362+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:08.526362+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9655,"log_dups_size":0,"ondisk_log_size":9655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39518691,"num_objects":9649,"num_object_clones":0,"num_object_copies":19298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7587,"num_write_kb":30348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5562}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9502","reported_seq":9632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.043090+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:06.043090+0000","last_peered":"2024-09-15T21:24:06.043090+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:06.043090+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9502,"log_dups_size":0,"ondisk_log_size":9502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38887424,"num_objects":9494,"num_object_clones":0,"num_object_copies":18988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6175,"num_write_kb":24700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5474},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877942+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877942+0000","last_peered":"2024-09-15T21:23:35.877942+0000","last_clean":"2024-09-15T21:23:35.877942+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877942+0000","last_undegraded":"2024-09-15T21:23:35.877942+0000","last_fullsized":"2024-09-15T21:23:35.877942+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9699","reported_seq":9816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.483151+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:08.483151+0000","last_peered":"2024-09-15T21:24:08.483151+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:08.483151+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9699,"log_dups_size":0,"ondisk_log_size":9699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39707107,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7558,"num_write_kb":30232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9682","reported_seq":18140,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.521606+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:08.521606+0000","last_peered":"2024-09-15T21:24:08.521606+0000","last_clean":"2024-09-15T21:24:08.521606+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:08.521606+0000","last_undegraded":"2024-09-15T21:24:08.521606+0000","last_fullsized":"2024-09-15T21:24:08.521606+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9682,"log_dups_size":0,"ondisk_log_size":9682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39632896,"num_objects":9676,"num_object_clones":0,"num_object_copies":19352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7375,"num_write_kb":29500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9642","reported_seq":9775,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.049535+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:06.049535+0000","last_peered":"2024-09-15T21:24:06.049535+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:06.049535+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9642,"log_dups_size":0,"ondisk_log_size":9642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39485440,"num_objects":9640,"num_object_clones":0,"num_object_copies":19280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6586,"num_write_kb":26344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5546},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9599","reported_seq":9730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.104552+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:06.104552+0000","last_peered":"2024-09-15T21:24:06.104552+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:06.104552+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9599,"log_dups_size":0,"ondisk_log_size":9599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39284736,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6508,"num_write_kb":26032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5452},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9489","reported_seq":10660,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:06.137215+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:06.137215+0000","last_peered":"2024-09-15T21:24:06.137215+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:06.137215+0000","last_undegraded":"2024-09-15T21:24:06.137215+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9489,"log_dups_size":0,"ondisk_log_size":9489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38850560,"num_objects":9485,"num_object_clones":0,"num_object_copies":18970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":9485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5973,"num_write_kb":23892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":519,"num_bytes_recovered":2125824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5922},{"shards":"0,2","objects":3563}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9713","reported_seq":9851,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:06.067697+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:06.067697+0000","last_peered":"2024-09-15T21:24:06.067697+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:06.067697+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9713,"log_dups_size":0,"ondisk_log_size":9713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39736262,"num_objects":9703,"num_object_clones":0,"num_object_copies":19406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9703,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6417,"num_write_kb":25668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5556},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":633074398,"num_objects":154568,"num_object_clones":0,"num_object_copies":309136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":154568,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":114433,"num_write_kb":457732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4684,"num_bytes_recovered":19177472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1286090752,"data_stored":1285942619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154640,"ondisk_log_size":154640,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716724,"kb_used_data":442084,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93655116,"statfs":{"total":96636764160,"available":95902838784,"internally_reserved":0,"allocated":452694016,"data_stored":452422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":849144,"kb_used_data":528744,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93522696,"statfs":{"total":96636764160,"available":95767240704,"internally_reserved":0,"allocated":541433856,"data_stored":541158204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328068779},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,6,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537584,"kb_used_data":285152,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93834256,"statfs":{"total":96636764160,"available":96086278144,"internally_reserved":0,"allocated":291995648,"data_stored":291680253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":258469551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":541167616,"data_stored":541131486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":292429824,"data_stored":292353951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":452493312,"data_stored":452457182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:12.772 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:13.338 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:13.339 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:13.375 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-15T21:24:12.753446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638743281,"num_objects":155954,"num_object_clones":0,"num_object_copies":311908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3482,"num_objects_unfound":0,"num_objects_dirty":155954,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":115819,"num_write_kb":463270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4766,"num_bytes_recovered":19509267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156027,"ondisk_log_size":156027,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2125552,"kb_used_data":1269760,"kb_used_omap":12,"kb_used_meta":855731,"kb_avail":280989968,"statfs":{"total":289910292480,"available":287733727232,"internally_reserved":0,"allocated":1300234240,"data_stored":1299371618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":876268547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,6,10,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23871488,"num_objects":5828,"num_object_clones":0,"num_object_copies":11656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":5828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5828,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003222"},"pg_stats":[{"pgid":"2.f","version":"45'10020","reported_seq":10092,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.110637+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:11.110637+0000","last_peered":"2024-09-15T21:24:11.110637+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:11.110637+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41041920,"num_objects":10020,"num_object_clones":0,"num_object_copies":20040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10020,"num_write_kb":40080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5888},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9734","reported_seq":9839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.484338+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:08.484338+0000","last_peered":"2024-09-15T21:24:08.484338+0000","last_clean":"2024-09-15T21:24:08.484338+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:08.484338+0000","last_undegraded":"2024-09-15T21:24:08.484338+0000","last_fullsized":"2024-09-15T21:24:08.484338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9734,"log_dups_size":0,"ondisk_log_size":9734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39846854,"num_objects":9730,"num_object_clones":0,"num_object_copies":19460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9730,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8015,"num_write_kb":32060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9933","reported_seq":10058,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.469682+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:08.469682+0000","last_peered":"2024-09-15T21:24:08.469682+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:08.469682+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9933,"log_dups_size":0,"ondisk_log_size":9933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40669184,"num_objects":9929,"num_object_clones":0,"num_object_copies":19858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8033,"num_write_kb":32132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5662}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9522","reported_seq":9592,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.467691+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:08.467691+0000","last_peered":"2024-09-15T21:24:08.467691+0000","last_clean":"2024-09-15T21:24:08.467691+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:08.467691+0000","last_undegraded":"2024-09-15T21:24:08.467691+0000","last_fullsized":"2024-09-15T21:24:08.467691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9522,"log_dups_size":0,"ondisk_log_size":9522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39002112,"num_objects":9522,"num_object_clones":0,"num_object_copies":19044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9522,"num_write_kb":38088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9709","reported_seq":9841,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.110232+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:11.110232+0000","last_peered":"2024-09-15T21:24:11.110232+0000","last_clean":"2024-09-15T21:24:11.110232+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:11.110232+0000","last_undegraded":"2024-09-15T21:24:11.110232+0000","last_fullsized":"2024-09-15T21:24:11.110232+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9709,"log_dups_size":0,"ondisk_log_size":9709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39739875,"num_objects":9703,"num_object_clones":0,"num_object_copies":19406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6164,"num_write_kb":24656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9750","reported_seq":9885,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.166059+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:11.166059+0000","last_peered":"2024-09-15T21:24:11.166059+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:11.166059+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9750,"log_dups_size":0,"ondisk_log_size":9750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39916003,"num_objects":9746,"num_object_clones":0,"num_object_copies":19492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6426,"num_write_kb":25704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5670},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9819","reported_seq":9955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.052145+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:11.052145+0000","last_peered":"2024-09-15T21:24:11.052145+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:11.052145+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9819,"log_dups_size":0,"ondisk_log_size":9819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40195014,"num_objects":9815,"num_object_clones":0,"num_object_copies":19630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6130,"num_write_kb":24520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5654},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9845","reported_seq":9973,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.065194+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:11.065194+0000","last_peered":"2024-09-15T21:24:11.065194+0000","last_clean":"2024-09-15T21:24:11.065194+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:11.065194+0000","last_undegraded":"2024-09-15T21:24:11.065194+0000","last_fullsized":"2024-09-15T21:24:11.065194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9845,"log_dups_size":0,"ondisk_log_size":9845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40316928,"num_objects":9843,"num_object_clones":0,"num_object_copies":19686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6617,"num_write_kb":26468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9655","reported_seq":9781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.526362+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:08.526362+0000","last_peered":"2024-09-15T21:24:08.526362+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:08.526362+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9655,"log_dups_size":0,"ondisk_log_size":9655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39518691,"num_objects":9649,"num_object_clones":0,"num_object_copies":19298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7587,"num_write_kb":30348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5562}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9637","reported_seq":9767,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.176494+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:11.176494+0000","last_peered":"2024-09-15T21:24:11.176494+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:11.176494+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9637,"log_dups_size":0,"ondisk_log_size":9637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39440384,"num_objects":9629,"num_object_clones":0,"num_object_copies":19258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6310,"num_write_kb":25240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5609},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9699","reported_seq":9816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.483151+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:08.483151+0000","last_peered":"2024-09-15T21:24:08.483151+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:08.483151+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9699,"log_dups_size":0,"ondisk_log_size":9699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39707107,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7558,"num_write_kb":30232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9682","reported_seq":18140,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.521606+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:08.521606+0000","last_peered":"2024-09-15T21:24:08.521606+0000","last_clean":"2024-09-15T21:24:08.521606+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:08.521606+0000","last_undegraded":"2024-09-15T21:24:08.521606+0000","last_fullsized":"2024-09-15T21:24:08.521606+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9682,"log_dups_size":0,"ondisk_log_size":9682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39632896,"num_objects":9676,"num_object_clones":0,"num_object_copies":19352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7375,"num_write_kb":29500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9799","reported_seq":9931,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.149504+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:11.149504+0000","last_peered":"2024-09-15T21:24:11.149504+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:11.149504+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9799,"log_dups_size":0,"ondisk_log_size":9799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40128512,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6743,"num_write_kb":26972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5703},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9735","reported_seq":9867,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.100599+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:11.100599+0000","last_peered":"2024-09-15T21:24:11.100599+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:11.100599+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9735,"log_dups_size":0,"ondisk_log_size":9735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39841792,"num_objects":9727,"num_object_clones":0,"num_object_copies":19454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6644,"num_write_kb":26576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5588},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9620","reported_seq":10953,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:11.149733+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:11.149733+0000","last_peered":"2024-09-15T21:24:11.149733+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:11.149733+0000","last_undegraded":"2024-09-15T21:24:11.149733+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9620,"log_dups_size":0,"ondisk_log_size":9620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39387136,"num_objects":9616,"num_object_clones":0,"num_object_copies":19232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3482,"num_objects_unfound":0,"num_objects_dirty":9616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6104,"num_write_kb":24416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":2457600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6134},{"shards":"0,2","objects":3482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9865","reported_seq":10003,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.124174+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:11.124174+0000","last_peered":"2024-09-15T21:24:11.124174+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:11.124174+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9865,"log_dups_size":0,"ondisk_log_size":9865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40358854,"num_objects":9855,"num_object_clones":0,"num_object_copies":19710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9855,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6569,"num_write_kb":26276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5708},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":638743262,"num_objects":155952,"num_object_clones":0,"num_object_copies":311904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3482,"num_objects_unfound":0,"num_objects_dirty":155952,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":115817,"num_write_kb":463268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4765,"num_bytes_recovered":19509248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1300197376,"data_stored":1300049243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156024,"ondisk_log_size":156024,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716724,"kb_used_data":442084,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93655116,"statfs":{"total":96636764160,"available":95902838784,"internally_reserved":0,"allocated":452694016,"data_stored":452422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859796,"kb_used_data":535236,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93512044,"statfs":{"total":96636764160,"available":95756333056,"internally_reserved":0,"allocated":548081664,"data_stored":547806012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332328619},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549032,"kb_used_data":292440,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93822808,"statfs":{"total":96636764160,"available":96074555392,"internally_reserved":0,"allocated":299458560,"data_stored":299143165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":262729391},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":547909632,"data_stored":547873502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299794432,"data_stored":299718559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":452493312,"data_stored":452457182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:13.377 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:13.377 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10020", 'reported_seq': 10092, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:11.110637+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:24:11.110637+0000', 'last_peered': '2024-09-15T21:24:11.110637+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:24:11.110637+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41041920, 'num_objects': 10020, 'num_object_clones': 0, 'num_object_copies': 20040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10020, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10020, 'num_write_kb': 40080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5888}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:13.377 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:24:13.377 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'9933", 'reported_seq': 10058, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:08.469682+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:24:08.469682+0000', 'last_peered': '2024-09-15T21:24:08.469682+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:24:08.469682+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 9933, 'log_dups_size': 0, 'ondisk_log_size': 9933, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40669184, 'num_objects': 9929, 'num_object_clones': 0, 'num_object_copies': 19858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9929, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8033, 'num_write_kb': 32132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 5662}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:13.377 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:13.377 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'9750", 'reported_seq': 9885, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:11.166059+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:24:11.166059+0000', 'last_peered': '2024-09-15T21:24:11.166059+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:24:11.166059+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 9750, 'log_dups_size': 0, 'ondisk_log_size': 9750, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39916003, 'num_objects': 9746, 'num_object_clones': 0, 'num_object_copies': 19492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9746, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6426, 'num_write_kb': 25704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5670}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:13.377 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:13.378 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'9819", 'reported_seq': 9955, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:11.052145+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:24:11.052145+0000', 'last_peered': '2024-09-15T21:24:11.052145+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:24:11.052145+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 9819, 'log_dups_size': 0, 'ondisk_log_size': 9819, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40195014, 'num_objects': 9815, 'num_object_clones': 0, 'num_object_copies': 19630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9815, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6130, 'num_write_kb': 24520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5654}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:13.378 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:24:13.378 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'9655", 'reported_seq': 9781, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:08.526362+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:24:08.526362+0000', 'last_peered': '2024-09-15T21:24:08.526362+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:24:08.526362+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 9655, 'log_dups_size': 0, 'ondisk_log_size': 9655, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39518691, 'num_objects': 9649, 'num_object_clones': 0, 'num_object_copies': 19298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9649, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7587, 'num_write_kb': 30348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 5562}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:13.378 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:13.378 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'9637", 'reported_seq': 9767, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:11.176494+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:24:11.176494+0000', 'last_peered': '2024-09-15T21:24:11.176494+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:24:11.176494+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 9637, 'log_dups_size': 0, 'ondisk_log_size': 9637, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39440384, 'num_objects': 9629, 'num_object_clones': 0, 'num_object_copies': 19258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9629, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6310, 'num_write_kb': 25240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5609}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:13.378 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:13.378 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'9699", 'reported_seq': 9816, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:08.483151+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:24:08.483151+0000', 'last_peered': '2024-09-15T21:24:08.483151+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:24:08.483151+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 9699, 'log_dups_size': 0, 'ondisk_log_size': 9699, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39707107, 'num_objects': 9695, 'num_object_clones': 0, 'num_object_copies': 19390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9695, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7558, 'num_write_kb': 30232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 5605}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:13.378 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:13.378 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'9799", 'reported_seq': 9931, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:11.149504+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:24:11.149504+0000', 'last_peered': '2024-09-15T21:24:11.149504+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:24:11.149504+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 9799, 'log_dups_size': 0, 'ondisk_log_size': 9799, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40128512, 'num_objects': 9797, 'num_object_clones': 0, 'num_object_copies': 19594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9797, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6743, 'num_write_kb': 26972, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5703}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:13.378 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:13.378 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'9735", 'reported_seq': 9867, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:11.100599+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:24:11.100599+0000', 'last_peered': '2024-09-15T21:24:11.100599+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:24:11.100599+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 9735, 'log_dups_size': 0, 'ondisk_log_size': 9735, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39841792, 'num_objects': 9727, 'num_object_clones': 0, 'num_object_copies': 19454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9727, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6644, 'num_write_kb': 26576, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5588}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:13.379 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:13.379 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'9620", 'reported_seq': 10953, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:24:11.149733+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:24:11.149733+0000', 'last_peered': '2024-09-15T21:24:11.149733+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:24:11.149733+0000', 'last_undegraded': '2024-09-15T21:24:11.149733+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 9620, 'log_dups_size': 0, 'ondisk_log_size': 9620, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39387136, 'num_objects': 9616, 'num_object_clones': 0, 'num_object_copies': 19232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3482, 'num_objects_unfound': 0, 'num_objects_dirty': 9616, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6104, 'num_write_kb': 24416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 600, 'num_bytes_recovered': 2457600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6134}, {'shards': '0,2', 'objects': 3482}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:13.379 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:24:13.379 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'9865", 'reported_seq': 10003, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:11.124174+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:24:11.124174+0000', 'last_peered': '2024-09-15T21:24:11.124174+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:24:11.124174+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 9865, 'log_dups_size': 0, 'ondisk_log_size': 9865, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40358854, 'num_objects': 9855, 'num_object_clones': 0, 'num_object_copies': 19710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9855, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6569, 'num_write_kb': 26276, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5708}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:13.379 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:13.768 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 222 20 157509 157489 2.77066 1.58203 0.0294442 0.0281896 2024-09-15T21:24:14.105 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:14.105 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/311908 objects degraded (13.217%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":305,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":155954,"data_bytes":638743281,"bytes_used":2176565248,"bytes_avail":287733727232,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":311908,"degraded_ratio":0.13217038357464381,"misplaced_objects":3482,"misplaced_total":311908,"misplaced_ratio":0.011163548225758877,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":83262,"recovering_keys_per_sec":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"write_bytes_sec":1988756,"read_op_per_sec":0,"write_op_per_sec":485},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:14.126 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:14.127 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:14.666 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:14.693 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-15T21:24:12.753446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638743281,"num_objects":155954,"num_object_clones":0,"num_object_copies":311908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3482,"num_objects_unfound":0,"num_objects_dirty":155954,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":115819,"num_write_kb":463270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4766,"num_bytes_recovered":19509267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156027,"ondisk_log_size":156027,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2125552,"kb_used_data":1269760,"kb_used_omap":12,"kb_used_meta":855731,"kb_avail":280989968,"statfs":{"total":289910292480,"available":287733727232,"internally_reserved":0,"allocated":1300234240,"data_stored":1299371618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":876268547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,6,10,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23871488,"num_objects":5828,"num_object_clones":0,"num_object_copies":11656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":5828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5828,"num_write_kb":23312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003222"},"pg_stats":[{"pgid":"2.f","version":"45'10020","reported_seq":10092,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.110637+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:11.110637+0000","last_peered":"2024-09-15T21:24:11.110637+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:11.110637+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41041920,"num_objects":10020,"num_object_clones":0,"num_object_copies":20040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10020,"num_write_kb":40080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5888},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9734","reported_seq":9839,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.484338+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:08.484338+0000","last_peered":"2024-09-15T21:24:08.484338+0000","last_clean":"2024-09-15T21:24:08.484338+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:08.484338+0000","last_undegraded":"2024-09-15T21:24:08.484338+0000","last_fullsized":"2024-09-15T21:24:08.484338+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9734,"log_dups_size":0,"ondisk_log_size":9734,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39846854,"num_objects":9730,"num_object_clones":0,"num_object_copies":19460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9730,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8015,"num_write_kb":32060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9933","reported_seq":10058,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.469682+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:08.469682+0000","last_peered":"2024-09-15T21:24:08.469682+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:08.469682+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":9933,"log_dups_size":0,"ondisk_log_size":9933,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40669184,"num_objects":9929,"num_object_clones":0,"num_object_copies":19858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8033,"num_write_kb":32132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5662}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9522","reported_seq":9592,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.467691+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:08.467691+0000","last_peered":"2024-09-15T21:24:08.467691+0000","last_clean":"2024-09-15T21:24:08.467691+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:08.467691+0000","last_undegraded":"2024-09-15T21:24:08.467691+0000","last_fullsized":"2024-09-15T21:24:08.467691+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9522,"log_dups_size":0,"ondisk_log_size":9522,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39002112,"num_objects":9522,"num_object_clones":0,"num_object_copies":19044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9522,"num_write_kb":38088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9709","reported_seq":9841,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.110232+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:11.110232+0000","last_peered":"2024-09-15T21:24:11.110232+0000","last_clean":"2024-09-15T21:24:11.110232+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:11.110232+0000","last_undegraded":"2024-09-15T21:24:11.110232+0000","last_fullsized":"2024-09-15T21:24:11.110232+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9709,"log_dups_size":0,"ondisk_log_size":9709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39739875,"num_objects":9703,"num_object_clones":0,"num_object_copies":19406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6164,"num_write_kb":24656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9750","reported_seq":9885,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.166059+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:11.166059+0000","last_peered":"2024-09-15T21:24:11.166059+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:11.166059+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9750,"log_dups_size":0,"ondisk_log_size":9750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39916003,"num_objects":9746,"num_object_clones":0,"num_object_copies":19492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6426,"num_write_kb":25704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5670},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9819","reported_seq":9955,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.052145+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:11.052145+0000","last_peered":"2024-09-15T21:24:11.052145+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:11.052145+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9819,"log_dups_size":0,"ondisk_log_size":9819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40195014,"num_objects":9815,"num_object_clones":0,"num_object_copies":19630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6130,"num_write_kb":24520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5654},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9845","reported_seq":9973,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.065194+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:11.065194+0000","last_peered":"2024-09-15T21:24:11.065194+0000","last_clean":"2024-09-15T21:24:11.065194+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:11.065194+0000","last_undegraded":"2024-09-15T21:24:11.065194+0000","last_fullsized":"2024-09-15T21:24:11.065194+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9845,"log_dups_size":0,"ondisk_log_size":9845,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40316928,"num_objects":9843,"num_object_clones":0,"num_object_copies":19686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6617,"num_write_kb":26468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9655","reported_seq":9781,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.526362+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:08.526362+0000","last_peered":"2024-09-15T21:24:08.526362+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:08.526362+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9655,"log_dups_size":0,"ondisk_log_size":9655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39518691,"num_objects":9649,"num_object_clones":0,"num_object_copies":19298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7587,"num_write_kb":30348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5562}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9637","reported_seq":9767,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.176494+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:11.176494+0000","last_peered":"2024-09-15T21:24:11.176494+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:11.176494+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9637,"log_dups_size":0,"ondisk_log_size":9637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39440384,"num_objects":9629,"num_object_clones":0,"num_object_copies":19258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6310,"num_write_kb":25240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5609},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9699","reported_seq":9816,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:08.483151+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:08.483151+0000","last_peered":"2024-09-15T21:24:08.483151+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:08.483151+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9699,"log_dups_size":0,"ondisk_log_size":9699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39707107,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7558,"num_write_kb":30232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9682","reported_seq":18140,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:08.521606+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:08.521606+0000","last_peered":"2024-09-15T21:24:08.521606+0000","last_clean":"2024-09-15T21:24:08.521606+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:08.521606+0000","last_undegraded":"2024-09-15T21:24:08.521606+0000","last_fullsized":"2024-09-15T21:24:08.521606+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9682,"log_dups_size":0,"ondisk_log_size":9682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39632896,"num_objects":9676,"num_object_clones":0,"num_object_copies":19352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7375,"num_write_kb":29500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9799","reported_seq":9931,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.149504+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:11.149504+0000","last_peered":"2024-09-15T21:24:11.149504+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:11.149504+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9799,"log_dups_size":0,"ondisk_log_size":9799,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40128512,"num_objects":9797,"num_object_clones":0,"num_object_copies":19594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6743,"num_write_kb":26972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5703},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9735","reported_seq":9867,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.100599+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:11.100599+0000","last_peered":"2024-09-15T21:24:11.100599+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:11.100599+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9735,"log_dups_size":0,"ondisk_log_size":9735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39841792,"num_objects":9727,"num_object_clones":0,"num_object_copies":19454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6644,"num_write_kb":26576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5588},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9620","reported_seq":10953,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:11.149733+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:11.149733+0000","last_peered":"2024-09-15T21:24:11.149733+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:11.149733+0000","last_undegraded":"2024-09-15T21:24:11.149733+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9620,"log_dups_size":0,"ondisk_log_size":9620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39387136,"num_objects":9616,"num_object_clones":0,"num_object_copies":19232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3482,"num_objects_unfound":0,"num_objects_dirty":9616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6104,"num_write_kb":24416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":600,"num_bytes_recovered":2457600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6134},{"shards":"0,2","objects":3482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9865","reported_seq":10003,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:11.124174+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:11.124174+0000","last_peered":"2024-09-15T21:24:11.124174+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:11.124174+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":9865,"log_dups_size":0,"ondisk_log_size":9865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40358854,"num_objects":9855,"num_object_clones":0,"num_object_copies":19710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9855,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6569,"num_write_kb":26276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5708},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":638743262,"num_objects":155952,"num_object_clones":0,"num_object_copies":311904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3482,"num_objects_unfound":0,"num_objects_dirty":155952,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":115817,"num_write_kb":463268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4765,"num_bytes_recovered":19509248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1300197376,"data_stored":1300049243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156024,"ondisk_log_size":156024,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716724,"kb_used_data":442084,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93655116,"statfs":{"total":96636764160,"available":95902838784,"internally_reserved":0,"allocated":452694016,"data_stored":452422441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859796,"kb_used_data":535236,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93512044,"statfs":{"total":96636764160,"available":95756333056,"internally_reserved":0,"allocated":548081664,"data_stored":547806012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332328619},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549032,"kb_used_data":292440,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93822808,"statfs":{"total":96636764160,"available":96074555392,"internally_reserved":0,"allocated":299458560,"data_stored":299143165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":262729391},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":547909632,"data_stored":547873502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299794432,"data_stored":299718559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":452493312,"data_stored":452457182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:14.768 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 223 20 157925 157905 2.76552 1.625 0.0473849 0.0282428 2024-09-15T21:24:15.768 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 224 20 158393 158373 2.76133 1.82812 0.0278229 0.0282864 2024-09-15T21:24:16.768 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 225 20 158858 158838 2.75713 1.81641 0.0507086 0.028328 2024-09-15T21:24:17.696 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:17.768 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 226 20 159318 159298 2.75289 1.79688 0.0240965 0.0283714 2024-09-15T21:24:18.263 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:18.264 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:18.287 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-15T21:24:16.754411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":647820017,"num_objects":158170,"num_object_clones":0,"num_object_copies":316340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3398,"num_objects_unfound":0,"num_objects_dirty":158170,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":118035,"num_write_kb":472134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4850,"num_bytes_recovered":19853331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158143,"ondisk_log_size":158143,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2144844,"kb_used_data":1280732,"kb_used_omap":12,"kb_used_meta":864051,"kb_avail":280970676,"statfs":{"total":289910292480,"available":287713972224,"internally_reserved":0,"allocated":1311469568,"data_stored":1310606946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":884788227},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,5,13,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23781376,"num_objects":5806,"num_object_clones":0,"num_object_copies":11612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":5806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5806,"num_write_kb":23224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003283"},"pg_stats":[{"pgid":"2.f","version":"45'10164","reported_seq":10238,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.267057+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:16.267057+0000","last_peered":"2024-09-15T21:24:16.267057+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:16.267057+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41631744,"num_objects":10164,"num_object_clones":0,"num_object_copies":20328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10164,"num_write_kb":40656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6032},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9867","reported_seq":9972,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.520633+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:13.520633+0000","last_peered":"2024-09-15T21:24:13.520633+0000","last_clean":"2024-09-15T21:24:13.520633+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:13.520633+0000","last_undegraded":"2024-09-15T21:24:13.520633+0000","last_fullsized":"2024-09-15T21:24:13.520633+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":9867,"log_dups_size":0,"ondisk_log_size":9867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40391622,"num_objects":9863,"num_object_clones":0,"num_object_copies":19726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9863,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8148,"num_write_kb":32592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10059","reported_seq":10184,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.492438+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:13.492438+0000","last_peered":"2024-09-15T21:24:13.492438+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:13.492438+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41185280,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8159,"num_write_kb":32636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5788}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9662","reported_seq":9732,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.567311+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:13.567311+0000","last_peered":"2024-09-15T21:24:13.567311+0000","last_clean":"2024-09-15T21:24:13.567311+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:13.567311+0000","last_undegraded":"2024-09-15T21:24:13.567311+0000","last_fullsized":"2024-09-15T21:24:13.567311+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9662,"log_dups_size":0,"ondisk_log_size":9662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39575552,"num_objects":9662,"num_object_clones":0,"num_object_copies":19324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9662,"num_write_kb":38648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9847","reported_seq":9979,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:16.176139+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:16.176139+0000","last_peered":"2024-09-15T21:24:16.176139+0000","last_clean":"2024-09-15T21:24:16.176139+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:16.176139+0000","last_undegraded":"2024-09-15T21:24:16.176139+0000","last_fullsized":"2024-09-15T21:24:16.176139+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9847,"log_dups_size":0,"ondisk_log_size":9847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40305123,"num_objects":9841,"num_object_clones":0,"num_object_copies":19682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6302,"num_write_kb":25208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9901","reported_seq":10037,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.115894+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:16.115894+0000","last_peered":"2024-09-15T21:24:16.115894+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:16.115894+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9901,"log_dups_size":0,"ondisk_log_size":9901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40534499,"num_objects":9897,"num_object_clones":0,"num_object_copies":19794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6577,"num_write_kb":26308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5821},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9947","reported_seq":10083,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.154828+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:16.154828+0000","last_peered":"2024-09-15T21:24:16.154828+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:16.154828+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9947,"log_dups_size":0,"ondisk_log_size":9947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40719302,"num_objects":9943,"num_object_clones":0,"num_object_copies":19886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6258,"num_write_kb":25032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5782},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9986","reported_seq":10113,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:16.233295+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:16.233295+0000","last_peered":"2024-09-15T21:24:16.233295+0000","last_clean":"2024-09-15T21:24:16.233295+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:16.233295+0000","last_undegraded":"2024-09-15T21:24:16.233295+0000","last_fullsized":"2024-09-15T21:24:16.233295+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9986,"log_dups_size":0,"ondisk_log_size":9986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894464,"num_objects":9984,"num_object_clones":0,"num_object_copies":19968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6758,"num_write_kb":27032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9787","reported_seq":9913,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.534956+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:13.534956+0000","last_peered":"2024-09-15T21:24:13.534956+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:13.534956+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9787,"log_dups_size":0,"ondisk_log_size":9787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40059363,"num_objects":9781,"num_object_clones":0,"num_object_copies":19562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7719,"num_write_kb":30876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5694}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9794","reported_seq":9925,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.087458+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:16.087458+0000","last_peered":"2024-09-15T21:24:16.087458+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:16.087458+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9794,"log_dups_size":0,"ondisk_log_size":9794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40083456,"num_objects":9786,"num_object_clones":0,"num_object_copies":19572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6467,"num_write_kb":25868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5766},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9842","reported_seq":9960,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:13.561292+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:13.561292+0000","last_peered":"2024-09-15T21:24:13.561292+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:13.561292+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9842,"log_dups_size":0,"ondisk_log_size":9842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40292835,"num_objects":9838,"num_object_clones":0,"num_object_copies":19676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9838,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7701,"num_write_kb":30804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5748}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9835","reported_seq":18293,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:13.532456+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:13.532456+0000","last_peered":"2024-09-15T21:24:13.532456+0000","last_clean":"2024-09-15T21:24:13.532456+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:13.532456+0000","last_undegraded":"2024-09-15T21:24:13.532456+0000","last_fullsized":"2024-09-15T21:24:13.532456+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9835,"log_dups_size":0,"ondisk_log_size":9835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40259584,"num_objects":9829,"num_object_clones":0,"num_object_copies":19658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7528,"num_write_kb":30112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9917","reported_seq":10050,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.121649+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:16.121649+0000","last_peered":"2024-09-15T21:24:16.121649+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:16.121649+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9917,"log_dups_size":0,"ondisk_log_size":9917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40611840,"num_objects":9915,"num_object_clones":0,"num_object_copies":19830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6861,"num_write_kb":27444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5821},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9861","reported_seq":9993,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.125576+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:16.125576+0000","last_peered":"2024-09-15T21:24:16.125576+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:16.125576+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9861,"log_dups_size":0,"ondisk_log_size":9861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40357888,"num_objects":9853,"num_object_clones":0,"num_object_copies":19706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6770,"num_write_kb":27080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9758","reported_seq":11262,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:16.252414+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:16.252414+0000","last_peered":"2024-09-15T21:24:16.252414+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:16.252414+0000","last_undegraded":"2024-09-15T21:24:16.252414+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9758,"log_dups_size":0,"ondisk_log_size":9758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39952384,"num_objects":9754,"num_object_clones":0,"num_object_copies":19508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3398,"num_objects_unfound":0,"num_objects_dirty":9754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6242,"num_write_kb":24968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":684,"num_bytes_recovered":2801664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6356},{"shards":"0,2","objects":3398}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10013","reported_seq":10150,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.233273+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:16.233273+0000","last_peered":"2024-09-15T21:24:16.233273+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:16.233273+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40965062,"num_objects":10003,"num_object_clones":0,"num_object_copies":20006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10003,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6717,"num_write_kb":26868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5856},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":647819998,"num_objects":158168,"num_object_clones":0,"num_object_copies":316336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3398,"num_objects_unfound":0,"num_objects_dirty":158168,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":118033,"num_write_kb":472132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4849,"num_bytes_recovered":19853312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1311514624,"data_stored":1311366491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158140,"ondisk_log_size":158140,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":725252,"kb_used_data":446452,"kb_used_omap":4,"kb_used_meta":278779,"kb_avail":93646588,"statfs":{"total":96636764160,"available":95894106112,"internally_reserved":0,"allocated":457166848,"data_stored":456895273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":285470377},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":870560,"kb_used_data":541840,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93501280,"statfs":{"total":96636764160,"available":95745310720,"internally_reserved":0,"allocated":554844160,"data_stored":554568508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336588459},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549032,"kb_used_data":292440,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93822808,"statfs":{"total":96636764160,"available":96074555392,"internally_reserved":0,"allocated":299458560,"data_stored":299143165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":262729391},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554790912,"data_stored":554754782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299794432,"data_stored":299718559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":456929280,"data_stored":456893150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:18.289 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:18.771 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 227 20 159731 159711 2.74783 1.61328 0.0400116 0.0284245 2024-09-15T21:24:18.837 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:18.840 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:18.869 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-15T21:24:18.755056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651199217,"num_objects":158995,"num_object_clones":0,"num_object_copies":317990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3398,"num_objects_unfound":0,"num_objects_dirty":158995,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":118860,"num_write_kb":475434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4850,"num_bytes_recovered":19853331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158768,"ondisk_log_size":158768,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2126988,"kb_used_data":1291996,"kb_used_omap":12,"kb_used_meta":834931,"kb_avail":280988532,"statfs":{"total":289910292480,"available":287732256768,"internally_reserved":0,"allocated":1323003904,"data_stored":1322141282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":854969347},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,9,10,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21364736,"num_objects":5216,"num_object_clones":0,"num_object_copies":10432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-165,"num_objects_unfound":0,"num_objects_dirty":5216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5216,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":675840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003470"},"pg_stats":[{"pgid":"2.f","version":"45'10164","reported_seq":10238,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.267057+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:16.267057+0000","last_peered":"2024-09-15T21:24:16.267057+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:16.267057+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41631744,"num_objects":10164,"num_object_clones":0,"num_object_copies":20328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10164,"num_write_kb":40656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6032},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10012","reported_seq":10117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.488321+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:18.488321+0000","last_peered":"2024-09-15T21:24:18.488321+0000","last_clean":"2024-09-15T21:24:18.488321+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:18.488321+0000","last_undegraded":"2024-09-15T21:24:18.488321+0000","last_fullsized":"2024-09-15T21:24:18.488321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40985542,"num_objects":10008,"num_object_clones":0,"num_object_copies":20016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10008,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8293,"num_write_kb":33172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10204","reported_seq":10327,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.497334+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:18.497334+0000","last_peered":"2024-09-15T21:24:18.497334+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:18.497334+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41779200,"num_objects":10200,"num_object_clones":0,"num_object_copies":20400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8304,"num_write_kb":33216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9805","reported_seq":9873,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.549772+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:18.549772+0000","last_peered":"2024-09-15T21:24:18.549772+0000","last_clean":"2024-09-15T21:24:18.549772+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:18.549772+0000","last_undegraded":"2024-09-15T21:24:18.549772+0000","last_fullsized":"2024-09-15T21:24:18.549772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9805,"log_dups_size":0,"ondisk_log_size":9805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40161280,"num_objects":9805,"num_object_clones":0,"num_object_copies":19610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9805,"num_write_kb":39220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9847","reported_seq":9979,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:16.176139+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:16.176139+0000","last_peered":"2024-09-15T21:24:16.176139+0000","last_clean":"2024-09-15T21:24:16.176139+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:16.176139+0000","last_undegraded":"2024-09-15T21:24:16.176139+0000","last_fullsized":"2024-09-15T21:24:16.176139+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9847,"log_dups_size":0,"ondisk_log_size":9847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40305123,"num_objects":9841,"num_object_clones":0,"num_object_copies":19682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6302,"num_write_kb":25208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9901","reported_seq":10037,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.115894+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:16.115894+0000","last_peered":"2024-09-15T21:24:16.115894+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:16.115894+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9901,"log_dups_size":0,"ondisk_log_size":9901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40534499,"num_objects":9897,"num_object_clones":0,"num_object_copies":19794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6577,"num_write_kb":26308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5821},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9947","reported_seq":10083,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.154828+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:16.154828+0000","last_peered":"2024-09-15T21:24:16.154828+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:16.154828+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9947,"log_dups_size":0,"ondisk_log_size":9947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40719302,"num_objects":9943,"num_object_clones":0,"num_object_copies":19886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6258,"num_write_kb":25032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5782},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9986","reported_seq":10113,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:16.233295+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:16.233295+0000","last_peered":"2024-09-15T21:24:16.233295+0000","last_clean":"2024-09-15T21:24:16.233295+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:16.233295+0000","last_undegraded":"2024-09-15T21:24:16.233295+0000","last_fullsized":"2024-09-15T21:24:16.233295+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9986,"log_dups_size":0,"ondisk_log_size":9986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894464,"num_objects":9984,"num_object_clones":0,"num_object_copies":19968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6758,"num_write_kb":27032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9922","reported_seq":10048,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.545536+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:18.545536+0000","last_peered":"2024-09-15T21:24:18.545536+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:18.545536+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9922,"log_dups_size":0,"ondisk_log_size":9922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40612323,"num_objects":9916,"num_object_clones":0,"num_object_copies":19832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7854,"num_write_kb":31416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5829}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9794","reported_seq":9925,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.087458+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:16.087458+0000","last_peered":"2024-09-15T21:24:16.087458+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:16.087458+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9794,"log_dups_size":0,"ondisk_log_size":9794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40083456,"num_objects":9786,"num_object_clones":0,"num_object_copies":19572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6467,"num_write_kb":25868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5766},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9966","reported_seq":10084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.528950+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:18.528950+0000","last_peered":"2024-09-15T21:24:18.528950+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:18.528950+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9966,"log_dups_size":0,"ondisk_log_size":9966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40800739,"num_objects":9962,"num_object_clones":0,"num_object_copies":19924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7825,"num_write_kb":31300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5872}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9968","reported_seq":18426,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.512470+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:18.512470+0000","last_peered":"2024-09-15T21:24:18.512470+0000","last_clean":"2024-09-15T21:24:18.512470+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:18.512470+0000","last_undegraded":"2024-09-15T21:24:18.512470+0000","last_fullsized":"2024-09-15T21:24:18.512470+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9968,"log_dups_size":0,"ondisk_log_size":9968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40804352,"num_objects":9962,"num_object_clones":0,"num_object_copies":19924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7661,"num_write_kb":30644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9917","reported_seq":10050,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.121649+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:16.121649+0000","last_peered":"2024-09-15T21:24:16.121649+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:16.121649+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9917,"log_dups_size":0,"ondisk_log_size":9917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40611840,"num_objects":9915,"num_object_clones":0,"num_object_copies":19830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6861,"num_write_kb":27444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5821},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9861","reported_seq":9993,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.125576+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:16.125576+0000","last_peered":"2024-09-15T21:24:16.125576+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:16.125576+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9861,"log_dups_size":0,"ondisk_log_size":9861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40357888,"num_objects":9853,"num_object_clones":0,"num_object_copies":19706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6770,"num_write_kb":27080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9758","reported_seq":11262,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:16.252414+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:16.252414+0000","last_peered":"2024-09-15T21:24:16.252414+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:16.252414+0000","last_undegraded":"2024-09-15T21:24:16.252414+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9758,"log_dups_size":0,"ondisk_log_size":9758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39952384,"num_objects":9754,"num_object_clones":0,"num_object_copies":19508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3398,"num_objects_unfound":0,"num_objects_dirty":9754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6242,"num_write_kb":24968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":684,"num_bytes_recovered":2801664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6356},{"shards":"0,2","objects":3398}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10013","reported_seq":10150,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.233273+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:16.233273+0000","last_peered":"2024-09-15T21:24:16.233273+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:16.233273+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40965062,"num_objects":10003,"num_object_clones":0,"num_object_copies":20006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10003,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6717,"num_write_kb":26868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5856},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":651199198,"num_objects":158993,"num_object_clones":0,"num_object_copies":317986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3398,"num_objects_unfound":0,"num_objects_dirty":158993,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":118858,"num_write_kb":475432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4849,"num_bytes_recovered":19853312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1323192320,"data_stored":1323044187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158765,"ondisk_log_size":158765,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713012,"kb_used_data":450852,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93658828,"statfs":{"total":96636764160,"available":95906639872,"internally_reserved":0,"allocated":461672448,"data_stored":461400873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":870560,"kb_used_data":541840,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93501280,"statfs":{"total":96636764160,"available":95745310720,"internally_reserved":0,"allocated":554844160,"data_stored":554568508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336588459},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543416,"kb_used_data":299304,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93828424,"statfs":{"total":96636764160,"available":96080306176,"internally_reserved":0,"allocated":306487296,"data_stored":306171901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":249949871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,5,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554790912,"data_stored":554754782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306896896,"data_stored":306821023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":461504512,"data_stored":461468382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:18.870 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:18.870 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10164", 'reported_seq': 10238, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:16.267057+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:24:16.267057+0000', 'last_peered': '2024-09-15T21:24:16.267057+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:24:16.267057+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41631744, 'num_objects': 10164, 'num_object_clones': 0, 'num_object_copies': 20328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10164, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10164, 'num_write_kb': 40656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6032}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:18.870 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:24:18.870 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'10204", 'reported_seq': 10327, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:18.497334+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:24:18.497334+0000', 'last_peered': '2024-09-15T21:24:18.497334+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:24:18.497334+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41779200, 'num_objects': 10200, 'num_object_clones': 0, 'num_object_copies': 20400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8304, 'num_write_kb': 33216, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 5933}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:18.870 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:18.871 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'9901", 'reported_seq': 10037, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:16.115894+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:24:16.115894+0000', 'last_peered': '2024-09-15T21:24:16.115894+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:24:16.115894+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 9901, 'log_dups_size': 0, 'ondisk_log_size': 9901, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40534499, 'num_objects': 9897, 'num_object_clones': 0, 'num_object_copies': 19794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9897, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6577, 'num_write_kb': 26308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5821}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:18.871 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:18.871 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'9947", 'reported_seq': 10083, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:16.154828+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:24:16.154828+0000', 'last_peered': '2024-09-15T21:24:16.154828+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:24:16.154828+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 9947, 'log_dups_size': 0, 'ondisk_log_size': 9947, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40719302, 'num_objects': 9943, 'num_object_clones': 0, 'num_object_copies': 19886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9943, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6258, 'num_write_kb': 25032, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5782}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:18.871 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:24:18.871 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'9922", 'reported_seq': 10048, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:18.545536+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:24:18.545536+0000', 'last_peered': '2024-09-15T21:24:18.545536+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:24:18.545536+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 9922, 'log_dups_size': 0, 'ondisk_log_size': 9922, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40612323, 'num_objects': 9916, 'num_object_clones': 0, 'num_object_copies': 19832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9916, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7854, 'num_write_kb': 31416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 5829}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:18.871 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:18.871 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'9794", 'reported_seq': 9925, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:16.087458+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:24:16.087458+0000', 'last_peered': '2024-09-15T21:24:16.087458+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:24:16.087458+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 9794, 'log_dups_size': 0, 'ondisk_log_size': 9794, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40083456, 'num_objects': 9786, 'num_object_clones': 0, 'num_object_copies': 19572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9786, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6467, 'num_write_kb': 25868, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5766}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:18.871 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:18.871 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'9966", 'reported_seq': 10084, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:18.528950+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:24:18.528950+0000', 'last_peered': '2024-09-15T21:24:18.528950+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:24:18.528950+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 9966, 'log_dups_size': 0, 'ondisk_log_size': 9966, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40800739, 'num_objects': 9962, 'num_object_clones': 0, 'num_object_copies': 19924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9962, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7825, 'num_write_kb': 31300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 5872}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:18.871 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:18.871 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'9917", 'reported_seq': 10050, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:16.121649+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:24:16.121649+0000', 'last_peered': '2024-09-15T21:24:16.121649+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:24:16.121649+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 9917, 'log_dups_size': 0, 'ondisk_log_size': 9917, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40611840, 'num_objects': 9915, 'num_object_clones': 0, 'num_object_copies': 19830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9915, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6861, 'num_write_kb': 27444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5821}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:18.872 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:18.872 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'9861", 'reported_seq': 9993, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:16.125576+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:24:16.125576+0000', 'last_peered': '2024-09-15T21:24:16.125576+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:24:16.125576+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 9861, 'log_dups_size': 0, 'ondisk_log_size': 9861, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40357888, 'num_objects': 9853, 'num_object_clones': 0, 'num_object_copies': 19706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6770, 'num_write_kb': 27080, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5714}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:18.872 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:18.872 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'9758", 'reported_seq': 11262, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:24:16.252414+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:24:16.252414+0000', 'last_peered': '2024-09-15T21:24:16.252414+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:24:16.252414+0000', 'last_undegraded': '2024-09-15T21:24:16.252414+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 9758, 'log_dups_size': 0, 'ondisk_log_size': 9758, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39952384, 'num_objects': 9754, 'num_object_clones': 0, 'num_object_copies': 19508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3398, 'num_objects_unfound': 0, 'num_objects_dirty': 9754, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6242, 'num_write_kb': 24968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 684, 'num_bytes_recovered': 2801664, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6356}, {'shards': '0,2', 'objects': 3398}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:18.872 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:24:18.872 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10013", 'reported_seq': 10150, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:16.233273+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:24:16.233273+0000', 'last_peered': '2024-09-15T21:24:16.233273+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:24:16.233273+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40965062, 'num_objects': 10003, 'num_object_clones': 0, 'num_object_copies': 20006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10003, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6717, 'num_write_kb': 26868, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5856}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:18.872 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:19.554 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:19.554 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/317990 objects degraded (12.964%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":311,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158995,"data_bytes":651199217,"bytes_used":2178035712,"bytes_avail":287732256768,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":317990,"degraded_ratio":0.12964244158621341,"misplaced_objects":3398,"misplaced_total":317990,"misplaced_ratio":0.010685870624862417,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":56303,"recovering_keys_per_sec":0,"num_objects_recovered":165,"num_bytes_recovered":675840,"num_keys_recovered":0,"write_bytes_sec":1779879,"read_op_per_sec":0,"write_op_per_sec":434},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:19.585 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:19.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:19.771 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 228 20 160130 160110 2.74262 1.55859 0.0226651 0.0284767 2024-09-15T21:24:20.150 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:20.150 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:20.174 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-15T21:24:18.755056+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651199217,"num_objects":158995,"num_object_clones":0,"num_object_copies":317990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3398,"num_objects_unfound":0,"num_objects_dirty":158995,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":118860,"num_write_kb":475434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4850,"num_bytes_recovered":19853331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158768,"ondisk_log_size":158768,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2126988,"kb_used_data":1291996,"kb_used_omap":12,"kb_used_meta":834931,"kb_avail":280988532,"statfs":{"total":289910292480,"available":287732256768,"internally_reserved":0,"allocated":1323003904,"data_stored":1322141282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":854969347},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,9,10,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21364736,"num_objects":5216,"num_object_clones":0,"num_object_copies":10432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-165,"num_objects_unfound":0,"num_objects_dirty":5216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5216,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":165,"num_bytes_recovered":675840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003470"},"pg_stats":[{"pgid":"2.f","version":"45'10164","reported_seq":10238,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.267057+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:16.267057+0000","last_peered":"2024-09-15T21:24:16.267057+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:16.267057+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41631744,"num_objects":10164,"num_object_clones":0,"num_object_copies":20328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10164,"num_write_kb":40656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6032},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10012","reported_seq":10117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.488321+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:18.488321+0000","last_peered":"2024-09-15T21:24:18.488321+0000","last_clean":"2024-09-15T21:24:18.488321+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:18.488321+0000","last_undegraded":"2024-09-15T21:24:18.488321+0000","last_fullsized":"2024-09-15T21:24:18.488321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40985542,"num_objects":10008,"num_object_clones":0,"num_object_copies":20016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10008,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8293,"num_write_kb":33172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10204","reported_seq":10327,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.497334+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:18.497334+0000","last_peered":"2024-09-15T21:24:18.497334+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:18.497334+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41779200,"num_objects":10200,"num_object_clones":0,"num_object_copies":20400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8304,"num_write_kb":33216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9805","reported_seq":9873,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.549772+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:18.549772+0000","last_peered":"2024-09-15T21:24:18.549772+0000","last_clean":"2024-09-15T21:24:18.549772+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:18.549772+0000","last_undegraded":"2024-09-15T21:24:18.549772+0000","last_fullsized":"2024-09-15T21:24:18.549772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9805,"log_dups_size":0,"ondisk_log_size":9805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40161280,"num_objects":9805,"num_object_clones":0,"num_object_copies":19610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9805,"num_write_kb":39220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9847","reported_seq":9979,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:16.176139+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:16.176139+0000","last_peered":"2024-09-15T21:24:16.176139+0000","last_clean":"2024-09-15T21:24:16.176139+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:16.176139+0000","last_undegraded":"2024-09-15T21:24:16.176139+0000","last_fullsized":"2024-09-15T21:24:16.176139+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9847,"log_dups_size":0,"ondisk_log_size":9847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40305123,"num_objects":9841,"num_object_clones":0,"num_object_copies":19682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6302,"num_write_kb":25208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9901","reported_seq":10037,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.115894+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:16.115894+0000","last_peered":"2024-09-15T21:24:16.115894+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:16.115894+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":9901,"log_dups_size":0,"ondisk_log_size":9901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40534499,"num_objects":9897,"num_object_clones":0,"num_object_copies":19794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6577,"num_write_kb":26308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5821},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9947","reported_seq":10083,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.154828+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:16.154828+0000","last_peered":"2024-09-15T21:24:16.154828+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:16.154828+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":9947,"log_dups_size":0,"ondisk_log_size":9947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40719302,"num_objects":9943,"num_object_clones":0,"num_object_copies":19886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6258,"num_write_kb":25032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5782},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9986","reported_seq":10113,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:16.233295+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:16.233295+0000","last_peered":"2024-09-15T21:24:16.233295+0000","last_clean":"2024-09-15T21:24:16.233295+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:16.233295+0000","last_undegraded":"2024-09-15T21:24:16.233295+0000","last_fullsized":"2024-09-15T21:24:16.233295+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":9986,"log_dups_size":0,"ondisk_log_size":9986,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894464,"num_objects":9984,"num_object_clones":0,"num_object_copies":19968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6758,"num_write_kb":27032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9922","reported_seq":10048,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.545536+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:18.545536+0000","last_peered":"2024-09-15T21:24:18.545536+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:18.545536+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9922,"log_dups_size":0,"ondisk_log_size":9922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40612323,"num_objects":9916,"num_object_clones":0,"num_object_copies":19832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7854,"num_write_kb":31416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5829}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9794","reported_seq":9925,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.087458+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:16.087458+0000","last_peered":"2024-09-15T21:24:16.087458+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:16.087458+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9794,"log_dups_size":0,"ondisk_log_size":9794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40083456,"num_objects":9786,"num_object_clones":0,"num_object_copies":19572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6467,"num_write_kb":25868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5766},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9966","reported_seq":10084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.528950+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:18.528950+0000","last_peered":"2024-09-15T21:24:18.528950+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:18.528950+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9966,"log_dups_size":0,"ondisk_log_size":9966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40800739,"num_objects":9962,"num_object_clones":0,"num_object_copies":19924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7825,"num_write_kb":31300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5872}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9968","reported_seq":18426,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.512470+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:18.512470+0000","last_peered":"2024-09-15T21:24:18.512470+0000","last_clean":"2024-09-15T21:24:18.512470+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:18.512470+0000","last_undegraded":"2024-09-15T21:24:18.512470+0000","last_fullsized":"2024-09-15T21:24:18.512470+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9968,"log_dups_size":0,"ondisk_log_size":9968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40804352,"num_objects":9962,"num_object_clones":0,"num_object_copies":19924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7661,"num_write_kb":30644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9917","reported_seq":10050,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.121649+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:16.121649+0000","last_peered":"2024-09-15T21:24:16.121649+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:16.121649+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":9917,"log_dups_size":0,"ondisk_log_size":9917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40611840,"num_objects":9915,"num_object_clones":0,"num_object_copies":19830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6861,"num_write_kb":27444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5821},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9861","reported_seq":9993,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.125576+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:16.125576+0000","last_peered":"2024-09-15T21:24:16.125576+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:16.125576+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9861,"log_dups_size":0,"ondisk_log_size":9861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40357888,"num_objects":9853,"num_object_clones":0,"num_object_copies":19706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6770,"num_write_kb":27080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5714},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9758","reported_seq":11262,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:16.252414+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:16.252414+0000","last_peered":"2024-09-15T21:24:16.252414+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:16.252414+0000","last_undegraded":"2024-09-15T21:24:16.252414+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9758,"log_dups_size":0,"ondisk_log_size":9758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39952384,"num_objects":9754,"num_object_clones":0,"num_object_copies":19508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3398,"num_objects_unfound":0,"num_objects_dirty":9754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6242,"num_write_kb":24968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":684,"num_bytes_recovered":2801664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6356},{"shards":"0,2","objects":3398}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10013","reported_seq":10150,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:16.233273+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:16.233273+0000","last_peered":"2024-09-15T21:24:16.233273+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:16.233273+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40965062,"num_objects":10003,"num_object_clones":0,"num_object_copies":20006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10003,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6717,"num_write_kb":26868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5856},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":651199198,"num_objects":158993,"num_object_clones":0,"num_object_copies":317986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3398,"num_objects_unfound":0,"num_objects_dirty":158993,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":118858,"num_write_kb":475432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4849,"num_bytes_recovered":19853312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1323192320,"data_stored":1323044187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158765,"ondisk_log_size":158765,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713012,"kb_used_data":450852,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93658828,"statfs":{"total":96636764160,"available":95906639872,"internally_reserved":0,"allocated":461672448,"data_stored":461400873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":870560,"kb_used_data":541840,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93501280,"statfs":{"total":96636764160,"available":95745310720,"internally_reserved":0,"allocated":554844160,"data_stored":554568508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":336588459},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543416,"kb_used_data":299304,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93828424,"statfs":{"total":96636764160,"available":96080306176,"internally_reserved":0,"allocated":306487296,"data_stored":306171901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":249949871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,5,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554790912,"data_stored":554754782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306896896,"data_stored":306821023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":461504512,"data_stored":461468382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:20.771 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 229 20 160563 160543 2.73803 1.69141 0.0259965 0.0285262 2024-09-15T21:24:21.771 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 230 20 161016 160996 2.73382 1.76953 0.0646696 0.0285696 2024-09-15T21:24:22.771 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 231 20 161468 161448 2.72963 1.76562 0.030224 0.0286133 2024-09-15T21:24:23.177 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:23.747 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:23.748 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:23.771 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-15T21:24:22.756420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656921329,"num_objects":160392,"num_object_clones":0,"num_object_copies":320784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3320,"num_objects_unfound":0,"num_objects_dirty":160392,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":120257,"num_write_kb":481022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4928,"num_bytes_recovered":20172819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159765,"ondisk_log_size":159765,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2136248,"kb_used_data":1305416,"kb_used_omap":12,"kb_used_meta":830771,"kb_avail":280979272,"statfs":{"total":289910292480,"available":287722774528,"internally_reserved":0,"allocated":1336745984,"data_stored":1335883362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":850709507},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,6,10,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23846912,"num_objects":5822,"num_object_clones":0,"num_object_copies":11644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-243,"num_objects_unfound":0,"num_objects_dirty":5822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5822,"num_write_kb":23288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003571"},"pg_stats":[{"pgid":"2.f","version":"45'10302","reported_seq":10375,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.215463+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:21.215463+0000","last_peered":"2024-09-15T21:24:21.215463+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:21.215463+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42196992,"num_objects":10302,"num_object_clones":0,"num_object_copies":20604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10302,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6170},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10012","reported_seq":10117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.488321+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:18.488321+0000","last_peered":"2024-09-15T21:24:18.488321+0000","last_clean":"2024-09-15T21:24:18.488321+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:18.488321+0000","last_undegraded":"2024-09-15T21:24:18.488321+0000","last_fullsized":"2024-09-15T21:24:18.488321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40985542,"num_objects":10008,"num_object_clones":0,"num_object_copies":20016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10008,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8293,"num_write_kb":33172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10204","reported_seq":10327,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.497334+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:18.497334+0000","last_peered":"2024-09-15T21:24:18.497334+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:18.497334+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41779200,"num_objects":10200,"num_object_clones":0,"num_object_copies":20400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8304,"num_write_kb":33216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9805","reported_seq":9873,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.549772+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:18.549772+0000","last_peered":"2024-09-15T21:24:18.549772+0000","last_clean":"2024-09-15T21:24:18.549772+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:18.549772+0000","last_undegraded":"2024-09-15T21:24:18.549772+0000","last_fullsized":"2024-09-15T21:24:18.549772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9805,"log_dups_size":0,"ondisk_log_size":9805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40161280,"num_objects":9805,"num_object_clones":0,"num_object_copies":19610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9805,"num_write_kb":39220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9987","reported_seq":10119,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:21.281692+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:21.281692+0000","last_peered":"2024-09-15T21:24:21.281692+0000","last_clean":"2024-09-15T21:24:21.281692+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:21.281692+0000","last_undegraded":"2024-09-15T21:24:21.281692+0000","last_fullsized":"2024-09-15T21:24:21.281692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9987,"log_dups_size":0,"ondisk_log_size":9987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40878563,"num_objects":9981,"num_object_clones":0,"num_object_copies":19962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6442,"num_write_kb":25768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10042","reported_seq":10178,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.233859+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:21.233859+0000","last_peered":"2024-09-15T21:24:21.233859+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:21.233859+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41112035,"num_objects":10038,"num_object_clones":0,"num_object_copies":20076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10038,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6718,"num_write_kb":26872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5962},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10071","reported_seq":10207,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.275004+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:21.275004+0000","last_peered":"2024-09-15T21:24:21.275004+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:21.275004+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41227206,"num_objects":10067,"num_object_clones":0,"num_object_copies":20134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6382,"num_write_kb":25528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5906},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10138","reported_seq":10267,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:21.316291+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:21.316291+0000","last_peered":"2024-09-15T21:24:21.316291+0000","last_clean":"2024-09-15T21:24:21.316291+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:21.316291+0000","last_undegraded":"2024-09-15T21:24:21.316291+0000","last_fullsized":"2024-09-15T21:24:21.316291+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41517056,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6910,"num_write_kb":27640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9922","reported_seq":10048,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.545536+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:18.545536+0000","last_peered":"2024-09-15T21:24:18.545536+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:18.545536+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9922,"log_dups_size":0,"ondisk_log_size":9922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40612323,"num_objects":9916,"num_object_clones":0,"num_object_copies":19832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7854,"num_write_kb":31416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5829}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9944","reported_seq":10075,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.242363+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:21.242363+0000","last_peered":"2024-09-15T21:24:21.242363+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:21.242363+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9944,"log_dups_size":0,"ondisk_log_size":9944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40697856,"num_objects":9936,"num_object_clones":0,"num_object_copies":19872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6617,"num_write_kb":26468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5916},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9966","reported_seq":10084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.528950+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:18.528950+0000","last_peered":"2024-09-15T21:24:18.528950+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:18.528950+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9966,"log_dups_size":0,"ondisk_log_size":9966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40800739,"num_objects":9962,"num_object_clones":0,"num_object_copies":19924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7825,"num_write_kb":31300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5872}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9968","reported_seq":18426,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.512470+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:18.512470+0000","last_peered":"2024-09-15T21:24:18.512470+0000","last_clean":"2024-09-15T21:24:18.512470+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:18.512470+0000","last_undegraded":"2024-09-15T21:24:18.512470+0000","last_fullsized":"2024-09-15T21:24:18.512470+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9968,"log_dups_size":0,"ondisk_log_size":9968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40804352,"num_objects":9962,"num_object_clones":0,"num_object_copies":19924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7661,"num_write_kb":30644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10053","reported_seq":10185,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.276096+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:21.276096+0000","last_peered":"2024-09-15T21:24:21.276096+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:21.276096+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6997,"num_write_kb":27988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5957},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9998","reported_seq":10130,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.193987+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:21.193987+0000","last_peered":"2024-09-15T21:24:21.193987+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:21.193987+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9998,"log_dups_size":0,"ondisk_log_size":9998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40919040,"num_objects":9990,"num_object_clones":0,"num_object_copies":19980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6907,"num_write_kb":27628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5851},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9905","reported_seq":11565,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:21.280075+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:21.280075+0000","last_peered":"2024-09-15T21:24:21.280075+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:21.280075+0000","last_undegraded":"2024-09-15T21:24:21.280075+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9905,"log_dups_size":0,"ondisk_log_size":9905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40554496,"num_objects":9901,"num_object_clones":0,"num_object_copies":19802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3320,"num_objects_unfound":0,"num_objects_dirty":9901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6389,"num_write_kb":25556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":762,"num_bytes_recovered":3121152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6581},{"shards":"0,2","objects":3320}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10145","reported_seq":10283,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.186969+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:21.186969+0000","last_peered":"2024-09-15T21:24:21.186969+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:21.186969+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41505734,"num_objects":10135,"num_object_clones":0,"num_object_copies":20270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10135,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6849,"num_write_kb":27396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5988},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":656921310,"num_objects":160390,"num_object_clones":0,"num_object_copies":320780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3320,"num_objects_unfound":0,"num_objects_dirty":160390,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":120255,"num_write_kb":481020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4927,"num_bytes_recovered":20172800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1336938496,"data_stored":1336790363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159762,"ondisk_log_size":159762,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713012,"kb_used_data":450852,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93658828,"statfs":{"total":96636764160,"available":95906639872,"internally_reserved":0,"allocated":461672448,"data_stored":461400873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864656,"kb_used_data":548416,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93507184,"statfs":{"total":96636764160,"available":95751356416,"internally_reserved":0,"allocated":561577984,"data_stored":561302332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323808939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558580,"kb_used_data":306148,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93813260,"statfs":{"total":96636764160,"available":96064778240,"internally_reserved":0,"allocated":313495552,"data_stored":313180157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":258469551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561496064,"data_stored":561459934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313937920,"data_stored":313862047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":461504512,"data_stored":461468382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:23.772 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:23.772 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 232 20 161897 161877 2.72508 1.67578 0.0251341 0.0286621 2024-09-15T21:24:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:24.351 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:24.380 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-15T21:24:22.756420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656921329,"num_objects":160392,"num_object_clones":0,"num_object_copies":320784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3320,"num_objects_unfound":0,"num_objects_dirty":160392,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":120257,"num_write_kb":481022,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4928,"num_bytes_recovered":20172819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159765,"ondisk_log_size":159765,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2136248,"kb_used_data":1305416,"kb_used_omap":12,"kb_used_meta":830771,"kb_avail":280979272,"statfs":{"total":289910292480,"available":287722774528,"internally_reserved":0,"allocated":1336745984,"data_stored":1335883362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":850709507},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,6,10,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23846912,"num_objects":5822,"num_object_clones":0,"num_object_copies":11644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-243,"num_objects_unfound":0,"num_objects_dirty":5822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5822,"num_write_kb":23288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":995328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003571"},"pg_stats":[{"pgid":"2.f","version":"45'10302","reported_seq":10375,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.215463+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:21.215463+0000","last_peered":"2024-09-15T21:24:21.215463+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:21.215463+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42196992,"num_objects":10302,"num_object_clones":0,"num_object_copies":20604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10302,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6170},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10012","reported_seq":10117,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.488321+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:18.488321+0000","last_peered":"2024-09-15T21:24:18.488321+0000","last_clean":"2024-09-15T21:24:18.488321+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:18.488321+0000","last_undegraded":"2024-09-15T21:24:18.488321+0000","last_fullsized":"2024-09-15T21:24:18.488321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40985542,"num_objects":10008,"num_object_clones":0,"num_object_copies":20016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10008,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8293,"num_write_kb":33172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10204","reported_seq":10327,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.497334+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:18.497334+0000","last_peered":"2024-09-15T21:24:18.497334+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:18.497334+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41779200,"num_objects":10200,"num_object_clones":0,"num_object_copies":20400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8304,"num_write_kb":33216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":5933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9805","reported_seq":9873,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.549772+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:18.549772+0000","last_peered":"2024-09-15T21:24:18.549772+0000","last_clean":"2024-09-15T21:24:18.549772+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:18.549772+0000","last_undegraded":"2024-09-15T21:24:18.549772+0000","last_fullsized":"2024-09-15T21:24:18.549772+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9805,"log_dups_size":0,"ondisk_log_size":9805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40161280,"num_objects":9805,"num_object_clones":0,"num_object_copies":19610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9805,"num_write_kb":39220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9987","reported_seq":10119,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:21.281692+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:21.281692+0000","last_peered":"2024-09-15T21:24:21.281692+0000","last_clean":"2024-09-15T21:24:21.281692+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:21.281692+0000","last_undegraded":"2024-09-15T21:24:21.281692+0000","last_fullsized":"2024-09-15T21:24:21.281692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9987,"log_dups_size":0,"ondisk_log_size":9987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40878563,"num_objects":9981,"num_object_clones":0,"num_object_copies":19962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6442,"num_write_kb":25768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10042","reported_seq":10178,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.233859+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:21.233859+0000","last_peered":"2024-09-15T21:24:21.233859+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:21.233859+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41112035,"num_objects":10038,"num_object_clones":0,"num_object_copies":20076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10038,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6718,"num_write_kb":26872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5962},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10071","reported_seq":10207,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.275004+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:21.275004+0000","last_peered":"2024-09-15T21:24:21.275004+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:21.275004+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41227206,"num_objects":10067,"num_object_clones":0,"num_object_copies":20134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6382,"num_write_kb":25528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5906},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10138","reported_seq":10267,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:21.316291+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:21.316291+0000","last_peered":"2024-09-15T21:24:21.316291+0000","last_clean":"2024-09-15T21:24:21.316291+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:21.316291+0000","last_undegraded":"2024-09-15T21:24:21.316291+0000","last_fullsized":"2024-09-15T21:24:21.316291+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41517056,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6910,"num_write_kb":27640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9922","reported_seq":10048,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.545536+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:18.545536+0000","last_peered":"2024-09-15T21:24:18.545536+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:18.545536+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":9922,"log_dups_size":0,"ondisk_log_size":9922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40612323,"num_objects":9916,"num_object_clones":0,"num_object_copies":19832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9916,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7854,"num_write_kb":31416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5829}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9944","reported_seq":10075,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.242363+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:21.242363+0000","last_peered":"2024-09-15T21:24:21.242363+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:21.242363+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9944,"log_dups_size":0,"ondisk_log_size":9944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40697856,"num_objects":9936,"num_object_clones":0,"num_object_copies":19872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6617,"num_write_kb":26468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5916},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9966","reported_seq":10084,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:18.528950+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:18.528950+0000","last_peered":"2024-09-15T21:24:18.528950+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:18.528950+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":9966,"log_dups_size":0,"ondisk_log_size":9966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40800739,"num_objects":9962,"num_object_clones":0,"num_object_copies":19924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9962,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7825,"num_write_kb":31300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":5872}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9968","reported_seq":18426,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:18.512470+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:18.512470+0000","last_peered":"2024-09-15T21:24:18.512470+0000","last_clean":"2024-09-15T21:24:18.512470+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:18.512470+0000","last_undegraded":"2024-09-15T21:24:18.512470+0000","last_fullsized":"2024-09-15T21:24:18.512470+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":9968,"log_dups_size":0,"ondisk_log_size":9968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40804352,"num_objects":9962,"num_object_clones":0,"num_object_copies":19924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7661,"num_write_kb":30644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10053","reported_seq":10185,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.276096+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:21.276096+0000","last_peered":"2024-09-15T21:24:21.276096+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:21.276096+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6997,"num_write_kb":27988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5957},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9998","reported_seq":10130,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.193987+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:21.193987+0000","last_peered":"2024-09-15T21:24:21.193987+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:21.193987+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9998,"log_dups_size":0,"ondisk_log_size":9998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40919040,"num_objects":9990,"num_object_clones":0,"num_object_copies":19980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6907,"num_write_kb":27628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5851},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9905","reported_seq":11565,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:21.280075+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:21.280075+0000","last_peered":"2024-09-15T21:24:21.280075+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:21.280075+0000","last_undegraded":"2024-09-15T21:24:21.280075+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9905,"log_dups_size":0,"ondisk_log_size":9905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40554496,"num_objects":9901,"num_object_clones":0,"num_object_copies":19802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3320,"num_objects_unfound":0,"num_objects_dirty":9901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6389,"num_write_kb":25556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":762,"num_bytes_recovered":3121152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6581},{"shards":"0,2","objects":3320}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10145","reported_seq":10283,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.186969+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:21.186969+0000","last_peered":"2024-09-15T21:24:21.186969+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:21.186969+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41505734,"num_objects":10135,"num_object_clones":0,"num_object_copies":20270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10135,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6849,"num_write_kb":27396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5988},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":656921310,"num_objects":160390,"num_object_clones":0,"num_object_copies":320780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3320,"num_objects_unfound":0,"num_objects_dirty":160390,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":120255,"num_write_kb":481020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4927,"num_bytes_recovered":20172800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1336938496,"data_stored":1336790363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159762,"ondisk_log_size":159762,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713012,"kb_used_data":450852,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93658828,"statfs":{"total":96636764160,"available":95906639872,"internally_reserved":0,"allocated":461672448,"data_stored":461400873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864656,"kb_used_data":548416,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93507184,"statfs":{"total":96636764160,"available":95751356416,"internally_reserved":0,"allocated":561577984,"data_stored":561302332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323808939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558580,"kb_used_data":306148,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93813260,"statfs":{"total":96636764160,"available":96064778240,"internally_reserved":0,"allocated":313495552,"data_stored":313180157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":258469551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561496064,"data_stored":561459934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313937920,"data_stored":313862047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":461504512,"data_stored":461468382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:24.381 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:24.381 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10302", 'reported_seq': 10375, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:21.215463+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:24:21.215463+0000', 'last_peered': '2024-09-15T21:24:21.215463+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:24:21.215463+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42196992, 'num_objects': 10302, 'num_object_clones': 0, 'num_object_copies': 20604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10302, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10302, 'num_write_kb': 41208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6170}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:24.382 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:24:24.382 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'10204", 'reported_seq': 10327, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:18.497334+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:24:18.497334+0000', 'last_peered': '2024-09-15T21:24:18.497334+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:24:18.497334+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41779200, 'num_objects': 10200, 'num_object_clones': 0, 'num_object_copies': 20400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8304, 'num_write_kb': 33216, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 5933}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:24.382 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:24.382 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'10042", 'reported_seq': 10178, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:21.233859+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:24:21.233859+0000', 'last_peered': '2024-09-15T21:24:21.233859+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:24:21.233859+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41112035, 'num_objects': 10038, 'num_object_clones': 0, 'num_object_copies': 20076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10038, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6718, 'num_write_kb': 26872, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5962}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:24.382 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:24.382 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10071", 'reported_seq': 10207, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:21.275004+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:24:21.275004+0000', 'last_peered': '2024-09-15T21:24:21.275004+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:24:21.275004+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41227206, 'num_objects': 10067, 'num_object_clones': 0, 'num_object_copies': 20134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10067, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6382, 'num_write_kb': 25528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5906}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:24.382 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:24:24.382 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'9922", 'reported_seq': 10048, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:18.545536+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:24:18.545536+0000', 'last_peered': '2024-09-15T21:24:18.545536+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:24:18.545536+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 9922, 'log_dups_size': 0, 'ondisk_log_size': 9922, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40612323, 'num_objects': 9916, 'num_object_clones': 0, 'num_object_copies': 19832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9916, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7854, 'num_write_kb': 31416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 5829}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:24.382 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:24.382 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'9944", 'reported_seq': 10075, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:21.242363+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:24:21.242363+0000', 'last_peered': '2024-09-15T21:24:21.242363+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:24:21.242363+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 9944, 'log_dups_size': 0, 'ondisk_log_size': 9944, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40697856, 'num_objects': 9936, 'num_object_clones': 0, 'num_object_copies': 19872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9936, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6617, 'num_write_kb': 26468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5916}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:24.383 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:24.383 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'9966", 'reported_seq': 10084, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:18.528950+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:24:18.528950+0000', 'last_peered': '2024-09-15T21:24:18.528950+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:24:18.528950+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 9966, 'log_dups_size': 0, 'ondisk_log_size': 9966, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40800739, 'num_objects': 9962, 'num_object_clones': 0, 'num_object_copies': 19924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9962, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7825, 'num_write_kb': 31300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 5872}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:24.383 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:24.383 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'10053", 'reported_seq': 10185, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:21.276096+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:24:21.276096+0000', 'last_peered': '2024-09-15T21:24:21.276096+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:24:21.276096+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41168896, 'num_objects': 10051, 'num_object_clones': 0, 'num_object_copies': 20102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10051, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6997, 'num_write_kb': 27988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5957}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:24.383 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:24.383 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'9998", 'reported_seq': 10130, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:21.193987+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:24:21.193987+0000', 'last_peered': '2024-09-15T21:24:21.193987+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:24:21.193987+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 9998, 'log_dups_size': 0, 'ondisk_log_size': 9998, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40919040, 'num_objects': 9990, 'num_object_clones': 0, 'num_object_copies': 19980, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9990, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6907, 'num_write_kb': 27628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5851}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:24.383 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:24.383 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'9905", 'reported_seq': 11565, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:24:21.280075+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:24:21.280075+0000', 'last_peered': '2024-09-15T21:24:21.280075+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:24:21.280075+0000', 'last_undegraded': '2024-09-15T21:24:21.280075+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 9905, 'log_dups_size': 0, 'ondisk_log_size': 9905, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40554496, 'num_objects': 9901, 'num_object_clones': 0, 'num_object_copies': 19802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3320, 'num_objects_unfound': 0, 'num_objects_dirty': 9901, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6389, 'num_write_kb': 25556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 762, 'num_bytes_recovered': 3121152, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6581}, {'shards': '0,2', 'objects': 3320}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:24.383 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:24:24.383 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10145", 'reported_seq': 10283, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:21.186969+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:24:21.186969+0000', 'last_peered': '2024-09-15T21:24:21.186969+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:24:21.186969+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41505734, 'num_objects': 10135, 'num_object_clones': 0, 'num_object_copies': 20270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10135, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6849, 'num_write_kb': 27396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5988}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:24.384 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:24.771 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 233 20 162299 162279 2.72013 1.57031 0.0269717 0.028714 2024-09-15T21:24:25.072 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:25.072 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/320784 objects degraded (12.851%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":316,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":160392,"data_bytes":656921329,"bytes_used":2187517952,"bytes_avail":287722774528,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":320784,"degraded_ratio":0.12851326749463815,"misplaced_objects":3320,"misplaced_total":320784,"misplaced_ratio":0.010349643373734351,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":82919,"recovering_keys_per_sec":0,"num_objects_recovered":243,"num_bytes_recovered":995328,"num_keys_recovered":0,"write_bytes_sec":1986651,"read_op_per_sec":0,"write_op_per_sec":485},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:25.103 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:25.104 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:25.672 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:25.672 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:25.704 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-15T21:24:24.756945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660103921,"num_objects":161169,"num_object_clones":0,"num_object_copies":322338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3320,"num_objects_unfound":0,"num_objects_dirty":161169,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":121034,"num_write_kb":484130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4928,"num_bytes_recovered":20172819,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160142,"ondisk_log_size":160142,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2144676,"kb_used_data":1309684,"kb_used_omap":12,"kb_used_meta":834931,"kb_avail":280970844,"statfs":{"total":289910292480,"available":287714144256,"internally_reserved":0,"allocated":1341116416,"data_stored":1340253794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":854969347},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,5,5,10,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21360640,"num_objects":5215,"num_object_clones":0,"num_object_copies":10430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-162,"num_objects_unfound":0,"num_objects_dirty":5215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5215,"num_write_kb":20860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":663552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003498"},"pg_stats":[{"pgid":"2.f","version":"45'10302","reported_seq":10375,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.215463+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:21.215463+0000","last_peered":"2024-09-15T21:24:21.215463+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:21.215463+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42196992,"num_objects":10302,"num_object_clones":0,"num_object_copies":20604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10302,"num_write_kb":41208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6170},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10129","reported_seq":10234,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:23.415785+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:23.415785+0000","last_peered":"2024-09-15T21:24:23.415785+0000","last_clean":"2024-09-15T21:24:23.415785+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:23.415785+0000","last_undegraded":"2024-09-15T21:24:23.415785+0000","last_fullsized":"2024-09-15T21:24:23.415785+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41464774,"num_objects":10125,"num_object_clones":0,"num_object_copies":20250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10125,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8410,"num_write_kb":33640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10338","reported_seq":10463,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.534426+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:23.534426+0000","last_peered":"2024-09-15T21:24:23.534426+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:23.534426+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42328064,"num_objects":10334,"num_object_clones":0,"num_object_copies":20668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8438,"num_write_kb":33752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":6067}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9923","reported_seq":9993,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:23.550455+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:23.550455+0000","last_peered":"2024-09-15T21:24:23.550455+0000","last_clean":"2024-09-15T21:24:23.550455+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:23.550455+0000","last_undegraded":"2024-09-15T21:24:23.550455+0000","last_fullsized":"2024-09-15T21:24:23.550455+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":9923,"log_dups_size":0,"ondisk_log_size":9923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40644608,"num_objects":9923,"num_object_clones":0,"num_object_copies":19846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9923,"num_write_kb":39692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9987","reported_seq":10119,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:21.281692+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:21.281692+0000","last_peered":"2024-09-15T21:24:21.281692+0000","last_clean":"2024-09-15T21:24:21.281692+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:21.281692+0000","last_undegraded":"2024-09-15T21:24:21.281692+0000","last_fullsized":"2024-09-15T21:24:21.281692+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":9987,"log_dups_size":0,"ondisk_log_size":9987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40878563,"num_objects":9981,"num_object_clones":0,"num_object_copies":19962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6442,"num_write_kb":25768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10042","reported_seq":10178,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.233859+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:21.233859+0000","last_peered":"2024-09-15T21:24:21.233859+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:21.233859+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41112035,"num_objects":10038,"num_object_clones":0,"num_object_copies":20076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10038,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6718,"num_write_kb":26872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5962},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10071","reported_seq":10207,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.275004+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:21.275004+0000","last_peered":"2024-09-15T21:24:21.275004+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:21.275004+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41227206,"num_objects":10067,"num_object_clones":0,"num_object_copies":20134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6382,"num_write_kb":25528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5906},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10138","reported_seq":10267,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:21.316291+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:21.316291+0000","last_peered":"2024-09-15T21:24:21.316291+0000","last_clean":"2024-09-15T21:24:21.316291+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:21.316291+0000","last_undegraded":"2024-09-15T21:24:21.316291+0000","last_fullsized":"2024-09-15T21:24:21.316291+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41517056,"num_objects":10136,"num_object_clones":0,"num_object_copies":20272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6910,"num_write_kb":27640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10057","reported_seq":10182,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.587402+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:23.587402+0000","last_peered":"2024-09-15T21:24:23.587402+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:23.587402+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41165283,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7989,"num_write_kb":31956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":5964}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888056+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888056+0000","last_peered":"2024-09-15T21:23:35.888056+0000","last_clean":"2024-09-15T21:23:35.888056+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888056+0000","last_undegraded":"2024-09-15T21:23:35.888056+0000","last_fullsized":"2024-09-15T21:23:35.888056+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9944","reported_seq":10075,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.242363+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:21.242363+0000","last_peered":"2024-09-15T21:24:21.242363+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:21.242363+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":9944,"log_dups_size":0,"ondisk_log_size":9944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40697856,"num_objects":9936,"num_object_clones":0,"num_object_copies":19872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6617,"num_write_kb":26468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5916},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10106","reported_seq":10224,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:23.514998+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:23.514998+0000","last_peered":"2024-09-15T21:24:23.514998+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:23.514998+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41374179,"num_objects":10102,"num_object_clones":0,"num_object_copies":20204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7965,"num_write_kb":31860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":6012}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10101","reported_seq":18559,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:23.509750+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:23.509750+0000","last_peered":"2024-09-15T21:24:23.509750+0000","last_clean":"2024-09-15T21:24:23.509750+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:23.509750+0000","last_undegraded":"2024-09-15T21:24:23.509750+0000","last_fullsized":"2024-09-15T21:24:23.509750+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41349120,"num_objects":10095,"num_object_clones":0,"num_object_copies":20190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7794,"num_write_kb":31176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10053","reported_seq":10185,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.276096+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:21.276096+0000","last_peered":"2024-09-15T21:24:21.276096+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:21.276096+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41168896,"num_objects":10051,"num_object_clones":0,"num_object_copies":20102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10051,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6997,"num_write_kb":27988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5957},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9998","reported_seq":10130,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.193987+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:21.193987+0000","last_peered":"2024-09-15T21:24:21.193987+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:21.193987+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":9998,"log_dups_size":0,"ondisk_log_size":9998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40919040,"num_objects":9990,"num_object_clones":0,"num_object_copies":19980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9990,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6907,"num_write_kb":27628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5851},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891281+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891281+0000","last_peered":"2024-09-15T21:23:35.891281+0000","last_clean":"2024-09-15T21:23:35.891281+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891281+0000","last_undegraded":"2024-09-15T21:23:35.891281+0000","last_fullsized":"2024-09-15T21:23:35.891281+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9905","reported_seq":11565,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:21.280075+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:21.280075+0000","last_peered":"2024-09-15T21:24:21.280075+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:21.280075+0000","last_undegraded":"2024-09-15T21:24:21.280075+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":9905,"log_dups_size":0,"ondisk_log_size":9905,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40554496,"num_objects":9901,"num_object_clones":0,"num_object_copies":19802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3320,"num_objects_unfound":0,"num_objects_dirty":9901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6389,"num_write_kb":25556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":762,"num_bytes_recovered":3121152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6581},{"shards":"0,2","objects":3320}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10145","reported_seq":10283,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:21.186969+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:21.186969+0000","last_peered":"2024-09-15T21:24:21.186969+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:21.186969+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41505734,"num_objects":10135,"num_object_clones":0,"num_object_copies":20270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10135,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6849,"num_write_kb":27396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5988},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":660103902,"num_objects":161167,"num_object_clones":0,"num_object_copies":322334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3320,"num_objects_unfound":0,"num_objects_dirty":161167,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":121032,"num_write_kb":484128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4927,"num_bytes_recovered":20172800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1341296640,"data_stored":1341148507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160139,"ondisk_log_size":160139,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":721440,"kb_used_data":455120,"kb_used_omap":4,"kb_used_meta":266299,"kb_avail":93650400,"statfs":{"total":96636764160,"available":95898009600,"internally_reserved":0,"allocated":466042880,"data_stored":465771305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":272690857},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864656,"kb_used_data":548416,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93507184,"statfs":{"total":96636764160,"available":95751356416,"internally_reserved":0,"allocated":561577984,"data_stored":561302332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":323808939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558580,"kb_used_data":306148,"kb_used_omap":4,"kb_used_meta":252411,"kb_avail":93813260,"statfs":{"total":96636764160,"available":96064778240,"internally_reserved":0,"allocated":313495552,"data_stored":313180157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":258469551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561496064,"data_stored":561459934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313937920,"data_stored":313862047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":465862656,"data_stored":465826526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:25.771 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 234 20 162701 162681 2.71521 1.57031 0.0818012 0.0287661 2024-09-15T21:24:26.771 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 235 20 163161 163141 2.71131 1.79688 0.03045 0.0288074 2024-09-15T21:24:27.771 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 236 20 163624 163604 2.70748 1.80859 0.103056 0.0288463 2024-09-15T21:24:28.707 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:28.771 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 237 20 164069 164049 2.70339 1.73828 0.0116745 0.0288891 2024-09-15T21:24:29.278 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:29.280 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:29.301 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-15T21:24:28.757928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669029105,"num_objects":163348,"num_object_clones":0,"num_object_copies":326696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3236,"num_objects_unfound":0,"num_objects_dirty":163348,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":123213,"num_write_kb":492846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5012,"num_bytes_recovered":20516883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160921,"ondisk_log_size":160921,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2192644,"kb_used_data":1327124,"kb_used_omap":12,"kb_used_meta":865459,"kb_avail":280922876,"statfs":{"total":289910292480,"available":287665025024,"internally_reserved":0,"allocated":1358974976,"data_stored":1358112354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":886230019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,2,2,7,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21209088,"num_objects":5178,"num_object_clones":0,"num_object_copies":10356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-162,"num_objects_unfound":0,"num_objects_dirty":5178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5178,"num_write_kb":20712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":663552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003517"},"pg_stats":[{"pgid":"2.f","version":"45'10437","reported_seq":10512,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.329655+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:26.329655+0000","last_peered":"2024-09-15T21:24:26.329655+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:26.329655+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42749952,"num_objects":10437,"num_object_clones":0,"num_object_copies":20874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10437,"num_write_kb":41748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6305},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10251","reported_seq":10356,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.606614+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:28.606614+0000","last_peered":"2024-09-15T21:24:28.606614+0000","last_clean":"2024-09-15T21:24:28.606614+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:28.606614+0000","last_undegraded":"2024-09-15T21:24:28.606614+0000","last_fullsized":"2024-09-15T21:24:28.606614+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41964486,"num_objects":10247,"num_object_clones":0,"num_object_copies":20494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10247,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8532,"num_write_kb":34128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10480","reported_seq":10605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.578121+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:28.578121+0000","last_peered":"2024-09-15T21:24:28.578121+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:28.578121+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42909696,"num_objects":10476,"num_object_clones":0,"num_object_copies":20952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8580,"num_write_kb":34320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":6209}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10055","reported_seq":10125,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.543098+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:28.543098+0000","last_peered":"2024-09-15T21:24:28.543098+0000","last_clean":"2024-09-15T21:24:28.543098+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:28.543098+0000","last_undegraded":"2024-09-15T21:24:28.543098+0000","last_fullsized":"2024-09-15T21:24:28.543098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41185280,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10055,"num_write_kb":40220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10122","reported_seq":10255,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:26.366681+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:26.366681+0000","last_peered":"2024-09-15T21:24:26.366681+0000","last_clean":"2024-09-15T21:24:26.366681+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:26.366681+0000","last_undegraded":"2024-09-15T21:24:26.366681+0000","last_fullsized":"2024-09-15T21:24:26.366681+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41431523,"num_objects":10116,"num_object_clones":0,"num_object_copies":20232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6577,"num_write_kb":26308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10194","reported_seq":10328,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.323636+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:26.323636+0000","last_peered":"2024-09-15T21:24:26.323636+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:26.323636+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41734627,"num_objects":10190,"num_object_clones":0,"num_object_copies":20380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6870,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6114},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10202","reported_seq":10336,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.370353+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:26.370353+0000","last_peered":"2024-09-15T21:24:26.370353+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:26.370353+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41763782,"num_objects":10198,"num_object_clones":0,"num_object_copies":20396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6513,"num_write_kb":26052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6037},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10286","reported_seq":10415,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:26.248628+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:26.248628+0000","last_peered":"2024-09-15T21:24:26.248628+0000","last_clean":"2024-09-15T21:24:26.248628+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:26.248628+0000","last_undegraded":"2024-09-15T21:24:26.248628+0000","last_fullsized":"2024-09-15T21:24:26.248628+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42123264,"num_objects":10284,"num_object_clones":0,"num_object_copies":20568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":28232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10183","reported_seq":10309,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.582250+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:28.582250+0000","last_peered":"2024-09-15T21:24:28.582250+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:28.582250+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41681379,"num_objects":10177,"num_object_clones":0,"num_object_copies":20354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8115,"num_write_kb":32460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":6090}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10093","reported_seq":10223,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.322723+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:26.322723+0000","last_peered":"2024-09-15T21:24:26.322723+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:26.322723+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41308160,"num_objects":10085,"num_object_clones":0,"num_object_copies":20170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6766,"num_write_kb":27064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6065},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10247","reported_seq":10366,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.599987+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:28.599987+0000","last_peered":"2024-09-15T21:24:28.599987+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:28.599987+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41951715,"num_objects":10243,"num_object_clones":0,"num_object_copies":20486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8106,"num_write_kb":32424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":6153}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10214","reported_seq":18672,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.600232+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:28.600232+0000","last_peered":"2024-09-15T21:24:28.600232+0000","last_clean":"2024-09-15T21:24:28.600232+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:28.600232+0000","last_undegraded":"2024-09-15T21:24:28.600232+0000","last_fullsized":"2024-09-15T21:24:28.600232+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41811968,"num_objects":10208,"num_object_clones":0,"num_object_copies":20416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7907,"num_write_kb":31628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10195","reported_seq":10327,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.321104+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:26.321104+0000","last_peered":"2024-09-15T21:24:26.321104+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:26.321104+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41750528,"num_objects":10193,"num_object_clones":0,"num_object_copies":20386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7139,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6099},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10142","reported_seq":10274,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.356027+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:26.356027+0000","last_peered":"2024-09-15T21:24:26.356027+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:26.356027+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41508864,"num_objects":10134,"num_object_clones":0,"num_object_copies":20268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7051,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5995},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10029","reported_seq":11858,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:26.376878+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:26.376878+0000","last_peered":"2024-09-15T21:24:26.376878+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:26.376878+0000","last_undegraded":"2024-09-15T21:24:26.376878+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41062400,"num_objects":10025,"num_object_clones":0,"num_object_copies":20050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3236,"num_objects_unfound":0,"num_objects_dirty":10025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6513,"num_write_kb":26052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":846,"num_bytes_recovered":3465216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6789},{"shards":"0,2","objects":3236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10288","reported_seq":10425,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.364993+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:26.364993+0000","last_peered":"2024-09-15T21:24:26.364993+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:26.364993+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42091462,"num_objects":10278,"num_object_clones":0,"num_object_copies":20556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6992,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6131},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":669029086,"num_objects":163346,"num_object_clones":0,"num_object_copies":326692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3236,"num_objects_unfound":0,"num_objects_dirty":163346,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":123211,"num_write_kb":492844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5011,"num_bytes_recovered":20516864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1359360000,"data_stored":1359211867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729796,"kb_used_data":459316,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93642044,"statfs":{"total":96636764160,"available":95889453056,"internally_reserved":0,"allocated":470339584,"data_stored":470068009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":276950697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":893332,"kb_used_data":554884,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93478508,"statfs":{"total":96636764160,"available":95721992192,"internally_reserved":0,"allocated":568201216,"data_stored":567925564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,2,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569516,"kb_used_data":312924,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93802324,"statfs":{"total":96636764160,"available":96053579776,"internally_reserved":0,"allocated":320434176,"data_stored":320118781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":262729391},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568254464,"data_stored":568218334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320856064,"data_stored":320780191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":470249472,"data_stored":470213342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:29.302 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:29.771 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 238 20 164476 164456 2.69871 1.58984 0.0578104 0.0289424 2024-09-15T21:24:29.853 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:29.854 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:29.878 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-15T21:24:28.757928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669029105,"num_objects":163348,"num_object_clones":0,"num_object_copies":326696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3236,"num_objects_unfound":0,"num_objects_dirty":163348,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":123213,"num_write_kb":492846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5012,"num_bytes_recovered":20516883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160921,"ondisk_log_size":160921,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2192644,"kb_used_data":1327124,"kb_used_omap":12,"kb_used_meta":865459,"kb_avail":280922876,"statfs":{"total":289910292480,"available":287665025024,"internally_reserved":0,"allocated":1358974976,"data_stored":1358112354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":886230019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,2,2,7,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21209088,"num_objects":5178,"num_object_clones":0,"num_object_copies":10356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-162,"num_objects_unfound":0,"num_objects_dirty":5178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5178,"num_write_kb":20712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":663552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003517"},"pg_stats":[{"pgid":"2.f","version":"45'10437","reported_seq":10512,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.329655+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:26.329655+0000","last_peered":"2024-09-15T21:24:26.329655+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:26.329655+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42749952,"num_objects":10437,"num_object_clones":0,"num_object_copies":20874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10437,"num_write_kb":41748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6305},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10251","reported_seq":10356,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.606614+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:28.606614+0000","last_peered":"2024-09-15T21:24:28.606614+0000","last_clean":"2024-09-15T21:24:28.606614+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:28.606614+0000","last_undegraded":"2024-09-15T21:24:28.606614+0000","last_fullsized":"2024-09-15T21:24:28.606614+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41964486,"num_objects":10247,"num_object_clones":0,"num_object_copies":20494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10247,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8532,"num_write_kb":34128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10480","reported_seq":10605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.578121+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:28.578121+0000","last_peered":"2024-09-15T21:24:28.578121+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:28.578121+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42909696,"num_objects":10476,"num_object_clones":0,"num_object_copies":20952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8580,"num_write_kb":34320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":6209}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10055","reported_seq":10125,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.543098+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:28.543098+0000","last_peered":"2024-09-15T21:24:28.543098+0000","last_clean":"2024-09-15T21:24:28.543098+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:28.543098+0000","last_undegraded":"2024-09-15T21:24:28.543098+0000","last_fullsized":"2024-09-15T21:24:28.543098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41185280,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10055,"num_write_kb":40220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10122","reported_seq":10255,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:26.366681+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:26.366681+0000","last_peered":"2024-09-15T21:24:26.366681+0000","last_clean":"2024-09-15T21:24:26.366681+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:26.366681+0000","last_undegraded":"2024-09-15T21:24:26.366681+0000","last_fullsized":"2024-09-15T21:24:26.366681+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41431523,"num_objects":10116,"num_object_clones":0,"num_object_copies":20232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6577,"num_write_kb":26308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10194","reported_seq":10328,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.323636+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:26.323636+0000","last_peered":"2024-09-15T21:24:26.323636+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:26.323636+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41734627,"num_objects":10190,"num_object_clones":0,"num_object_copies":20380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6870,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6114},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10202","reported_seq":10336,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.370353+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:26.370353+0000","last_peered":"2024-09-15T21:24:26.370353+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:26.370353+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41763782,"num_objects":10198,"num_object_clones":0,"num_object_copies":20396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6513,"num_write_kb":26052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6037},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10286","reported_seq":10415,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:26.248628+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:26.248628+0000","last_peered":"2024-09-15T21:24:26.248628+0000","last_clean":"2024-09-15T21:24:26.248628+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:26.248628+0000","last_undegraded":"2024-09-15T21:24:26.248628+0000","last_fullsized":"2024-09-15T21:24:26.248628+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42123264,"num_objects":10284,"num_object_clones":0,"num_object_copies":20568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":28232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10183","reported_seq":10309,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.582250+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:28.582250+0000","last_peered":"2024-09-15T21:24:28.582250+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:28.582250+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41681379,"num_objects":10177,"num_object_clones":0,"num_object_copies":20354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8115,"num_write_kb":32460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":6090}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10093","reported_seq":10223,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.322723+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:26.322723+0000","last_peered":"2024-09-15T21:24:26.322723+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:26.322723+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41308160,"num_objects":10085,"num_object_clones":0,"num_object_copies":20170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6766,"num_write_kb":27064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6065},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10247","reported_seq":10366,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.599987+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:28.599987+0000","last_peered":"2024-09-15T21:24:28.599987+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:28.599987+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41951715,"num_objects":10243,"num_object_clones":0,"num_object_copies":20486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8106,"num_write_kb":32424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":6153}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10214","reported_seq":18672,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.600232+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:28.600232+0000","last_peered":"2024-09-15T21:24:28.600232+0000","last_clean":"2024-09-15T21:24:28.600232+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:28.600232+0000","last_undegraded":"2024-09-15T21:24:28.600232+0000","last_fullsized":"2024-09-15T21:24:28.600232+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41811968,"num_objects":10208,"num_object_clones":0,"num_object_copies":20416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7907,"num_write_kb":31628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10195","reported_seq":10327,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.321104+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:26.321104+0000","last_peered":"2024-09-15T21:24:26.321104+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:26.321104+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41750528,"num_objects":10193,"num_object_clones":0,"num_object_copies":20386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7139,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6099},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10142","reported_seq":10274,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.356027+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:26.356027+0000","last_peered":"2024-09-15T21:24:26.356027+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:26.356027+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41508864,"num_objects":10134,"num_object_clones":0,"num_object_copies":20268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7051,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5995},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10029","reported_seq":11858,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:26.376878+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:26.376878+0000","last_peered":"2024-09-15T21:24:26.376878+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:26.376878+0000","last_undegraded":"2024-09-15T21:24:26.376878+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41062400,"num_objects":10025,"num_object_clones":0,"num_object_copies":20050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3236,"num_objects_unfound":0,"num_objects_dirty":10025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6513,"num_write_kb":26052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":846,"num_bytes_recovered":3465216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6789},{"shards":"0,2","objects":3236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10288","reported_seq":10425,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.364993+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:26.364993+0000","last_peered":"2024-09-15T21:24:26.364993+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:26.364993+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42091462,"num_objects":10278,"num_object_clones":0,"num_object_copies":20556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6992,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6131},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":669029086,"num_objects":163346,"num_object_clones":0,"num_object_copies":326692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3236,"num_objects_unfound":0,"num_objects_dirty":163346,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":123211,"num_write_kb":492844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5011,"num_bytes_recovered":20516864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1359360000,"data_stored":1359211867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729796,"kb_used_data":459316,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93642044,"statfs":{"total":96636764160,"available":95889453056,"internally_reserved":0,"allocated":470339584,"data_stored":470068009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":276950697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":893332,"kb_used_data":554884,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93478508,"statfs":{"total":96636764160,"available":95721992192,"internally_reserved":0,"allocated":568201216,"data_stored":567925564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,2,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569516,"kb_used_data":312924,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93802324,"statfs":{"total":96636764160,"available":96053579776,"internally_reserved":0,"allocated":320434176,"data_stored":320118781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":262729391},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568254464,"data_stored":568218334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320856064,"data_stored":320780191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":470249472,"data_stored":470213342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:29.879 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:29.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10437", 'reported_seq': 10512, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:26.329655+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:24:26.329655+0000', 'last_peered': '2024-09-15T21:24:26.329655+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:24:26.329655+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42749952, 'num_objects': 10437, 'num_object_clones': 0, 'num_object_copies': 20874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10437, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10437, 'num_write_kb': 41748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6305}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.879 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:24:29.879 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'10480", 'reported_seq': 10605, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:28.578121+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:24:28.578121+0000', 'last_peered': '2024-09-15T21:24:28.578121+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:24:28.578121+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42909696, 'num_objects': 10476, 'num_object_clones': 0, 'num_object_copies': 20952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10476, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8580, 'num_write_kb': 34320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4267}, {'shards': '2', 'objects': 6209}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:29.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:29.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'10194", 'reported_seq': 10328, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:26.323636+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:24:26.323636+0000', 'last_peered': '2024-09-15T21:24:26.323636+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:24:26.323636+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41734627, 'num_objects': 10190, 'num_object_clones': 0, 'num_object_copies': 20380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10190, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6870, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6114}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:29.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10202", 'reported_seq': 10336, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:26.370353+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:24:26.370353+0000', 'last_peered': '2024-09-15T21:24:26.370353+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:24:26.370353+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41763782, 'num_objects': 10198, 'num_object_clones': 0, 'num_object_copies': 20396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10198, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6513, 'num_write_kb': 26052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6037}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:24:29.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'10183", 'reported_seq': 10309, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:28.582250+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:24:28.582250+0000', 'last_peered': '2024-09-15T21:24:28.582250+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:24:28.582250+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41681379, 'num_objects': 10177, 'num_object_clones': 0, 'num_object_copies': 20354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10177, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8115, 'num_write_kb': 32460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4085}, {'shards': '2', 'objects': 6090}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:29.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:29.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'10093", 'reported_seq': 10223, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:26.322723+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:24:26.322723+0000', 'last_peered': '2024-09-15T21:24:26.322723+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:24:26.322723+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41308160, 'num_objects': 10085, 'num_object_clones': 0, 'num_object_copies': 20170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10085, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6766, 'num_write_kb': 27064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6065}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.880 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:29.880 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'10247", 'reported_seq': 10366, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:28.599987+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:24:28.599987+0000', 'last_peered': '2024-09-15T21:24:28.599987+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:24:28.599987+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41951715, 'num_objects': 10243, 'num_object_clones': 0, 'num_object_copies': 20486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10243, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8106, 'num_write_kb': 32424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4090}, {'shards': '2', 'objects': 6153}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:29.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:29.881 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'10195", 'reported_seq': 10327, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:26.321104+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:24:26.321104+0000', 'last_peered': '2024-09-15T21:24:26.321104+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:24:26.321104+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41750528, 'num_objects': 10193, 'num_object_clones': 0, 'num_object_copies': 20386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10193, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7139, 'num_write_kb': 28556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6099}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:29.881 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'10142", 'reported_seq': 10274, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:26.356027+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:24:26.356027+0000', 'last_peered': '2024-09-15T21:24:26.356027+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:24:26.356027+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41508864, 'num_objects': 10134, 'num_object_clones': 0, 'num_object_copies': 20268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10134, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7051, 'num_write_kb': 28204, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5995}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:29.881 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'10029", 'reported_seq': 11858, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:24:26.376878+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:24:26.376878+0000', 'last_peered': '2024-09-15T21:24:26.376878+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:24:26.376878+0000', 'last_undegraded': '2024-09-15T21:24:26.376878+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41062400, 'num_objects': 10025, 'num_object_clones': 0, 'num_object_copies': 20050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3236, 'num_objects_unfound': 0, 'num_objects_dirty': 10025, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6513, 'num_write_kb': 26052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 846, 'num_bytes_recovered': 3465216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6789}, {'shards': '0,2', 'objects': 3236}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.881 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:24:29.882 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10288", 'reported_seq': 10425, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:26.364993+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:24:26.364993+0000', 'last_peered': '2024-09-15T21:24:26.364993+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:24:26.364993+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42091462, 'num_objects': 10278, 'num_object_clones': 0, 'num_object_copies': 20556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10278, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6992, 'num_write_kb': 27968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6131}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:29.882 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:30.593 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:30.593 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/326696 objects degraded (12.619%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":322,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":163348,"data_bytes":669029105,"bytes_used":2245267456,"bytes_avail":287665025024,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":326696,"degraded_ratio":0.12618764845605701,"misplaced_objects":3236,"misplaced_total":326696,"misplaced_ratio":0.0099052329994857597,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":55279,"recovering_keys_per_sec":0,"num_objects_recovered":162,"num_bytes_recovered":663552,"num_keys_recovered":0,"write_bytes_sec":1766906,"read_op_per_sec":0,"write_op_per_sec":431},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:30.612 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:30.613 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:30.771 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout: 239 20 164874 164854 2.69392 1.55469 0.0794975 0.0289921 2024-09-15T21:24:31.187 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:31.188 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:31.213 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-15T21:24:30.758420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":669029105,"num_objects":163348,"num_object_clones":0,"num_object_copies":326696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3236,"num_objects_unfound":0,"num_objects_dirty":163348,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":123213,"num_write_kb":492846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5012,"num_bytes_recovered":20516883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160921,"ondisk_log_size":160921,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2192644,"kb_used_data":1327124,"kb_used_omap":12,"kb_used_meta":865459,"kb_avail":280922876,"statfs":{"total":289910292480,"available":287665025024,"internally_reserved":0,"allocated":1358974976,"data_stored":1358112354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":886230019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,2,2,7,6,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17829888,"num_objects":4353,"num_object_clones":0,"num_object_copies":8706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-162,"num_objects_unfound":0,"num_objects_dirty":4353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4353,"num_write_kb":17412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":663552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003363"},"pg_stats":[{"pgid":"2.f","version":"45'10437","reported_seq":10512,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.329655+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:26.329655+0000","last_peered":"2024-09-15T21:24:26.329655+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:26.329655+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42749952,"num_objects":10437,"num_object_clones":0,"num_object_copies":20874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10437,"num_write_kb":41748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6305},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10251","reported_seq":10356,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.606614+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:28.606614+0000","last_peered":"2024-09-15T21:24:28.606614+0000","last_clean":"2024-09-15T21:24:28.606614+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:28.606614+0000","last_undegraded":"2024-09-15T21:24:28.606614+0000","last_fullsized":"2024-09-15T21:24:28.606614+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41964486,"num_objects":10247,"num_object_clones":0,"num_object_copies":20494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10247,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8532,"num_write_kb":34128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10480","reported_seq":10605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.578121+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:28.578121+0000","last_peered":"2024-09-15T21:24:28.578121+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:28.578121+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42909696,"num_objects":10476,"num_object_clones":0,"num_object_copies":20952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8580,"num_write_kb":34320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4267},{"shards":"2","objects":6209}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10055","reported_seq":10125,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.543098+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:28.543098+0000","last_peered":"2024-09-15T21:24:28.543098+0000","last_clean":"2024-09-15T21:24:28.543098+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:28.543098+0000","last_undegraded":"2024-09-15T21:24:28.543098+0000","last_fullsized":"2024-09-15T21:24:28.543098+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41185280,"num_objects":10055,"num_object_clones":0,"num_object_copies":20110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10055,"num_write_kb":40220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10122","reported_seq":10255,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:26.366681+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:26.366681+0000","last_peered":"2024-09-15T21:24:26.366681+0000","last_clean":"2024-09-15T21:24:26.366681+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:26.366681+0000","last_undegraded":"2024-09-15T21:24:26.366681+0000","last_fullsized":"2024-09-15T21:24:26.366681+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41431523,"num_objects":10116,"num_object_clones":0,"num_object_copies":20232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6577,"num_write_kb":26308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10194","reported_seq":10328,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.323636+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:26.323636+0000","last_peered":"2024-09-15T21:24:26.323636+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:26.323636+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41734627,"num_objects":10190,"num_object_clones":0,"num_object_copies":20380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6870,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6114},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10202","reported_seq":10336,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.370353+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:26.370353+0000","last_peered":"2024-09-15T21:24:26.370353+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:26.370353+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41763782,"num_objects":10198,"num_object_clones":0,"num_object_copies":20396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6513,"num_write_kb":26052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6037},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10286","reported_seq":10415,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:26.248628+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:26.248628+0000","last_peered":"2024-09-15T21:24:26.248628+0000","last_clean":"2024-09-15T21:24:26.248628+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:26.248628+0000","last_undegraded":"2024-09-15T21:24:26.248628+0000","last_fullsized":"2024-09-15T21:24:26.248628+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42123264,"num_objects":10284,"num_object_clones":0,"num_object_copies":20568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7058,"num_write_kb":28232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10183","reported_seq":10309,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.582250+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:28.582250+0000","last_peered":"2024-09-15T21:24:28.582250+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:28.582250+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41681379,"num_objects":10177,"num_object_clones":0,"num_object_copies":20354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8115,"num_write_kb":32460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4085},{"shards":"2","objects":6090}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10093","reported_seq":10223,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.322723+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:26.322723+0000","last_peered":"2024-09-15T21:24:26.322723+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:26.322723+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41308160,"num_objects":10085,"num_object_clones":0,"num_object_copies":20170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6766,"num_write_kb":27064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6065},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10247","reported_seq":10366,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:28.599987+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:28.599987+0000","last_peered":"2024-09-15T21:24:28.599987+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:28.599987+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41951715,"num_objects":10243,"num_object_clones":0,"num_object_copies":20486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10243,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8106,"num_write_kb":32424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4090},{"shards":"2","objects":6153}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10214","reported_seq":18672,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.600232+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:28.600232+0000","last_peered":"2024-09-15T21:24:28.600232+0000","last_clean":"2024-09-15T21:24:28.600232+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:28.600232+0000","last_undegraded":"2024-09-15T21:24:28.600232+0000","last_fullsized":"2024-09-15T21:24:28.600232+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41811968,"num_objects":10208,"num_object_clones":0,"num_object_copies":20416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7907,"num_write_kb":31628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10195","reported_seq":10327,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.321104+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:26.321104+0000","last_peered":"2024-09-15T21:24:26.321104+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:26.321104+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41750528,"num_objects":10193,"num_object_clones":0,"num_object_copies":20386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7139,"num_write_kb":28556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6099},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10142","reported_seq":10274,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.356027+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:26.356027+0000","last_peered":"2024-09-15T21:24:26.356027+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:26.356027+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41508864,"num_objects":10134,"num_object_clones":0,"num_object_copies":20268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7051,"num_write_kb":28204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5995},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10029","reported_seq":11858,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:26.376878+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:26.376878+0000","last_peered":"2024-09-15T21:24:26.376878+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:26.376878+0000","last_undegraded":"2024-09-15T21:24:26.376878+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41062400,"num_objects":10025,"num_object_clones":0,"num_object_copies":20050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3236,"num_objects_unfound":0,"num_objects_dirty":10025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6513,"num_write_kb":26052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":846,"num_bytes_recovered":3465216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6789},{"shards":"0,2","objects":3236}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10288","reported_seq":10425,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:26.364993+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:26.364993+0000","last_peered":"2024-09-15T21:24:26.364993+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:26.364993+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42091462,"num_objects":10278,"num_object_clones":0,"num_object_copies":20556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6992,"num_write_kb":27968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6131},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":669029086,"num_objects":163346,"num_object_clones":0,"num_object_copies":326692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3236,"num_objects_unfound":0,"num_objects_dirty":163346,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":123211,"num_write_kb":492844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5011,"num_bytes_recovered":20516864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1359360000,"data_stored":1359211867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160918,"ondisk_log_size":160918,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729796,"kb_used_data":459316,"kb_used_omap":4,"kb_used_meta":270459,"kb_avail":93642044,"statfs":{"total":96636764160,"available":95889453056,"internally_reserved":0,"allocated":470339584,"data_stored":470068009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":276950697},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":893332,"kb_used_data":554884,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93478508,"statfs":{"total":96636764160,"available":95721992192,"internally_reserved":0,"allocated":568201216,"data_stored":567925564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,0,2,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569516,"kb_used_data":312924,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93802324,"statfs":{"total":96636764160,"available":96053579776,"internally_reserved":0,"allocated":320434176,"data_stored":320118781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":262729391},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568254464,"data_stored":568218334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320856064,"data_stored":320780191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":470249472,"data_stored":470213342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:31.772 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Total time run: 240.015 2024-09-15T21:24:31.772 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Total writes made: 165284 2024-09-15T21:24:31.772 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Write size: 4096 2024-09-15T21:24:31.772 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Object size: 4096 2024-09-15T21:24:31.772 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Bandwidth (MB/sec): 2.69 2024-09-15T21:24:31.772 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Stddev Bandwidth: 1.95364 2024-09-15T21:24:31.772 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Max bandwidth (MB/sec): 11.8594 2024-09-15T21:24:31.772 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Min bandwidth (MB/sec): 0 2024-09-15T21:24:31.772 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Average IOPS: 688 2024-09-15T21:24:31.772 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Stddev IOPS: 500.133 2024-09-15T21:24:31.772 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Max IOPS: 3036 2024-09-15T21:24:31.772 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Min IOPS: 0 2024-09-15T21:24:31.773 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Average Latency(s): 0.0290381 2024-09-15T21:24:31.773 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Stddev Latency(s): 0.0424761 2024-09-15T21:24:31.773 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Max latency(s): 2.60418 2024-09-15T21:24:31.773 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Min latency(s): 0.00235499 2024-09-15T21:24:31.783 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Cleaning up (deleting benchmark objects) 2024-09-15T21:24:34.215 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:34.803 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:34.806 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:34.830 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-15T21:24:34.759658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":675414793,"num_objects":164908,"num_object_clones":0,"num_object_copies":329816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3159,"num_objects_unfound":0,"num_objects_dirty":164908,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":125391,"num_write_kb":500319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5089,"num_bytes_recovered":20832275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2207224,"kb_used_data":1341704,"kb_used_omap":12,"kb_used_meta":865459,"kb_avail":280908296,"statfs":{"total":289910292480,"available":287650095104,"internally_reserved":0,"allocated":1373904896,"data_stored":1373034130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":886230019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,3,7,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18493464,"num_objects":4516,"num_object_clones":0,"num_object_copies":9032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-161,"num_objects_unfound":0,"num_objects_dirty":4516,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5134,"num_write_kb":19297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003237"},"pg_stats":[{"pgid":"2.f","version":"45'10572","reported_seq":10647,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.479634+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:31.479634+0000","last_peered":"2024-09-15T21:24:31.479634+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:31.479634+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43302912,"num_objects":10572,"num_object_clones":0,"num_object_copies":21144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10572,"num_write_kb":42288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6440},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10394","reported_seq":10498,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.640321+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:33.640321+0000","last_peered":"2024-09-15T21:24:33.640321+0000","last_clean":"2024-09-15T21:24:33.640321+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:33.640321+0000","last_undegraded":"2024-09-15T21:24:33.640321+0000","last_fullsized":"2024-09-15T21:24:33.640321+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42107846,"num_objects":10282,"num_object_clones":0,"num_object_copies":20564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10282,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8675,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10597","reported_seq":10723,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.367011+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:33.367011+0000","last_peered":"2024-09-15T21:24:33.367011+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:33.367011+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43040792,"num_objects":10509,"num_object_clones":0,"num_object_copies":21018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10509,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8697,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4225},{"shards":"2","objects":6242}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10214","reported_seq":10284,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.596266+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:33.596266+0000","last_peered":"2024-09-15T21:24:33.596266+0000","last_clean":"2024-09-15T21:24:33.596266+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:33.596266+0000","last_undegraded":"2024-09-15T21:24:33.596266+0000","last_fullsized":"2024-09-15T21:24:33.596266+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41369600,"num_objects":10100,"num_object_clones":0,"num_object_copies":20200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10214,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10245","reported_seq":10378,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:31.385820+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:31.385820+0000","last_peered":"2024-09-15T21:24:31.385820+0000","last_clean":"2024-09-15T21:24:31.385820+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:31.385820+0000","last_undegraded":"2024-09-15T21:24:31.385820+0000","last_fullsized":"2024-09-15T21:24:31.385820+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41935331,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6700,"num_write_kb":26800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10312","reported_seq":10448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.430700+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:31.430700+0000","last_peered":"2024-09-15T21:24:31.430700+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:31.430700+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42217955,"num_objects":10308,"num_object_clones":0,"num_object_copies":20616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6988,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6232},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10332","reported_seq":10468,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.307741+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:31.307741+0000","last_peered":"2024-09-15T21:24:31.307741+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:31.307741+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42296262,"num_objects":10328,"num_object_clones":0,"num_object_copies":20656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6643,"num_write_kb":26572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6167},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10427","reported_seq":10556,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:31.421417+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:31.421417+0000","last_peered":"2024-09-15T21:24:31.421417+0000","last_clean":"2024-09-15T21:24:31.421417+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:31.421417+0000","last_undegraded":"2024-09-15T21:24:31.421417+0000","last_fullsized":"2024-09-15T21:24:31.421417+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42700800,"num_objects":10425,"num_object_clones":0,"num_object_copies":20850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7199,"num_write_kb":28796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10306","reported_seq":10432,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.621115+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:33.621115+0000","last_peered":"2024-09-15T21:24:33.621115+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:33.621115+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41808355,"num_objects":10208,"num_object_clones":0,"num_object_copies":20416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10208,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8238,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4039},{"shards":"2","objects":6121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10239","reported_seq":10370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.428173+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:31.428173+0000","last_peered":"2024-09-15T21:24:31.428173+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:31.428173+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906176,"num_objects":10231,"num_object_clones":0,"num_object_copies":20462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6912,"num_write_kb":27648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6211},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10377","reported_seq":10496,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.560959+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:33.560959+0000","last_peered":"2024-09-15T21:24:33.560959+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:33.560959+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42050019,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8236,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4037},{"shards":"2","objects":6177}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10340","reported_seq":18798,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.558446+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:33.558446+0000","last_peered":"2024-09-15T21:24:33.558446+0000","last_clean":"2024-09-15T21:24:33.558446+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:33.558446+0000","last_undegraded":"2024-09-15T21:24:33.558446+0000","last_fullsized":"2024-09-15T21:24:33.558446+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41861120,"num_objects":10220,"num_object_clones":0,"num_object_copies":20440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8033,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10334","reported_seq":10465,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.442088+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:31.442088+0000","last_peered":"2024-09-15T21:24:31.442088+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:31.442088+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42319872,"num_objects":10332,"num_object_clones":0,"num_object_copies":20664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7278,"num_write_kb":29112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6238},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10290","reported_seq":10422,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.425281+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:31.425281+0000","last_peered":"2024-09-15T21:24:31.425281+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:31.425281+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42115072,"num_objects":10282,"num_object_clones":0,"num_object_copies":20564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7199,"num_write_kb":28796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6143},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10190","reported_seq":12171,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:31.455055+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:31.455055+0000","last_peered":"2024-09-15T21:24:31.455055+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:31.455055+0000","last_undegraded":"2024-09-15T21:24:31.455055+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41721856,"num_objects":10186,"num_object_clones":0,"num_object_copies":20372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3159,"num_objects_unfound":0,"num_objects_dirty":10186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6674,"num_write_kb":26696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":923,"num_bytes_recovered":3780608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7027},{"shards":"0,2","objects":3159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10427","reported_seq":10566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.359297+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:31.359297+0000","last_peered":"2024-09-15T21:24:31.359297+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:31.359297+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42660806,"num_objects":10417,"num_object_clones":0,"num_object_copies":20834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10417,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7131,"num_write_kb":28524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6270},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":675414774,"num_objects":164906,"num_object_clones":0,"num_object_copies":329812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3159,"num_objects_unfound":0,"num_objects_dirty":164906,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":125389,"num_write_kb":500317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5088,"num_bytes_recovered":20832256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1372749824,"data_stored":1372593547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735252,"kb_used_data":460612,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93636588,"statfs":{"total":96636764160,"available":95883866112,"internally_reserved":0,"allocated":471666688,"data_stored":471391041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":908384,"kb_used_data":561616,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93463456,"statfs":{"total":96636764160,"available":95706578944,"internally_reserved":0,"allocated":575094784,"data_stored":574819132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355069611},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563588,"kb_used_data":319476,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93808252,"statfs":{"total":96636764160,"available":96059650048,"internally_reserved":0,"allocated":327143424,"data_stored":326823957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":249949871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":575115264,"data_stored":575079134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326656000,"data_stored":326576055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":470978560,"data_stored":470938358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:34.832 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:35.376 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:35.377 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:35.404 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-15T21:24:34.759658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":675414793,"num_objects":164908,"num_object_clones":0,"num_object_copies":329816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3159,"num_objects_unfound":0,"num_objects_dirty":164908,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":125391,"num_write_kb":500319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5089,"num_bytes_recovered":20832275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2207224,"kb_used_data":1341704,"kb_used_omap":12,"kb_used_meta":865459,"kb_avail":280908296,"statfs":{"total":289910292480,"available":287650095104,"internally_reserved":0,"allocated":1373904896,"data_stored":1373034130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":886230019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,3,7,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18493464,"num_objects":4516,"num_object_clones":0,"num_object_copies":9032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-161,"num_objects_unfound":0,"num_objects_dirty":4516,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5134,"num_write_kb":19297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003237"},"pg_stats":[{"pgid":"2.f","version":"45'10572","reported_seq":10647,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.479634+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:31.479634+0000","last_peered":"2024-09-15T21:24:31.479634+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:31.479634+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43302912,"num_objects":10572,"num_object_clones":0,"num_object_copies":21144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10572,"num_write_kb":42288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6440},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10394","reported_seq":10498,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.640321+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:33.640321+0000","last_peered":"2024-09-15T21:24:33.640321+0000","last_clean":"2024-09-15T21:24:33.640321+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:33.640321+0000","last_undegraded":"2024-09-15T21:24:33.640321+0000","last_fullsized":"2024-09-15T21:24:33.640321+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42107846,"num_objects":10282,"num_object_clones":0,"num_object_copies":20564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10282,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8675,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10597","reported_seq":10723,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.367011+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:33.367011+0000","last_peered":"2024-09-15T21:24:33.367011+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:33.367011+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43040792,"num_objects":10509,"num_object_clones":0,"num_object_copies":21018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10509,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8697,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4225},{"shards":"2","objects":6242}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10214","reported_seq":10284,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.596266+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:33.596266+0000","last_peered":"2024-09-15T21:24:33.596266+0000","last_clean":"2024-09-15T21:24:33.596266+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:33.596266+0000","last_undegraded":"2024-09-15T21:24:33.596266+0000","last_fullsized":"2024-09-15T21:24:33.596266+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41369600,"num_objects":10100,"num_object_clones":0,"num_object_copies":20200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10214,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10245","reported_seq":10378,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:31.385820+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:31.385820+0000","last_peered":"2024-09-15T21:24:31.385820+0000","last_clean":"2024-09-15T21:24:31.385820+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:31.385820+0000","last_undegraded":"2024-09-15T21:24:31.385820+0000","last_fullsized":"2024-09-15T21:24:31.385820+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41935331,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6700,"num_write_kb":26800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10312","reported_seq":10448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.430700+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:31.430700+0000","last_peered":"2024-09-15T21:24:31.430700+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:31.430700+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42217955,"num_objects":10308,"num_object_clones":0,"num_object_copies":20616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6988,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6232},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10332","reported_seq":10468,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.307741+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:31.307741+0000","last_peered":"2024-09-15T21:24:31.307741+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:31.307741+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42296262,"num_objects":10328,"num_object_clones":0,"num_object_copies":20656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6643,"num_write_kb":26572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6167},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10427","reported_seq":10556,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:31.421417+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:31.421417+0000","last_peered":"2024-09-15T21:24:31.421417+0000","last_clean":"2024-09-15T21:24:31.421417+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:31.421417+0000","last_undegraded":"2024-09-15T21:24:31.421417+0000","last_fullsized":"2024-09-15T21:24:31.421417+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42700800,"num_objects":10425,"num_object_clones":0,"num_object_copies":20850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7199,"num_write_kb":28796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10306","reported_seq":10432,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.621115+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:33.621115+0000","last_peered":"2024-09-15T21:24:33.621115+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:33.621115+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41808355,"num_objects":10208,"num_object_clones":0,"num_object_copies":20416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10208,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8238,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4039},{"shards":"2","objects":6121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10239","reported_seq":10370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.428173+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:31.428173+0000","last_peered":"2024-09-15T21:24:31.428173+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:31.428173+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906176,"num_objects":10231,"num_object_clones":0,"num_object_copies":20462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6912,"num_write_kb":27648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6211},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10377","reported_seq":10496,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.560959+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:33.560959+0000","last_peered":"2024-09-15T21:24:33.560959+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:33.560959+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42050019,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8236,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4037},{"shards":"2","objects":6177}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10340","reported_seq":18798,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.558446+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:33.558446+0000","last_peered":"2024-09-15T21:24:33.558446+0000","last_clean":"2024-09-15T21:24:33.558446+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:33.558446+0000","last_undegraded":"2024-09-15T21:24:33.558446+0000","last_fullsized":"2024-09-15T21:24:33.558446+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41861120,"num_objects":10220,"num_object_clones":0,"num_object_copies":20440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8033,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10334","reported_seq":10465,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.442088+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:31.442088+0000","last_peered":"2024-09-15T21:24:31.442088+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:31.442088+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42319872,"num_objects":10332,"num_object_clones":0,"num_object_copies":20664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7278,"num_write_kb":29112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6238},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10290","reported_seq":10422,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.425281+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:31.425281+0000","last_peered":"2024-09-15T21:24:31.425281+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:31.425281+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42115072,"num_objects":10282,"num_object_clones":0,"num_object_copies":20564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7199,"num_write_kb":28796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6143},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10190","reported_seq":12171,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:31.455055+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:31.455055+0000","last_peered":"2024-09-15T21:24:31.455055+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:31.455055+0000","last_undegraded":"2024-09-15T21:24:31.455055+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41721856,"num_objects":10186,"num_object_clones":0,"num_object_copies":20372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3159,"num_objects_unfound":0,"num_objects_dirty":10186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6674,"num_write_kb":26696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":923,"num_bytes_recovered":3780608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7027},{"shards":"0,2","objects":3159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10427","reported_seq":10566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.359297+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:31.359297+0000","last_peered":"2024-09-15T21:24:31.359297+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:31.359297+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42660806,"num_objects":10417,"num_object_clones":0,"num_object_copies":20834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10417,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7131,"num_write_kb":28524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6270},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":675414774,"num_objects":164906,"num_object_clones":0,"num_object_copies":329812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3159,"num_objects_unfound":0,"num_objects_dirty":164906,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":125389,"num_write_kb":500317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5088,"num_bytes_recovered":20832256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1372749824,"data_stored":1372593547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735252,"kb_used_data":460612,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93636588,"statfs":{"total":96636764160,"available":95883866112,"internally_reserved":0,"allocated":471666688,"data_stored":471391041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":908384,"kb_used_data":561616,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93463456,"statfs":{"total":96636764160,"available":95706578944,"internally_reserved":0,"allocated":575094784,"data_stored":574819132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355069611},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563588,"kb_used_data":319476,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93808252,"statfs":{"total":96636764160,"available":96059650048,"internally_reserved":0,"allocated":327143424,"data_stored":326823957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":249949871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":575115264,"data_stored":575079134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326656000,"data_stored":326576055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":470978560,"data_stored":470938358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:35.406 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:35.406 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10572", 'reported_seq': 10647, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:31.479634+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:24:31.479634+0000', 'last_peered': '2024-09-15T21:24:31.479634+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:24:31.479634+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43302912, 'num_objects': 10572, 'num_object_clones': 0, 'num_object_copies': 21144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10572, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10572, 'num_write_kb': 42288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6440}, {'shards': '0,2', 'objects': 4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:35.406 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:24:35.406 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'10597", 'reported_seq': 10723, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:33.367011+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:24:33.367011+0000', 'last_peered': '2024-09-15T21:24:33.367011+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:24:33.367011+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 43040792, 'num_objects': 10509, 'num_object_clones': 0, 'num_object_copies': 21018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10509, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8697, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4225}, {'shards': '2', 'objects': 6242}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:35.406 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:35.406 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'10312", 'reported_seq': 10448, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:31.430700+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:24:31.430700+0000', 'last_peered': '2024-09-15T21:24:31.430700+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:24:31.430700+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42217955, 'num_objects': 10308, 'num_object_clones': 0, 'num_object_copies': 20616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10308, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 6988, 'num_write_kb': 27952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6232}, {'shards': '0,2', 'objects': 4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:35.406 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:35.406 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10332", 'reported_seq': 10468, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:31.307741+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:24:31.307741+0000', 'last_peered': '2024-09-15T21:24:31.307741+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:24:31.307741+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42296262, 'num_objects': 10328, 'num_object_clones': 0, 'num_object_copies': 20656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10328, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6643, 'num_write_kb': 26572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6167}, {'shards': '0,2', 'objects': 4161}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:35.406 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:24:35.406 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'10306", 'reported_seq': 10432, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:33.621115+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:24:33.621115+0000', 'last_peered': '2024-09-15T21:24:33.621115+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:24:33.621115+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41808355, 'num_objects': 10208, 'num_object_clones': 0, 'num_object_copies': 20416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10208, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8238, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4039}, {'shards': '2', 'objects': 6121}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:35.407 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:35.407 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'10239", 'reported_seq': 10370, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:31.428173+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:24:31.428173+0000', 'last_peered': '2024-09-15T21:24:31.428173+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:24:31.428173+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41906176, 'num_objects': 10231, 'num_object_clones': 0, 'num_object_copies': 20462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10231, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6912, 'num_write_kb': 27648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6211}, {'shards': '0,2', 'objects': 4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:35.407 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:35.407 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'10377", 'reported_seq': 10496, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:33.560959+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:24:33.560959+0000', 'last_peered': '2024-09-15T21:24:33.560959+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:24:33.560959+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42050019, 'num_objects': 10267, 'num_object_clones': 0, 'num_object_copies': 20534, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10267, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8236, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4037}, {'shards': '2', 'objects': 6177}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:35.407 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:35.407 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'10334", 'reported_seq': 10465, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:31.442088+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:24:31.442088+0000', 'last_peered': '2024-09-15T21:24:31.442088+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:24:31.442088+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42319872, 'num_objects': 10332, 'num_object_clones': 0, 'num_object_copies': 20664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10332, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7278, 'num_write_kb': 29112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6238}, {'shards': '0,2', 'objects': 4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:35.407 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:35.407 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'10290", 'reported_seq': 10422, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:31.425281+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:24:31.425281+0000', 'last_peered': '2024-09-15T21:24:31.425281+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:24:31.425281+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42115072, 'num_objects': 10282, 'num_object_clones': 0, 'num_object_copies': 20564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10282, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7199, 'num_write_kb': 28796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6143}, {'shards': '0,2', 'objects': 4139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:35.407 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:35.407 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'10190", 'reported_seq': 12171, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:24:31.455055+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:24:31.455055+0000', 'last_peered': '2024-09-15T21:24:31.455055+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:24:31.455055+0000', 'last_undegraded': '2024-09-15T21:24:31.455055+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41721856, 'num_objects': 10186, 'num_object_clones': 0, 'num_object_copies': 20372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3159, 'num_objects_unfound': 0, 'num_objects_dirty': 10186, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6674, 'num_write_kb': 26696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 923, 'num_bytes_recovered': 3780608, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7027}, {'shards': '0,2', 'objects': 3159}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:35.408 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:24:35.408 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10427", 'reported_seq': 10566, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:31.359297+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:24:31.359297+0000', 'last_peered': '2024-09-15T21:24:31.359297+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:24:31.359297+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42660806, 'num_objects': 10417, 'num_object_clones': 0, 'num_object_copies': 20834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10417, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7131, 'num_write_kb': 28524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6270}, {'shards': '0,2', 'objects': 4147}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:35.408 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:36.096 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:36.097 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/329816 objects degraded (12.499%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":327,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":164908,"data_bytes":675414793,"bytes_used":2260197376,"bytes_avail":287650095104,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":329816,"degraded_ratio":0.12499393601280713,"misplaced_objects":3159,"misplaced_total":329816,"misplaced_ratio":0.0095780677711208677,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":54939,"recovering_keys_per_sec":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":1646233,"read_op_per_sec":0,"write_op_per_sec":427},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:36.120 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:36.120 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:36.669 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:36.671 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:36.693 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":201,"stamp":"2024-09-15T21:24:34.759658+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":675414793,"num_objects":164908,"num_object_clones":0,"num_object_copies":329816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3159,"num_objects_unfound":0,"num_objects_dirty":164908,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":125391,"num_write_kb":500319,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5089,"num_bytes_recovered":20832275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2207224,"kb_used_data":1341704,"kb_used_omap":12,"kb_used_meta":865459,"kb_avail":280908296,"statfs":{"total":289910292480,"available":287650095104,"internally_reserved":0,"allocated":1373904896,"data_stored":1373034130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":886230019},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,3,7,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18493464,"num_objects":4516,"num_object_clones":0,"num_object_copies":9032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-161,"num_objects_unfound":0,"num_objects_dirty":4516,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5134,"num_write_kb":19297,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003237"},"pg_stats":[{"pgid":"2.f","version":"45'10572","reported_seq":10647,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.479634+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:31.479634+0000","last_peered":"2024-09-15T21:24:31.479634+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:31.479634+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43302912,"num_objects":10572,"num_object_clones":0,"num_object_copies":21144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10572,"num_write_kb":42288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6440},{"shards":"0,2","objects":4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10394","reported_seq":10498,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.640321+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:33.640321+0000","last_peered":"2024-09-15T21:24:33.640321+0000","last_clean":"2024-09-15T21:24:33.640321+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:33.640321+0000","last_undegraded":"2024-09-15T21:24:33.640321+0000","last_fullsized":"2024-09-15T21:24:33.640321+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42107846,"num_objects":10282,"num_object_clones":0,"num_object_copies":20564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10282,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8675,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10597","reported_seq":10723,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.367011+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:33.367011+0000","last_peered":"2024-09-15T21:24:33.367011+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:33.367011+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43040792,"num_objects":10509,"num_object_clones":0,"num_object_copies":21018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10509,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8697,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4225},{"shards":"2","objects":6242}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10214","reported_seq":10284,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.596266+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:33.596266+0000","last_peered":"2024-09-15T21:24:33.596266+0000","last_clean":"2024-09-15T21:24:33.596266+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:33.596266+0000","last_undegraded":"2024-09-15T21:24:33.596266+0000","last_fullsized":"2024-09-15T21:24:33.596266+0000","mapping_epoch":35,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41369600,"num_objects":10100,"num_object_clones":0,"num_object_copies":20200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10214,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10245","reported_seq":10378,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:31.385820+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:31.385820+0000","last_peered":"2024-09-15T21:24:31.385820+0000","last_clean":"2024-09-15T21:24:31.385820+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:31.385820+0000","last_undegraded":"2024-09-15T21:24:31.385820+0000","last_fullsized":"2024-09-15T21:24:31.385820+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41935331,"num_objects":10239,"num_object_clones":0,"num_object_copies":20478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6700,"num_write_kb":26800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10312","reported_seq":10448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.430700+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:31.430700+0000","last_peered":"2024-09-15T21:24:31.430700+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:31.430700+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42217955,"num_objects":10308,"num_object_clones":0,"num_object_copies":20616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6988,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6232},{"shards":"0,2","objects":4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10332","reported_seq":10468,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.307741+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:31.307741+0000","last_peered":"2024-09-15T21:24:31.307741+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:31.307741+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42296262,"num_objects":10328,"num_object_clones":0,"num_object_copies":20656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6643,"num_write_kb":26572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6167},{"shards":"0,2","objects":4161}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10427","reported_seq":10556,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:31.421417+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:31.421417+0000","last_peered":"2024-09-15T21:24:31.421417+0000","last_clean":"2024-09-15T21:24:31.421417+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:31.421417+0000","last_undegraded":"2024-09-15T21:24:31.421417+0000","last_fullsized":"2024-09-15T21:24:31.421417+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42700800,"num_objects":10425,"num_object_clones":0,"num_object_copies":20850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10425,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7199,"num_write_kb":28796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10306","reported_seq":10432,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.621115+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:33.621115+0000","last_peered":"2024-09-15T21:24:33.621115+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:33.621115+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41808355,"num_objects":10208,"num_object_clones":0,"num_object_copies":20416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10208,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8238,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4039},{"shards":"2","objects":6121}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10239","reported_seq":10370,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.428173+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:31.428173+0000","last_peered":"2024-09-15T21:24:31.428173+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:31.428173+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41906176,"num_objects":10231,"num_object_clones":0,"num_object_copies":20462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6912,"num_write_kb":27648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6211},{"shards":"0,2","objects":4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10377","reported_seq":10496,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:33.560959+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:33.560959+0000","last_peered":"2024-09-15T21:24:33.560959+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:33.560959+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42050019,"num_objects":10267,"num_object_clones":0,"num_object_copies":20534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10267,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8236,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4037},{"shards":"2","objects":6177}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10340","reported_seq":18798,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:33.558446+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:33.558446+0000","last_peered":"2024-09-15T21:24:33.558446+0000","last_clean":"2024-09-15T21:24:33.558446+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:33.558446+0000","last_undegraded":"2024-09-15T21:24:33.558446+0000","last_fullsized":"2024-09-15T21:24:33.558446+0000","mapping_epoch":44,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41861120,"num_objects":10220,"num_object_clones":0,"num_object_copies":20440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8033,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10334","reported_seq":10465,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.442088+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:31.442088+0000","last_peered":"2024-09-15T21:24:31.442088+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:31.442088+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42319872,"num_objects":10332,"num_object_clones":0,"num_object_copies":20664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7278,"num_write_kb":29112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6238},{"shards":"0,2","objects":4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10290","reported_seq":10422,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.425281+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:31.425281+0000","last_peered":"2024-09-15T21:24:31.425281+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:31.425281+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42115072,"num_objects":10282,"num_object_clones":0,"num_object_copies":20564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7199,"num_write_kb":28796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6143},{"shards":"0,2","objects":4139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10190","reported_seq":12171,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:31.455055+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:31.455055+0000","last_peered":"2024-09-15T21:24:31.455055+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:31.455055+0000","last_undegraded":"2024-09-15T21:24:31.455055+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41721856,"num_objects":10186,"num_object_clones":0,"num_object_copies":20372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3159,"num_objects_unfound":0,"num_objects_dirty":10186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6674,"num_write_kb":26696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":923,"num_bytes_recovered":3780608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7027},{"shards":"0,2","objects":3159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10427","reported_seq":10566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:31.359297+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:31.359297+0000","last_peered":"2024-09-15T21:24:31.359297+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:31.359297+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42660806,"num_objects":10417,"num_object_clones":0,"num_object_copies":20834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10417,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7131,"num_write_kb":28524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6270},{"shards":"0,2","objects":4147}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":675414774,"num_objects":164906,"num_object_clones":0,"num_object_copies":329812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3159,"num_objects_unfound":0,"num_objects_dirty":164906,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":125389,"num_write_kb":500317,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5088,"num_bytes_recovered":20832256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1372749824,"data_stored":1372593547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735252,"kb_used_data":460612,"kb_used_omap":4,"kb_used_meta":274619,"kb_avail":93636588,"statfs":{"total":96636764160,"available":95883866112,"internally_reserved":0,"allocated":471666688,"data_stored":471391041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":281210537},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":908384,"kb_used_data":561616,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93463456,"statfs":{"total":96636764160,"available":95706578944,"internally_reserved":0,"allocated":575094784,"data_stored":574819132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355069611},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563588,"kb_used_data":319476,"kb_used_omap":4,"kb_used_meta":244091,"kb_avail":93808252,"statfs":{"total":96636764160,"available":96059650048,"internally_reserved":0,"allocated":327143424,"data_stored":326823957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":249949871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":575115264,"data_stored":575079134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":326656000,"data_stored":326576055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":470978560,"data_stored":470938358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:39.695 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:40.255 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:40.257 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:40.278 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-15T21:24:38.760579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666829577,"num_objects":162812,"num_object_clones":0,"num_object_copies":325624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3073,"num_objects_unfound":0,"num_objects_dirty":162812,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":127647,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5175,"num_bytes_recovered":21184531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160955,"ondisk_log_size":160955,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2176044,"kb_used_data":1331324,"kb_used_omap":12,"kb_used_meta":844659,"kb_avail":280939476,"statfs":{"total":289910292480,"available":287682023424,"internally_reserved":0,"allocated":1363275776,"data_stored":1362405010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":864930819},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,8,2,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":978968,"num_objects":240,"num_object_clones":0,"num_object_copies":480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-163,"num_objects_unfound":0,"num_objects_dirty":240,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5210,"num_write_kb":10897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":667648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003043"},"pg_stats":[{"pgid":"2.f","version":"45'10697","reported_seq":10773,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.605141+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:36.605141+0000","last_peered":"2024-09-15T21:24:36.605141+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:36.605141+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42823680,"num_objects":10455,"num_object_clones":0,"num_object_copies":20910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10697,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6323},{"shards":"0,2","objects":4011}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10531","reported_seq":10636,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:38.553936+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:38.553936+0000","last_peered":"2024-09-15T21:24:38.553936+0000","last_clean":"2024-09-15T21:24:38.553936+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:38.553936+0000","last_undegraded":"2024-09-15T21:24:38.553936+0000","last_fullsized":"2024-09-15T21:24:38.553936+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41546694,"num_objects":10145,"num_object_clones":0,"num_object_copies":20290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10145,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8812,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10722","reported_seq":10848,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.543657+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:38.543657+0000","last_peered":"2024-09-15T21:24:38.543657+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:38.543657+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42528792,"num_objects":10384,"num_object_clones":0,"num_object_copies":20768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10384,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8822,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4100},{"shards":"2","objects":6117}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10347","reported_seq":10417,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:38.629595+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:38.629595+0000","last_peered":"2024-09-15T21:24:38.629595+0000","last_clean":"2024-09-15T21:24:38.629595+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:38.629595+0000","last_undegraded":"2024-09-15T21:24:38.629595+0000","last_fullsized":"2024-09-15T21:24:38.629595+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40824832,"num_objects":9967,"num_object_clones":0,"num_object_copies":19934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10347,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10393","reported_seq":10526,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:36.367070+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:36.367070+0000","last_peered":"2024-09-15T21:24:36.367070+0000","last_clean":"2024-09-15T21:24:36.367070+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:36.367070+0000","last_undegraded":"2024-09-15T21:24:36.367070+0000","last_fullsized":"2024-09-15T21:24:36.367070+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41419235,"num_objects":10113,"num_object_clones":0,"num_object_copies":20226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6848,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10464","reported_seq":10600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.543711+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:36.543711+0000","last_peered":"2024-09-15T21:24:36.543711+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:36.543711+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41660899,"num_objects":10172,"num_object_clones":0,"num_object_copies":20344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10172,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7140,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6096},{"shards":"0,2","objects":3932}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10474","reported_seq":10610,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.429357+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:36.429357+0000","last_peered":"2024-09-15T21:24:36.429357+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:36.429357+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41763782,"num_objects":10198,"num_object_clones":0,"num_object_copies":20396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6785,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6037},{"shards":"0,2","objects":4025}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10578","reported_seq":10706,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:36.574288+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:36.574288+0000","last_peered":"2024-09-15T21:24:36.574288+0000","last_clean":"2024-09-15T21:24:36.574288+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:36.574288+0000","last_undegraded":"2024-09-15T21:24:36.574288+0000","last_fullsized":"2024-09-15T21:24:36.574288+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42147840,"num_objects":10290,"num_object_clones":0,"num_object_copies":20580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7350,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10454","reported_seq":10581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.586704+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:38.586704+0000","last_peered":"2024-09-15T21:24:38.586704+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:38.586704+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41202147,"num_objects":10060,"num_object_clones":0,"num_object_copies":20120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10060,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8386,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3891},{"shards":"2","objects":5973}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10386","reported_seq":10516,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.567544+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:36.567544+0000","last_peered":"2024-09-15T21:24:36.567544+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:36.567544+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41353216,"num_objects":10096,"num_object_clones":0,"num_object_copies":20192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7059,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6076},{"shards":"0,2","objects":3879}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10513","reported_seq":10632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.591259+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:38.591259+0000","last_peered":"2024-09-15T21:24:38.591259+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:38.591259+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41492963,"num_objects":10131,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10131,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8372,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3901},{"shards":"2","objects":6041}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10480","reported_seq":18938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:38.634033+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:38.634033+0000","last_peered":"2024-09-15T21:24:38.634033+0000","last_clean":"2024-09-15T21:24:38.634033+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:38.634033+0000","last_undegraded":"2024-09-15T21:24:38.634033+0000","last_fullsized":"2024-09-15T21:24:38.634033+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41287680,"num_objects":10080,"num_object_clones":0,"num_object_copies":20160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8173,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10483","reported_seq":10615,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.624472+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:36.624472+0000","last_peered":"2024-09-15T21:24:36.624472+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:36.624472+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41791488,"num_objects":10203,"num_object_clones":0,"num_object_copies":20406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7427,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6109},{"shards":"0,2","objects":3955}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10428","reported_seq":10559,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.583667+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:36.583667+0000","last_peered":"2024-09-15T21:24:36.583667+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:36.583667+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41623552,"num_objects":10162,"num_object_clones":0,"num_object_copies":20324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7337,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6023},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10341","reported_seq":12496,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:36.575941+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:36.575941+0000","last_peered":"2024-09-15T21:24:36.575941+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:36.575941+0000","last_undegraded":"2024-09-15T21:24:36.575941+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41152512,"num_objects":10047,"num_object_clones":0,"num_object_copies":20094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3073,"num_objects_unfound":0,"num_objects_dirty":10047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6825,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1009,"num_bytes_recovered":4132864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6974},{"shards":"0,2","objects":3073}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10561","reported_seq":10700,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.502525+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:36.502525+0000","last_peered":"2024-09-15T21:24:36.502525+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:36.502525+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42210246,"num_objects":10307,"num_object_clones":0,"num_object_copies":20614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10307,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7265,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6160},{"shards":"0,2","objects":4025}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":666829558,"num_objects":162810,"num_object_clones":0,"num_object_copies":325620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3073,"num_objects_unfound":0,"num_objects_dirty":162810,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":127645,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5174,"num_bytes_recovered":21184512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1361223680,"data_stored":1361067403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160952,"ondisk_log_size":160952,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718540,"kb_used_data":456380,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93653300,"statfs":{"total":96636764160,"available":95900979200,"internally_reserved":0,"allocated":467333120,"data_stored":467057473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,6,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":890628,"kb_used_data":556340,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93481212,"statfs":{"total":96636764160,"available":95724761088,"internally_reserved":0,"allocated":569692160,"data_stored":569416508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566876,"kb_used_data":318604,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93804964,"statfs":{"total":96636764160,"available":96056283136,"internally_reserved":0,"allocated":326250496,"data_stored":325931029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":254209711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568827904,"data_stored":568791774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325791744,"data_stored":325711799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466604032,"data_stored":466563830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:40.280 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:40.831 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:40.831 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:40.854 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-15T21:24:40.761064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666829577,"num_objects":162812,"num_object_clones":0,"num_object_copies":325624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3073,"num_objects_unfound":0,"num_objects_dirty":162812,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":127647,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5175,"num_bytes_recovered":21184531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160955,"ondisk_log_size":160955,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2176044,"kb_used_data":1331324,"kb_used_omap":12,"kb_used_meta":844659,"kb_avail":280939476,"statfs":{"total":289910292480,"available":287682023424,"internally_reserved":0,"allocated":1363275776,"data_stored":1362405010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":864930819},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,8,2,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2199528,"num_objects":-536,"num_object_clones":0,"num_object_copies":-1072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-163,"num_objects_unfound":0,"num_objects_dirty":-536,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4434,"num_write_kb":7793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":667648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003135"},"pg_stats":[{"pgid":"2.f","version":"45'10697","reported_seq":10773,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.605141+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:36.605141+0000","last_peered":"2024-09-15T21:24:36.605141+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:36.605141+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42823680,"num_objects":10455,"num_object_clones":0,"num_object_copies":20910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10697,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6323},{"shards":"0,2","objects":4011}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10531","reported_seq":10636,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:38.553936+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:38.553936+0000","last_peered":"2024-09-15T21:24:38.553936+0000","last_clean":"2024-09-15T21:24:38.553936+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:38.553936+0000","last_undegraded":"2024-09-15T21:24:38.553936+0000","last_fullsized":"2024-09-15T21:24:38.553936+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41546694,"num_objects":10145,"num_object_clones":0,"num_object_copies":20290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10145,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8812,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10722","reported_seq":10848,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.543657+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:38.543657+0000","last_peered":"2024-09-15T21:24:38.543657+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:38.543657+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42528792,"num_objects":10384,"num_object_clones":0,"num_object_copies":20768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10384,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8822,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4100},{"shards":"2","objects":6117}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10347","reported_seq":10417,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:38.629595+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:38.629595+0000","last_peered":"2024-09-15T21:24:38.629595+0000","last_clean":"2024-09-15T21:24:38.629595+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:38.629595+0000","last_undegraded":"2024-09-15T21:24:38.629595+0000","last_fullsized":"2024-09-15T21:24:38.629595+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40824832,"num_objects":9967,"num_object_clones":0,"num_object_copies":19934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10347,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10393","reported_seq":10526,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:36.367070+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:36.367070+0000","last_peered":"2024-09-15T21:24:36.367070+0000","last_clean":"2024-09-15T21:24:36.367070+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:36.367070+0000","last_undegraded":"2024-09-15T21:24:36.367070+0000","last_fullsized":"2024-09-15T21:24:36.367070+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41419235,"num_objects":10113,"num_object_clones":0,"num_object_copies":20226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6848,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10464","reported_seq":10600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.543711+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:36.543711+0000","last_peered":"2024-09-15T21:24:36.543711+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:36.543711+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41660899,"num_objects":10172,"num_object_clones":0,"num_object_copies":20344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10172,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7140,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6096},{"shards":"0,2","objects":3932}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10474","reported_seq":10610,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.429357+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:36.429357+0000","last_peered":"2024-09-15T21:24:36.429357+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:36.429357+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41763782,"num_objects":10198,"num_object_clones":0,"num_object_copies":20396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6785,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6037},{"shards":"0,2","objects":4025}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10578","reported_seq":10706,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:36.574288+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:36.574288+0000","last_peered":"2024-09-15T21:24:36.574288+0000","last_clean":"2024-09-15T21:24:36.574288+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:36.574288+0000","last_undegraded":"2024-09-15T21:24:36.574288+0000","last_fullsized":"2024-09-15T21:24:36.574288+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42147840,"num_objects":10290,"num_object_clones":0,"num_object_copies":20580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7350,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10454","reported_seq":10581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.586704+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:38.586704+0000","last_peered":"2024-09-15T21:24:38.586704+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:38.586704+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41202147,"num_objects":10060,"num_object_clones":0,"num_object_copies":20120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10060,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8386,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3891},{"shards":"2","objects":5973}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10386","reported_seq":10516,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.567544+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:36.567544+0000","last_peered":"2024-09-15T21:24:36.567544+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:36.567544+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41353216,"num_objects":10096,"num_object_clones":0,"num_object_copies":20192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7059,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6076},{"shards":"0,2","objects":3879}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10513","reported_seq":10632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.591259+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:38.591259+0000","last_peered":"2024-09-15T21:24:38.591259+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:38.591259+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41492963,"num_objects":10131,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10131,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8372,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3901},{"shards":"2","objects":6041}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10480","reported_seq":18938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:38.634033+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:38.634033+0000","last_peered":"2024-09-15T21:24:38.634033+0000","last_clean":"2024-09-15T21:24:38.634033+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:38.634033+0000","last_undegraded":"2024-09-15T21:24:38.634033+0000","last_fullsized":"2024-09-15T21:24:38.634033+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41287680,"num_objects":10080,"num_object_clones":0,"num_object_copies":20160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8173,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10483","reported_seq":10615,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.624472+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:36.624472+0000","last_peered":"2024-09-15T21:24:36.624472+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:36.624472+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41791488,"num_objects":10203,"num_object_clones":0,"num_object_copies":20406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7427,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6109},{"shards":"0,2","objects":3955}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10428","reported_seq":10559,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.583667+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:36.583667+0000","last_peered":"2024-09-15T21:24:36.583667+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:36.583667+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41623552,"num_objects":10162,"num_object_clones":0,"num_object_copies":20324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7337,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6023},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10341","reported_seq":12496,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:36.575941+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:36.575941+0000","last_peered":"2024-09-15T21:24:36.575941+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:36.575941+0000","last_undegraded":"2024-09-15T21:24:36.575941+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41152512,"num_objects":10047,"num_object_clones":0,"num_object_copies":20094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3073,"num_objects_unfound":0,"num_objects_dirty":10047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6825,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1009,"num_bytes_recovered":4132864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6974},{"shards":"0,2","objects":3073}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10561","reported_seq":10700,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.502525+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:36.502525+0000","last_peered":"2024-09-15T21:24:36.502525+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:36.502525+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42210246,"num_objects":10307,"num_object_clones":0,"num_object_copies":20614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10307,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7265,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6160},{"shards":"0,2","objects":4025}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":666829558,"num_objects":162810,"num_object_clones":0,"num_object_copies":325620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3073,"num_objects_unfound":0,"num_objects_dirty":162810,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":127645,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5174,"num_bytes_recovered":21184512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1361223680,"data_stored":1361067403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160952,"ondisk_log_size":160952,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718540,"kb_used_data":456380,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93653300,"statfs":{"total":96636764160,"available":95900979200,"internally_reserved":0,"allocated":467333120,"data_stored":467057473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,6,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":890628,"kb_used_data":556340,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93481212,"statfs":{"total":96636764160,"available":95724761088,"internally_reserved":0,"allocated":569692160,"data_stored":569416508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566876,"kb_used_data":318604,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93804964,"statfs":{"total":96636764160,"available":96056283136,"internally_reserved":0,"allocated":326250496,"data_stored":325931029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":254209711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568827904,"data_stored":568791774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325791744,"data_stored":325711799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466604032,"data_stored":466563830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:40.856 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:40.856 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10697", 'reported_seq': 10773, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:36.605141+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:24:36.605141+0000', 'last_peered': '2024-09-15T21:24:36.605141+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:24:36.605141+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42823680, 'num_objects': 10455, 'num_object_clones': 0, 'num_object_copies': 20910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10455, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10697, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6323}, {'shards': '0,2', 'objects': 4011}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:40.856 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:24:40.856 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'10722", 'reported_seq': 10848, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:38.543657+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:24:38.543657+0000', 'last_peered': '2024-09-15T21:24:38.543657+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:24:38.543657+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42528792, 'num_objects': 10384, 'num_object_clones': 0, 'num_object_copies': 20768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10384, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8822, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4100}, {'shards': '2', 'objects': 6117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:40.856 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:40.856 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'10464", 'reported_seq': 10600, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:36.543711+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:24:36.543711+0000', 'last_peered': '2024-09-15T21:24:36.543711+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:24:36.543711+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41660899, 'num_objects': 10172, 'num_object_clones': 0, 'num_object_copies': 20344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10172, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7140, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6096}, {'shards': '0,2', 'objects': 3932}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:40.856 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:40.856 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10474", 'reported_seq': 10610, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:36.429357+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:24:36.429357+0000', 'last_peered': '2024-09-15T21:24:36.429357+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:24:36.429357+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41763782, 'num_objects': 10198, 'num_object_clones': 0, 'num_object_copies': 20396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10198, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6785, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6037}, {'shards': '0,2', 'objects': 4025}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:40.857 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:24:40.857 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'10454", 'reported_seq': 10581, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:38.586704+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:24:38.586704+0000', 'last_peered': '2024-09-15T21:24:38.586704+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:24:38.586704+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41202147, 'num_objects': 10060, 'num_object_clones': 0, 'num_object_copies': 20120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10060, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8386, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3891}, {'shards': '2', 'objects': 5973}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:40.857 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:40.857 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'10386", 'reported_seq': 10516, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:36.567544+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:24:36.567544+0000', 'last_peered': '2024-09-15T21:24:36.567544+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:24:36.567544+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41353216, 'num_objects': 10096, 'num_object_clones': 0, 'num_object_copies': 20192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7059, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6076}, {'shards': '0,2', 'objects': 3879}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:40.857 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:40.857 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'10513", 'reported_seq': 10632, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:38.591259+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:24:38.591259+0000', 'last_peered': '2024-09-15T21:24:38.591259+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:24:38.591259+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41492963, 'num_objects': 10131, 'num_object_clones': 0, 'num_object_copies': 20262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10131, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8372, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3901}, {'shards': '2', 'objects': 6041}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:40.857 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:40.857 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'10483", 'reported_seq': 10615, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:36.624472+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:24:36.624472+0000', 'last_peered': '2024-09-15T21:24:36.624472+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:24:36.624472+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41791488, 'num_objects': 10203, 'num_object_clones': 0, 'num_object_copies': 20406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10203, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7427, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6109}, {'shards': '0,2', 'objects': 3955}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:40.857 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:40.857 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'10428", 'reported_seq': 10559, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:36.583667+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:24:36.583667+0000', 'last_peered': '2024-09-15T21:24:36.583667+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:24:36.583667+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41623552, 'num_objects': 10162, 'num_object_clones': 0, 'num_object_copies': 20324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10162, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7337, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6023}, {'shards': '0,2', 'objects': 4010}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:40.857 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:40.858 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'10341", 'reported_seq': 12496, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:24:36.575941+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:24:36.575941+0000', 'last_peered': '2024-09-15T21:24:36.575941+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:24:36.575941+0000', 'last_undegraded': '2024-09-15T21:24:36.575941+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41152512, 'num_objects': 10047, 'num_object_clones': 0, 'num_object_copies': 20094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3073, 'num_objects_unfound': 0, 'num_objects_dirty': 10047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6825, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1009, 'num_bytes_recovered': 4132864, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6974}, {'shards': '0,2', 'objects': 3073}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:40.858 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:24:40.858 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10561", 'reported_seq': 10700, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:36.502525+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:24:36.502525+0000', 'last_peered': '2024-09-15T21:24:36.502525+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:24:36.502525+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42210246, 'num_objects': 10307, 'num_object_clones': 0, 'num_object_copies': 20614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10307, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7265, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6160}, {'shards': '0,2', 'objects': 4025}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:40.858 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:41.550 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:41.550 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/325624 objects degraded (12.660%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":333,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":162812,"data_bytes":666829577,"bytes_used":2228269056,"bytes_avail":287682023424,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":325624,"degraded_ratio":0.12660307594034836,"misplaced_objects":3073,"misplaced_total":325624,"misplaced_ratio":0.0094372650664570172,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":55622,"recovering_keys_per_sec":0,"num_objects_recovered":163,"num_bytes_recovered":667648,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":664828,"read_op_per_sec":0,"write_op_per_sec":369},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:41.578 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:41.578 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:42.184 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:42.187 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:42.221 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-15T21:24:40.761064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666829577,"num_objects":162812,"num_object_clones":0,"num_object_copies":325624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3073,"num_objects_unfound":0,"num_objects_dirty":162812,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":127647,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5175,"num_bytes_recovered":21184531,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160955,"ondisk_log_size":160955,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2176044,"kb_used_data":1331324,"kb_used_omap":12,"kb_used_meta":844659,"kb_avail":280939476,"statfs":{"total":289910292480,"available":287682023424,"internally_reserved":0,"allocated":1363275776,"data_stored":1362405010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":864930819},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,8,2,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-2199528,"num_objects":-536,"num_object_clones":0,"num_object_copies":-1072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-163,"num_objects_unfound":0,"num_objects_dirty":-536,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4434,"num_write_kb":7793,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":667648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003135"},"pg_stats":[{"pgid":"2.f","version":"45'10697","reported_seq":10773,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.605141+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:36.605141+0000","last_peered":"2024-09-15T21:24:36.605141+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:36.605141+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42823680,"num_objects":10455,"num_object_clones":0,"num_object_copies":20910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10697,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6323},{"shards":"0,2","objects":4011}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10531","reported_seq":10636,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:38.553936+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:38.553936+0000","last_peered":"2024-09-15T21:24:38.553936+0000","last_clean":"2024-09-15T21:24:38.553936+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:38.553936+0000","last_undegraded":"2024-09-15T21:24:38.553936+0000","last_fullsized":"2024-09-15T21:24:38.553936+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41546694,"num_objects":10145,"num_object_clones":0,"num_object_copies":20290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10145,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8812,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10722","reported_seq":10848,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.543657+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:38.543657+0000","last_peered":"2024-09-15T21:24:38.543657+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:38.543657+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42528792,"num_objects":10384,"num_object_clones":0,"num_object_copies":20768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10384,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8822,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4100},{"shards":"2","objects":6117}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10347","reported_seq":10417,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:38.629595+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:38.629595+0000","last_peered":"2024-09-15T21:24:38.629595+0000","last_clean":"2024-09-15T21:24:38.629595+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:38.629595+0000","last_undegraded":"2024-09-15T21:24:38.629595+0000","last_fullsized":"2024-09-15T21:24:38.629595+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40824832,"num_objects":9967,"num_object_clones":0,"num_object_copies":19934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10347,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10393","reported_seq":10526,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:36.367070+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:36.367070+0000","last_peered":"2024-09-15T21:24:36.367070+0000","last_clean":"2024-09-15T21:24:36.367070+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:36.367070+0000","last_undegraded":"2024-09-15T21:24:36.367070+0000","last_fullsized":"2024-09-15T21:24:36.367070+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41419235,"num_objects":10113,"num_object_clones":0,"num_object_copies":20226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6848,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10464","reported_seq":10600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.543711+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:36.543711+0000","last_peered":"2024-09-15T21:24:36.543711+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:36.543711+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41660899,"num_objects":10172,"num_object_clones":0,"num_object_copies":20344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10172,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7140,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6096},{"shards":"0,2","objects":3932}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10474","reported_seq":10610,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.429357+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:36.429357+0000","last_peered":"2024-09-15T21:24:36.429357+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:36.429357+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41763782,"num_objects":10198,"num_object_clones":0,"num_object_copies":20396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6785,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6037},{"shards":"0,2","objects":4025}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10578","reported_seq":10706,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:36.574288+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:36.574288+0000","last_peered":"2024-09-15T21:24:36.574288+0000","last_clean":"2024-09-15T21:24:36.574288+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:36.574288+0000","last_undegraded":"2024-09-15T21:24:36.574288+0000","last_fullsized":"2024-09-15T21:24:36.574288+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42147840,"num_objects":10290,"num_object_clones":0,"num_object_copies":20580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7350,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10454","reported_seq":10581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.586704+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:38.586704+0000","last_peered":"2024-09-15T21:24:38.586704+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:38.586704+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41202147,"num_objects":10060,"num_object_clones":0,"num_object_copies":20120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10060,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8386,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3891},{"shards":"2","objects":5973}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10386","reported_seq":10516,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.567544+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:36.567544+0000","last_peered":"2024-09-15T21:24:36.567544+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:36.567544+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41353216,"num_objects":10096,"num_object_clones":0,"num_object_copies":20192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7059,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6076},{"shards":"0,2","objects":3879}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10513","reported_seq":10632,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:38.591259+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:38.591259+0000","last_peered":"2024-09-15T21:24:38.591259+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:38.591259+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41492963,"num_objects":10131,"num_object_clones":0,"num_object_copies":20262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10131,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8372,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3901},{"shards":"2","objects":6041}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10480","reported_seq":18938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:38.634033+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:38.634033+0000","last_peered":"2024-09-15T21:24:38.634033+0000","last_clean":"2024-09-15T21:24:38.634033+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:38.634033+0000","last_undegraded":"2024-09-15T21:24:38.634033+0000","last_fullsized":"2024-09-15T21:24:38.634033+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41287680,"num_objects":10080,"num_object_clones":0,"num_object_copies":20160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8173,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10483","reported_seq":10615,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.624472+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:36.624472+0000","last_peered":"2024-09-15T21:24:36.624472+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:36.624472+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41791488,"num_objects":10203,"num_object_clones":0,"num_object_copies":20406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7427,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6109},{"shards":"0,2","objects":3955}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10428","reported_seq":10559,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.583667+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:36.583667+0000","last_peered":"2024-09-15T21:24:36.583667+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:36.583667+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41623552,"num_objects":10162,"num_object_clones":0,"num_object_copies":20324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7337,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6023},{"shards":"0,2","objects":4010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10341","reported_seq":12496,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:36.575941+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:36.575941+0000","last_peered":"2024-09-15T21:24:36.575941+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:36.575941+0000","last_undegraded":"2024-09-15T21:24:36.575941+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41152512,"num_objects":10047,"num_object_clones":0,"num_object_copies":20094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3073,"num_objects_unfound":0,"num_objects_dirty":10047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6825,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1009,"num_bytes_recovered":4132864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6974},{"shards":"0,2","objects":3073}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10561","reported_seq":10700,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:36.502525+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:36.502525+0000","last_peered":"2024-09-15T21:24:36.502525+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:36.502525+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42210246,"num_objects":10307,"num_object_clones":0,"num_object_copies":20614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10307,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7265,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6160},{"shards":"0,2","objects":4025}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":666829558,"num_objects":162810,"num_object_clones":0,"num_object_copies":325620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":3073,"num_objects_unfound":0,"num_objects_dirty":162810,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":127645,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5174,"num_bytes_recovered":21184512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1361223680,"data_stored":1361067403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160952,"ondisk_log_size":160952,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718540,"kb_used_data":456380,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93653300,"statfs":{"total":96636764160,"available":95900979200,"internally_reserved":0,"allocated":467333120,"data_stored":467057473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":268431017},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,6,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":890628,"kb_used_data":556340,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93481212,"statfs":{"total":96636764160,"available":95724761088,"internally_reserved":0,"allocated":569692160,"data_stored":569416508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":566876,"kb_used_data":318604,"kb_used_omap":4,"kb_used_meta":248251,"kb_avail":93804964,"statfs":{"total":96636764160,"available":96056283136,"internally_reserved":0,"allocated":326250496,"data_stored":325931029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":254209711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568827904,"data_stored":568791774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":325791744,"data_stored":325711799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":466604032,"data_stored":466563830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:45.224 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:45.799 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:45.800 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:45.820 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-15T21:24:44.762042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657507081,"num_objects":160536,"num_object_clones":0,"num_object_copies":321072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2984,"num_objects_unfound":0,"num_objects_dirty":160536,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":129923,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5264,"num_bytes_recovered":21549075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160731,"ondisk_log_size":160731,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2198544,"kb_used_data":1319136,"kb_used_omap":12,"kb_used_meta":879347,"kb_avail":280916976,"statfs":{"total":289910292480,"available":287658983424,"internally_reserved":0,"allocated":1350795264,"data_stored":1349924498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":900451331},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,5,5,4,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17174504,"num_objects":-4192,"num_object_clones":0,"num_object_copies":-8384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-175,"num_objects_unfound":0,"num_objects_dirty":-4192,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5330,"num_write_kb":2273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":716800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003027"},"pg_stats":[{"pgid":"2.f","version":"45'10856","reported_seq":10932,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.637263+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:41.637263+0000","last_peered":"2024-09-15T21:24:41.637263+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:41.637263+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42172416,"num_objects":10296,"num_object_clones":0,"num_object_copies":20592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10856,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6164},{"shards":"0,2","objects":3852}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10678","reported_seq":10783,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:43.584531+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:43.584531+0000","last_peered":"2024-09-15T21:24:43.584531+0000","last_clean":"2024-09-15T21:24:43.584531+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:43.584531+0000","last_undegraded":"2024-09-15T21:24:43.584531+0000","last_fullsized":"2024-09-15T21:24:43.584531+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40944582,"num_objects":9998,"num_object_clones":0,"num_object_copies":19996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9998,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8959,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10851","reported_seq":10977,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:43.683220+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:43.683220+0000","last_peered":"2024-09-15T21:24:43.683220+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:43.683220+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42000408,"num_objects":10255,"num_object_clones":0,"num_object_copies":20510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8951,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3971},{"shards":"2","objects":5988}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10487","reported_seq":10557,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:43.645245+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:43.645245+0000","last_peered":"2024-09-15T21:24:43.645245+0000","last_clean":"2024-09-15T21:24:43.645245+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:43.645245+0000","last_undegraded":"2024-09-15T21:24:43.645245+0000","last_fullsized":"2024-09-15T21:24:43.645245+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40251392,"num_objects":9827,"num_object_clones":0,"num_object_copies":19654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10487,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10521","reported_seq":10654,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:41.641538+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:41.641538+0000","last_peered":"2024-09-15T21:24:41.641538+0000","last_clean":"2024-09-15T21:24:41.641538+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:41.641538+0000","last_undegraded":"2024-09-15T21:24:41.641538+0000","last_fullsized":"2024-09-15T21:24:41.641538+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894947,"num_objects":9985,"num_object_clones":0,"num_object_copies":19970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6976,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10633","reported_seq":10767,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.616752+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:41.616752+0000","last_peered":"2024-09-15T21:24:41.616752+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:41.616752+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40968675,"num_objects":10003,"num_object_clones":0,"num_object_copies":20006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10003,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7309,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5927},{"shards":"0,2","objects":3763}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10615","reported_seq":10746,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.545110+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:41.545110+0000","last_peered":"2024-09-15T21:24:41.545110+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:41.545110+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41186246,"num_objects":10057,"num_object_clones":0,"num_object_copies":20114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6926,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5896},{"shards":"0,2","objects":3884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10721","reported_seq":10850,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:41.631675+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:41.631675+0000","last_peered":"2024-09-15T21:24:41.631675+0000","last_clean":"2024-09-15T21:24:41.631675+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:41.631675+0000","last_undegraded":"2024-09-15T21:24:41.631675+0000","last_fullsized":"2024-09-15T21:24:41.631675+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41562112,"num_objects":10147,"num_object_clones":0,"num_object_copies":20294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7493,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10597","reported_seq":10724,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:43.658140+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:43.658140+0000","last_peered":"2024-09-15T21:24:43.658140+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:43.658140+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40616419,"num_objects":9917,"num_object_clones":0,"num_object_copies":19834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8529,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3748},{"shards":"2","objects":5830}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10526","reported_seq":10657,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.566652+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:41.566652+0000","last_peered":"2024-09-15T21:24:41.566652+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:41.566652+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40779776,"num_objects":9956,"num_object_clones":0,"num_object_copies":19912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7199,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5936},{"shards":"0,2","objects":3739}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10659","reported_seq":10778,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:43.614042+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:43.614042+0000","last_peered":"2024-09-15T21:24:43.614042+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:43.614042+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894947,"num_objects":9985,"num_object_clones":0,"num_object_copies":19970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9985,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8518,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3755},{"shards":"2","objects":5895}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10619","reported_seq":19077,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:43.604385+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:43.604385+0000","last_peered":"2024-09-15T21:24:43.604385+0000","last_clean":"2024-09-15T21:24:43.604385+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:43.604385+0000","last_undegraded":"2024-09-15T21:24:43.604385+0000","last_fullsized":"2024-09-15T21:24:43.604385+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40718336,"num_objects":9941,"num_object_clones":0,"num_object_copies":19882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8312,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10620","reported_seq":10754,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.640081+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:41.640081+0000","last_peered":"2024-09-15T21:24:41.640081+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:41.640081+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41230336,"num_objects":10066,"num_object_clones":0,"num_object_copies":20132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7564,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5972},{"shards":"0,2","objects":3818}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10551","reported_seq":10684,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.554098+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:41.554098+0000","last_peered":"2024-09-15T21:24:41.554098+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:41.554098+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41119744,"num_objects":10039,"num_object_clones":0,"num_object_copies":20078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7460,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5900},{"shards":"0,2","objects":3887}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10487","reported_seq":12819,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:41.682370+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:41.682370+0000","last_peered":"2024-09-15T21:24:41.682370+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:41.682370+0000","last_undegraded":"2024-09-15T21:24:41.682370+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40554496,"num_objects":9901,"num_object_clones":0,"num_object_copies":19802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2984,"num_objects_unfound":0,"num_objects_dirty":9901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6971,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1098,"num_bytes_recovered":4497408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6917},{"shards":"0,2","objects":2984}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10707","reported_seq":10846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.625746+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:41.625746+0000","last_peered":"2024-09-15T21:24:41.625746+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:41.625746+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41612230,"num_objects":10161,"num_object_clones":0,"num_object_copies":20322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10161,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7411,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6014},{"shards":"0,2","objects":3879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":657507062,"num_objects":160534,"num_object_clones":0,"num_object_copies":321068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2984,"num_objects_unfound":0,"num_objects_dirty":160534,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":129921,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5263,"num_bytes_recovered":21549056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1348714496,"data_stored":1348558219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160728,"ondisk_log_size":160728,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736208,"kb_used_data":451840,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93635632,"statfs":{"total":96636764160,"available":95882887168,"internally_reserved":0,"allocated":462684160,"data_stored":462408513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":291172009},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":887888,"kb_used_data":549440,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93483952,"statfs":{"total":96636764160,"available":95727566848,"internally_reserved":0,"allocated":562626560,"data_stored":562350908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574448,"kb_used_data":317856,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93797392,"statfs":{"total":96636764160,"available":96048529408,"internally_reserved":0,"allocated":325484544,"data_stored":325165077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":262729391},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561758208,"data_stored":561722078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324993024,"data_stored":324913079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":461963264,"data_stored":461923062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:45.822 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:46.398 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:46.400 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:46.427 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":206,"stamp":"2024-09-15T21:24:44.762042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657507081,"num_objects":160536,"num_object_clones":0,"num_object_copies":321072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2984,"num_objects_unfound":0,"num_objects_dirty":160536,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":129923,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5264,"num_bytes_recovered":21549075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160731,"ondisk_log_size":160731,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2198544,"kb_used_data":1319136,"kb_used_omap":12,"kb_used_meta":879347,"kb_avail":280916976,"statfs":{"total":289910292480,"available":287658983424,"internally_reserved":0,"allocated":1350795264,"data_stored":1349924498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":900451331},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,5,5,4,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17174504,"num_objects":-4192,"num_object_clones":0,"num_object_copies":-8384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-175,"num_objects_unfound":0,"num_objects_dirty":-4192,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5330,"num_write_kb":2273,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":716800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003027"},"pg_stats":[{"pgid":"2.f","version":"45'10856","reported_seq":10932,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.637263+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:41.637263+0000","last_peered":"2024-09-15T21:24:41.637263+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:41.637263+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42172416,"num_objects":10296,"num_object_clones":0,"num_object_copies":20592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10856,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6164},{"shards":"0,2","objects":3852}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10678","reported_seq":10783,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:43.584531+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:43.584531+0000","last_peered":"2024-09-15T21:24:43.584531+0000","last_clean":"2024-09-15T21:24:43.584531+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:43.584531+0000","last_undegraded":"2024-09-15T21:24:43.584531+0000","last_fullsized":"2024-09-15T21:24:43.584531+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40944582,"num_objects":9998,"num_object_clones":0,"num_object_copies":19996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9998,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8959,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10851","reported_seq":10977,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:43.683220+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:43.683220+0000","last_peered":"2024-09-15T21:24:43.683220+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:43.683220+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42000408,"num_objects":10255,"num_object_clones":0,"num_object_copies":20510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8951,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3971},{"shards":"2","objects":5988}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10487","reported_seq":10557,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:43.645245+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:43.645245+0000","last_peered":"2024-09-15T21:24:43.645245+0000","last_clean":"2024-09-15T21:24:43.645245+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:43.645245+0000","last_undegraded":"2024-09-15T21:24:43.645245+0000","last_fullsized":"2024-09-15T21:24:43.645245+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40251392,"num_objects":9827,"num_object_clones":0,"num_object_copies":19654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10487,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10521","reported_seq":10654,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:41.641538+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:41.641538+0000","last_peered":"2024-09-15T21:24:41.641538+0000","last_clean":"2024-09-15T21:24:41.641538+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:41.641538+0000","last_undegraded":"2024-09-15T21:24:41.641538+0000","last_fullsized":"2024-09-15T21:24:41.641538+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894947,"num_objects":9985,"num_object_clones":0,"num_object_copies":19970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6976,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10633","reported_seq":10767,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.616752+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:41.616752+0000","last_peered":"2024-09-15T21:24:41.616752+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:41.616752+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40968675,"num_objects":10003,"num_object_clones":0,"num_object_copies":20006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10003,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7309,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5927},{"shards":"0,2","objects":3763}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10615","reported_seq":10746,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.545110+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:41.545110+0000","last_peered":"2024-09-15T21:24:41.545110+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:41.545110+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41186246,"num_objects":10057,"num_object_clones":0,"num_object_copies":20114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6926,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5896},{"shards":"0,2","objects":3884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10721","reported_seq":10850,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:41.631675+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:41.631675+0000","last_peered":"2024-09-15T21:24:41.631675+0000","last_clean":"2024-09-15T21:24:41.631675+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:41.631675+0000","last_undegraded":"2024-09-15T21:24:41.631675+0000","last_fullsized":"2024-09-15T21:24:41.631675+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41562112,"num_objects":10147,"num_object_clones":0,"num_object_copies":20294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7493,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10597","reported_seq":10724,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:43.658140+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:43.658140+0000","last_peered":"2024-09-15T21:24:43.658140+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:43.658140+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40616419,"num_objects":9917,"num_object_clones":0,"num_object_copies":19834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8529,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3748},{"shards":"2","objects":5830}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10526","reported_seq":10657,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.566652+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:41.566652+0000","last_peered":"2024-09-15T21:24:41.566652+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:41.566652+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40779776,"num_objects":9956,"num_object_clones":0,"num_object_copies":19912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7199,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5936},{"shards":"0,2","objects":3739}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10659","reported_seq":10778,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:43.614042+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:43.614042+0000","last_peered":"2024-09-15T21:24:43.614042+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:43.614042+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894947,"num_objects":9985,"num_object_clones":0,"num_object_copies":19970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9985,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8518,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3755},{"shards":"2","objects":5895}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10619","reported_seq":19077,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:43.604385+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:43.604385+0000","last_peered":"2024-09-15T21:24:43.604385+0000","last_clean":"2024-09-15T21:24:43.604385+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:43.604385+0000","last_undegraded":"2024-09-15T21:24:43.604385+0000","last_fullsized":"2024-09-15T21:24:43.604385+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40718336,"num_objects":9941,"num_object_clones":0,"num_object_copies":19882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8312,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10620","reported_seq":10754,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.640081+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:41.640081+0000","last_peered":"2024-09-15T21:24:41.640081+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:41.640081+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41230336,"num_objects":10066,"num_object_clones":0,"num_object_copies":20132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7564,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5972},{"shards":"0,2","objects":3818}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10551","reported_seq":10684,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.554098+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:41.554098+0000","last_peered":"2024-09-15T21:24:41.554098+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:41.554098+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41119744,"num_objects":10039,"num_object_clones":0,"num_object_copies":20078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7460,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5900},{"shards":"0,2","objects":3887}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10487","reported_seq":12819,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:41.682370+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:41.682370+0000","last_peered":"2024-09-15T21:24:41.682370+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:41.682370+0000","last_undegraded":"2024-09-15T21:24:41.682370+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40554496,"num_objects":9901,"num_object_clones":0,"num_object_copies":19802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2984,"num_objects_unfound":0,"num_objects_dirty":9901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6971,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1098,"num_bytes_recovered":4497408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6917},{"shards":"0,2","objects":2984}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10707","reported_seq":10846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.625746+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:41.625746+0000","last_peered":"2024-09-15T21:24:41.625746+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:41.625746+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41612230,"num_objects":10161,"num_object_clones":0,"num_object_copies":20322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10161,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7411,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6014},{"shards":"0,2","objects":3879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":657507062,"num_objects":160534,"num_object_clones":0,"num_object_copies":321068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2984,"num_objects_unfound":0,"num_objects_dirty":160534,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":129921,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5263,"num_bytes_recovered":21549056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1348714496,"data_stored":1348558219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160728,"ondisk_log_size":160728,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736208,"kb_used_data":451840,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93635632,"statfs":{"total":96636764160,"available":95882887168,"internally_reserved":0,"allocated":462684160,"data_stored":462408513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":291172009},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":887888,"kb_used_data":549440,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93483952,"statfs":{"total":96636764160,"available":95727566848,"internally_reserved":0,"allocated":562626560,"data_stored":562350908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574448,"kb_used_data":317856,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93797392,"statfs":{"total":96636764160,"available":96048529408,"internally_reserved":0,"allocated":325484544,"data_stored":325165077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":262729391},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561758208,"data_stored":561722078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324993024,"data_stored":324913079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":461963264,"data_stored":461923062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:46.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:46.429 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10856", 'reported_seq': 10932, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:41.637263+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:24:41.637263+0000', 'last_peered': '2024-09-15T21:24:41.637263+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:24:41.637263+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42172416, 'num_objects': 10296, 'num_object_clones': 0, 'num_object_copies': 20592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10296, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10856, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6164}, {'shards': '0,2', 'objects': 3852}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:46.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:24:46.429 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'10851", 'reported_seq': 10977, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:43.683220+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:24:43.683220+0000', 'last_peered': '2024-09-15T21:24:43.683220+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:24:43.683220+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 42000408, 'num_objects': 10255, 'num_object_clones': 0, 'num_object_copies': 20510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10255, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8951, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3971}, {'shards': '2', 'objects': 5988}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:46.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:46.429 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'10633", 'reported_seq': 10767, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:41.616752+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:24:41.616752+0000', 'last_peered': '2024-09-15T21:24:41.616752+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:24:41.616752+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40968675, 'num_objects': 10003, 'num_object_clones': 0, 'num_object_copies': 20006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10003, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7309, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5927}, {'shards': '0,2', 'objects': 3763}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:46.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:46.429 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10615", 'reported_seq': 10746, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:41.545110+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:24:41.545110+0000', 'last_peered': '2024-09-15T21:24:41.545110+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:24:41.545110+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41186246, 'num_objects': 10057, 'num_object_clones': 0, 'num_object_copies': 20114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10057, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6926, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5896}, {'shards': '0,2', 'objects': 3884}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:46.429 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:24:46.429 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'10597", 'reported_seq': 10724, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:43.658140+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:24:43.658140+0000', 'last_peered': '2024-09-15T21:24:43.658140+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:24:43.658140+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40616419, 'num_objects': 9917, 'num_object_clones': 0, 'num_object_copies': 19834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9917, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8529, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3748}, {'shards': '2', 'objects': 5830}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:46.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:46.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'10526", 'reported_seq': 10657, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:41.566652+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:24:41.566652+0000', 'last_peered': '2024-09-15T21:24:41.566652+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:24:41.566652+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40779776, 'num_objects': 9956, 'num_object_clones': 0, 'num_object_copies': 19912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9956, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7199, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5936}, {'shards': '0,2', 'objects': 3739}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:46.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:46.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'10659", 'reported_seq': 10778, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:43.614042+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:24:43.614042+0000', 'last_peered': '2024-09-15T21:24:43.614042+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:24:43.614042+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40894947, 'num_objects': 9985, 'num_object_clones': 0, 'num_object_copies': 19970, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9985, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8518, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3755}, {'shards': '2', 'objects': 5895}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:46.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:46.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'10620", 'reported_seq': 10754, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:41.640081+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:24:41.640081+0000', 'last_peered': '2024-09-15T21:24:41.640081+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:24:41.640081+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41230336, 'num_objects': 10066, 'num_object_clones': 0, 'num_object_copies': 20132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10066, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7564, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5972}, {'shards': '0,2', 'objects': 3818}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:46.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:46.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'10551", 'reported_seq': 10684, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:41.554098+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:24:41.554098+0000', 'last_peered': '2024-09-15T21:24:41.554098+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:24:41.554098+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41119744, 'num_objects': 10039, 'num_object_clones': 0, 'num_object_copies': 20078, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10039, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7460, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5900}, {'shards': '0,2', 'objects': 3887}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:46.430 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:46.430 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'10487", 'reported_seq': 12819, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:24:41.682370+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:24:41.682370+0000', 'last_peered': '2024-09-15T21:24:41.682370+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:24:41.682370+0000', 'last_undegraded': '2024-09-15T21:24:41.682370+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40554496, 'num_objects': 9901, 'num_object_clones': 0, 'num_object_copies': 19802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2984, 'num_objects_unfound': 0, 'num_objects_dirty': 9901, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6971, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1098, 'num_bytes_recovered': 4497408, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6917}, {'shards': '0,2', 'objects': 2984}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:46.431 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:24:46.431 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10707", 'reported_seq': 10846, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:41.625746+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:24:41.625746+0000', 'last_peered': '2024-09-15T21:24:41.625746+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:24:41.625746+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41612230, 'num_objects': 10161, 'num_object_clones': 0, 'num_object_copies': 20322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10161, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7411, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6014}, {'shards': '0,2', 'objects': 3879}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:46.431 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:47.122 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:47.123 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/321072 objects degraded (12.840%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":338,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":160536,"data_bytes":657507081,"bytes_used":2251309056,"bytes_avail":287658983424,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":321072,"degraded_ratio":0.12839799172771216,"misplaced_objects":2984,"misplaced_total":321072,"misplaced_ratio":0.0092938655504061392,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":59718,"recovering_keys_per_sec":0,"num_objects_recovered":175,"num_bytes_recovered":716800,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":193913,"read_op_per_sec":0,"write_op_per_sec":444},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:47.145 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:47.146 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:47.690 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:47.690 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:47.715 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-15T21:24:46.762518+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657507081,"num_objects":160536,"num_object_clones":0,"num_object_copies":321072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2984,"num_objects_unfound":0,"num_objects_dirty":160536,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":129923,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5264,"num_bytes_recovered":21549075,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160731,"ondisk_log_size":160731,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2198544,"kb_used_data":1319136,"kb_used_omap":12,"kb_used_meta":879347,"kb_avail":280916976,"statfs":{"total":289910292480,"available":287658983424,"internally_reserved":0,"allocated":1350795264,"data_stored":1349924498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":900451331},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,5,5,4,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17907712,"num_objects":-4372,"num_object_clones":0,"num_object_copies":-8744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-175,"num_objects_unfound":0,"num_objects_dirty":-4372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4532,"num_write_kb":320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":716800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002859"},"pg_stats":[{"pgid":"2.f","version":"45'10856","reported_seq":10932,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.637263+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:41.637263+0000","last_peered":"2024-09-15T21:24:41.637263+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:41.637263+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42172416,"num_objects":10296,"num_object_clones":0,"num_object_copies":20592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10296,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10856,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6164},{"shards":"0,2","objects":3852}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10678","reported_seq":10783,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:43.584531+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:43.584531+0000","last_peered":"2024-09-15T21:24:43.584531+0000","last_clean":"2024-09-15T21:24:43.584531+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:43.584531+0000","last_undegraded":"2024-09-15T21:24:43.584531+0000","last_fullsized":"2024-09-15T21:24:43.584531+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40944582,"num_objects":9998,"num_object_clones":0,"num_object_copies":19996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9998,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8959,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10851","reported_seq":10977,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:43.683220+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:43.683220+0000","last_peered":"2024-09-15T21:24:43.683220+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:43.683220+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42000408,"num_objects":10255,"num_object_clones":0,"num_object_copies":20510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10255,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8951,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3971},{"shards":"2","objects":5988}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10487","reported_seq":10557,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:43.645245+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:43.645245+0000","last_peered":"2024-09-15T21:24:43.645245+0000","last_clean":"2024-09-15T21:24:43.645245+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:43.645245+0000","last_undegraded":"2024-09-15T21:24:43.645245+0000","last_fullsized":"2024-09-15T21:24:43.645245+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40251392,"num_objects":9827,"num_object_clones":0,"num_object_copies":19654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10487,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10521","reported_seq":10654,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:41.641538+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:41.641538+0000","last_peered":"2024-09-15T21:24:41.641538+0000","last_clean":"2024-09-15T21:24:41.641538+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:41.641538+0000","last_undegraded":"2024-09-15T21:24:41.641538+0000","last_fullsized":"2024-09-15T21:24:41.641538+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894947,"num_objects":9985,"num_object_clones":0,"num_object_copies":19970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6976,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10633","reported_seq":10767,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.616752+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:41.616752+0000","last_peered":"2024-09-15T21:24:41.616752+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:41.616752+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40968675,"num_objects":10003,"num_object_clones":0,"num_object_copies":20006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10003,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7309,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5927},{"shards":"0,2","objects":3763}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10615","reported_seq":10746,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.545110+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:41.545110+0000","last_peered":"2024-09-15T21:24:41.545110+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:41.545110+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41186246,"num_objects":10057,"num_object_clones":0,"num_object_copies":20114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6926,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5896},{"shards":"0,2","objects":3884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10721","reported_seq":10850,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:41.631675+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:41.631675+0000","last_peered":"2024-09-15T21:24:41.631675+0000","last_clean":"2024-09-15T21:24:41.631675+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:41.631675+0000","last_undegraded":"2024-09-15T21:24:41.631675+0000","last_fullsized":"2024-09-15T21:24:41.631675+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41562112,"num_objects":10147,"num_object_clones":0,"num_object_copies":20294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7493,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10597","reported_seq":10724,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:43.658140+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:43.658140+0000","last_peered":"2024-09-15T21:24:43.658140+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:43.658140+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40616419,"num_objects":9917,"num_object_clones":0,"num_object_copies":19834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9917,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8529,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3748},{"shards":"2","objects":5830}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10526","reported_seq":10657,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.566652+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:41.566652+0000","last_peered":"2024-09-15T21:24:41.566652+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:41.566652+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40779776,"num_objects":9956,"num_object_clones":0,"num_object_copies":19912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7199,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5936},{"shards":"0,2","objects":3739}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10659","reported_seq":10778,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:43.614042+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:43.614042+0000","last_peered":"2024-09-15T21:24:43.614042+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:43.614042+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40894947,"num_objects":9985,"num_object_clones":0,"num_object_copies":19970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9985,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8518,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3755},{"shards":"2","objects":5895}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10619","reported_seq":19077,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:43.604385+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:43.604385+0000","last_peered":"2024-09-15T21:24:43.604385+0000","last_clean":"2024-09-15T21:24:43.604385+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:43.604385+0000","last_undegraded":"2024-09-15T21:24:43.604385+0000","last_fullsized":"2024-09-15T21:24:43.604385+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40718336,"num_objects":9941,"num_object_clones":0,"num_object_copies":19882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8312,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10620","reported_seq":10754,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.640081+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:41.640081+0000","last_peered":"2024-09-15T21:24:41.640081+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:41.640081+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41230336,"num_objects":10066,"num_object_clones":0,"num_object_copies":20132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7564,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5972},{"shards":"0,2","objects":3818}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10551","reported_seq":10684,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.554098+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:41.554098+0000","last_peered":"2024-09-15T21:24:41.554098+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:41.554098+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41119744,"num_objects":10039,"num_object_clones":0,"num_object_copies":20078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7460,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5900},{"shards":"0,2","objects":3887}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10487","reported_seq":12819,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:41.682370+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:41.682370+0000","last_peered":"2024-09-15T21:24:41.682370+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:41.682370+0000","last_undegraded":"2024-09-15T21:24:41.682370+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40554496,"num_objects":9901,"num_object_clones":0,"num_object_copies":19802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2984,"num_objects_unfound":0,"num_objects_dirty":9901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6971,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1098,"num_bytes_recovered":4497408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6917},{"shards":"0,2","objects":2984}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10707","reported_seq":10846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:41.625746+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:41.625746+0000","last_peered":"2024-09-15T21:24:41.625746+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:41.625746+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41612230,"num_objects":10161,"num_object_clones":0,"num_object_copies":20322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10161,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7411,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6014},{"shards":"0,2","objects":3879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":657507062,"num_objects":160534,"num_object_clones":0,"num_object_copies":321068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2984,"num_objects_unfound":0,"num_objects_dirty":160534,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":129921,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5263,"num_bytes_recovered":21549056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1348714496,"data_stored":1348558219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160728,"ondisk_log_size":160728,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":736208,"kb_used_data":451840,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93635632,"statfs":{"total":96636764160,"available":95882887168,"internally_reserved":0,"allocated":462684160,"data_stored":462408513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":291172009},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":887888,"kb_used_data":549440,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93483952,"statfs":{"total":96636764160,"available":95727566848,"internally_reserved":0,"allocated":562626560,"data_stored":562350908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":574448,"kb_used_data":317856,"kb_used_omap":4,"kb_used_meta":256571,"kb_avail":93797392,"statfs":{"total":96636764160,"available":96048529408,"internally_reserved":0,"allocated":325484544,"data_stored":325165077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":262729391},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":561758208,"data_stored":561722078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324993024,"data_stored":324913079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":461963264,"data_stored":461923062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:50.718 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:51.271 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:51.272 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:51.302 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-15T21:24:50.763482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648168201,"num_objects":158256,"num_object_clones":0,"num_object_copies":316512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2902,"num_objects_unfound":0,"num_objects_dirty":158256,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":132203,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5346,"num_bytes_recovered":21884947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2200300,"kb_used_data":1307000,"kb_used_omap":12,"kb_used_meta":893235,"kb_avail":280915220,"statfs":{"total":289910292480,"available":287657185280,"internally_reserved":0,"allocated":1338368000,"data_stored":1337497234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":914672643},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,6,11,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18661376,"num_objects":-4556,"num_object_clones":0,"num_object_copies":-9112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-171,"num_objects_unfound":0,"num_objects_dirty":-4556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4556,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":700416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002902"},"pg_stats":[{"pgid":"2.f","version":"45'10987","reported_seq":11063,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.762978+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:46.762978+0000","last_peered":"2024-09-15T21:24:46.762978+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:46.762978+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41635840,"num_objects":10165,"num_object_clones":0,"num_object_copies":20330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10987,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6033},{"shards":"0,2","objects":3721}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10815","reported_seq":10920,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:48.673247+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:48.673247+0000","last_peered":"2024-09-15T21:24:48.673247+0000","last_clean":"2024-09-15T21:24:48.673247+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:48.673247+0000","last_undegraded":"2024-09-15T21:24:48.673247+0000","last_fullsized":"2024-09-15T21:24:48.673247+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40383430,"num_objects":9861,"num_object_clones":0,"num_object_copies":19722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9861,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9096,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10994","reported_seq":11120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:48.582109+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:48.582109+0000","last_peered":"2024-09-15T21:24:48.582109+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:48.582109+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41414680,"num_objects":10112,"num_object_clones":0,"num_object_copies":20224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9094,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5845}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10637","reported_seq":10708,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:48.678834+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:48.678834+0000","last_peered":"2024-09-15T21:24:48.678834+0000","last_clean":"2024-09-15T21:24:48.678834+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:48.678834+0000","last_undegraded":"2024-09-15T21:24:48.678834+0000","last_fullsized":"2024-09-15T21:24:48.678834+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39636992,"num_objects":9677,"num_object_clones":0,"num_object_copies":19354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10637,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10664","reported_seq":10797,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:46.664895+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:46.664895+0000","last_peered":"2024-09-15T21:24:46.664895+0000","last_clean":"2024-09-15T21:24:46.664895+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:46.664895+0000","last_undegraded":"2024-09-15T21:24:46.664895+0000","last_fullsized":"2024-09-15T21:24:46.664895+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40309219,"num_objects":9842,"num_object_clones":0,"num_object_copies":19684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7119,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10785","reported_seq":10918,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.750662+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:46.750662+0000","last_peered":"2024-09-15T21:24:46.750662+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:46.750662+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40346083,"num_objects":9851,"num_object_clones":0,"num_object_copies":19702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9851,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7461,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5775},{"shards":"0,2","objects":3611}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10730","reported_seq":10865,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.661436+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:46.661436+0000","last_peered":"2024-09-15T21:24:46.661436+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:46.661436+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40715206,"num_objects":9942,"num_object_clones":0,"num_object_copies":19884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7041,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5781},{"shards":"0,2","objects":3769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10852","reported_seq":10981,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:46.645222+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:46.645222+0000","last_peered":"2024-09-15T21:24:46.645222+0000","last_clean":"2024-09-15T21:24:46.645222+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:46.645222+0000","last_undegraded":"2024-09-15T21:24:46.645222+0000","last_fullsized":"2024-09-15T21:24:46.645222+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41025536,"num_objects":10016,"num_object_clones":0,"num_object_copies":20032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7624,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10743","reported_seq":10870,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:48.687288+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:48.687288+0000","last_peered":"2024-09-15T21:24:48.687288+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:48.687288+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40018403,"num_objects":9771,"num_object_clones":0,"num_object_copies":19542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9771,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8675,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3602},{"shards":"2","objects":5684}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10672","reported_seq":10803,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.693121+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:46.693121+0000","last_peered":"2024-09-15T21:24:46.693121+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:46.693121+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40181760,"num_objects":9810,"num_object_clones":0,"num_object_copies":19620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7345,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5790},{"shards":"0,2","objects":3593}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10804","reported_seq":10923,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:48.649129+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:48.649129+0000","last_peered":"2024-09-15T21:24:48.649129+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:48.649129+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40301027,"num_objects":9840,"num_object_clones":0,"num_object_copies":19680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9840,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8663,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5750}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10772","reported_seq":19231,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:48.621470+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:48.621470+0000","last_peered":"2024-09-15T21:24:48.621470+0000","last_clean":"2024-09-15T21:24:48.621470+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:48.621470+0000","last_undegraded":"2024-09-15T21:24:48.621470+0000","last_fullsized":"2024-09-15T21:24:48.621470+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40091648,"num_objects":9788,"num_object_clones":0,"num_object_copies":19576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8465,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10762","reported_seq":10896,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.734070+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:46.734070+0000","last_peered":"2024-09-15T21:24:46.734070+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:46.734070+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40648704,"num_objects":9924,"num_object_clones":0,"num_object_copies":19848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7706,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5830},{"shards":"0,2","objects":3676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10677","reported_seq":10810,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.752190+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:46.752190+0000","last_peered":"2024-09-15T21:24:46.752190+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:46.752190+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40603648,"num_objects":9913,"num_object_clones":0,"num_object_copies":19826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7586,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5774},{"shards":"0,2","objects":3761}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10642","reported_seq":13139,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:46.724695+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:46.724695+0000","last_peered":"2024-09-15T21:24:46.724695+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:46.724695+0000","last_undegraded":"2024-09-15T21:24:46.724695+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39919616,"num_objects":9746,"num_object_clones":0,"num_object_copies":19492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2902,"num_objects_unfound":0,"num_objects_dirty":9746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7126,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1180,"num_bytes_recovered":4833280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6844},{"shards":"0,2","objects":2902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10872","reported_seq":11011,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.712670+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:46.712670+0000","last_peered":"2024-09-15T21:24:46.712670+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:46.712670+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40936390,"num_objects":9996,"num_object_clones":0,"num_object_copies":19992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9996,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7576,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5849},{"shards":"0,2","objects":3714}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":648168182,"num_objects":158254,"num_object_clones":0,"num_object_copies":316508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2902,"num_objects_unfound":0,"num_objects_dirty":158254,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":132201,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5345,"num_bytes_recovered":21884928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1336201216,"data_stored":1336044939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735804,"kb_used_data":447276,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93636036,"statfs":{"total":96636764160,"available":95883300864,"internally_reserved":0,"allocated":458010624,"data_stored":457734977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":295431849},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182462,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":885364,"kb_used_data":542756,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93486476,"statfs":{"total":96636764160,"available":95730151424,"internally_reserved":0,"allocated":555782144,"data_stored":555506492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,6,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579132,"kb_used_data":316968,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93792708,"statfs":{"total":96636764160,"available":96043732992,"internally_reserved":0,"allocated":324575232,"data_stored":324255765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":268431023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554868736,"data_stored":554832606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324087808,"data_stored":324007863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":457244672,"data_stored":457204470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:51.303 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:51.895 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:51.901 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:51.923 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-15T21:24:50.763482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648168201,"num_objects":158256,"num_object_clones":0,"num_object_copies":316512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2902,"num_objects_unfound":0,"num_objects_dirty":158256,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":132203,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5346,"num_bytes_recovered":21884947,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2200300,"kb_used_data":1307000,"kb_used_omap":12,"kb_used_meta":893235,"kb_avail":280915220,"statfs":{"total":289910292480,"available":287657185280,"internally_reserved":0,"allocated":1338368000,"data_stored":1337497234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":914672643},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,6,11,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18661376,"num_objects":-4556,"num_object_clones":0,"num_object_copies":-9112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-171,"num_objects_unfound":0,"num_objects_dirty":-4556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4556,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":700416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002902"},"pg_stats":[{"pgid":"2.f","version":"45'10987","reported_seq":11063,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.762978+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:46.762978+0000","last_peered":"2024-09-15T21:24:46.762978+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:46.762978+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41635840,"num_objects":10165,"num_object_clones":0,"num_object_copies":20330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10987,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6033},{"shards":"0,2","objects":3721}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10815","reported_seq":10920,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:48.673247+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:48.673247+0000","last_peered":"2024-09-15T21:24:48.673247+0000","last_clean":"2024-09-15T21:24:48.673247+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:48.673247+0000","last_undegraded":"2024-09-15T21:24:48.673247+0000","last_fullsized":"2024-09-15T21:24:48.673247+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40383430,"num_objects":9861,"num_object_clones":0,"num_object_copies":19722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9861,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9096,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10994","reported_seq":11120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:48.582109+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:48.582109+0000","last_peered":"2024-09-15T21:24:48.582109+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:48.582109+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41414680,"num_objects":10112,"num_object_clones":0,"num_object_copies":20224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9094,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5845}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10637","reported_seq":10708,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:48.678834+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:48.678834+0000","last_peered":"2024-09-15T21:24:48.678834+0000","last_clean":"2024-09-15T21:24:48.678834+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:48.678834+0000","last_undegraded":"2024-09-15T21:24:48.678834+0000","last_fullsized":"2024-09-15T21:24:48.678834+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39636992,"num_objects":9677,"num_object_clones":0,"num_object_copies":19354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10637,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10664","reported_seq":10797,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:46.664895+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:46.664895+0000","last_peered":"2024-09-15T21:24:46.664895+0000","last_clean":"2024-09-15T21:24:46.664895+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:46.664895+0000","last_undegraded":"2024-09-15T21:24:46.664895+0000","last_fullsized":"2024-09-15T21:24:46.664895+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40309219,"num_objects":9842,"num_object_clones":0,"num_object_copies":19684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7119,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10785","reported_seq":10918,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.750662+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:46.750662+0000","last_peered":"2024-09-15T21:24:46.750662+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:46.750662+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40346083,"num_objects":9851,"num_object_clones":0,"num_object_copies":19702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9851,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7461,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5775},{"shards":"0,2","objects":3611}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10730","reported_seq":10865,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.661436+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:46.661436+0000","last_peered":"2024-09-15T21:24:46.661436+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:46.661436+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40715206,"num_objects":9942,"num_object_clones":0,"num_object_copies":19884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7041,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5781},{"shards":"0,2","objects":3769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10852","reported_seq":10981,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:46.645222+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:46.645222+0000","last_peered":"2024-09-15T21:24:46.645222+0000","last_clean":"2024-09-15T21:24:46.645222+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:46.645222+0000","last_undegraded":"2024-09-15T21:24:46.645222+0000","last_fullsized":"2024-09-15T21:24:46.645222+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41025536,"num_objects":10016,"num_object_clones":0,"num_object_copies":20032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7624,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10743","reported_seq":10870,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:48.687288+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:48.687288+0000","last_peered":"2024-09-15T21:24:48.687288+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:48.687288+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40018403,"num_objects":9771,"num_object_clones":0,"num_object_copies":19542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9771,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8675,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3602},{"shards":"2","objects":5684}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10672","reported_seq":10803,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.693121+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:46.693121+0000","last_peered":"2024-09-15T21:24:46.693121+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:46.693121+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40181760,"num_objects":9810,"num_object_clones":0,"num_object_copies":19620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7345,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5790},{"shards":"0,2","objects":3593}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10804","reported_seq":10923,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:48.649129+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:48.649129+0000","last_peered":"2024-09-15T21:24:48.649129+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:48.649129+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40301027,"num_objects":9840,"num_object_clones":0,"num_object_copies":19680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9840,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8663,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5750}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10772","reported_seq":19231,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:48.621470+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:48.621470+0000","last_peered":"2024-09-15T21:24:48.621470+0000","last_clean":"2024-09-15T21:24:48.621470+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:48.621470+0000","last_undegraded":"2024-09-15T21:24:48.621470+0000","last_fullsized":"2024-09-15T21:24:48.621470+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40091648,"num_objects":9788,"num_object_clones":0,"num_object_copies":19576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8465,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10762","reported_seq":10896,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.734070+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:46.734070+0000","last_peered":"2024-09-15T21:24:46.734070+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:46.734070+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40648704,"num_objects":9924,"num_object_clones":0,"num_object_copies":19848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7706,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5830},{"shards":"0,2","objects":3676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10677","reported_seq":10810,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.752190+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:46.752190+0000","last_peered":"2024-09-15T21:24:46.752190+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:46.752190+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40603648,"num_objects":9913,"num_object_clones":0,"num_object_copies":19826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7586,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5774},{"shards":"0,2","objects":3761}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10642","reported_seq":13139,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:46.724695+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:46.724695+0000","last_peered":"2024-09-15T21:24:46.724695+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:46.724695+0000","last_undegraded":"2024-09-15T21:24:46.724695+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39919616,"num_objects":9746,"num_object_clones":0,"num_object_copies":19492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2902,"num_objects_unfound":0,"num_objects_dirty":9746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7126,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1180,"num_bytes_recovered":4833280,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6844},{"shards":"0,2","objects":2902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10872","reported_seq":11011,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:46.712670+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:46.712670+0000","last_peered":"2024-09-15T21:24:46.712670+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:46.712670+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40936390,"num_objects":9996,"num_object_clones":0,"num_object_copies":19992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9996,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7576,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5849},{"shards":"0,2","objects":3714}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":648168182,"num_objects":158254,"num_object_clones":0,"num_object_copies":316508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2902,"num_objects_unfound":0,"num_objects_dirty":158254,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":132201,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5345,"num_bytes_recovered":21884928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1336201216,"data_stored":1336044939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735804,"kb_used_data":447276,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93636036,"statfs":{"total":96636764160,"available":95883300864,"internally_reserved":0,"allocated":458010624,"data_stored":457734977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":295431849},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182462,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":885364,"kb_used_data":542756,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93486476,"statfs":{"total":96636764160,"available":95730151424,"internally_reserved":0,"allocated":555782144,"data_stored":555506492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,0,6,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579132,"kb_used_data":316968,"kb_used_omap":4,"kb_used_meta":262139,"kb_avail":93792708,"statfs":{"total":96636764160,"available":96043732992,"internally_reserved":0,"allocated":324575232,"data_stored":324255765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":268431023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":554868736,"data_stored":554832606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":324087808,"data_stored":324007863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":457244672,"data_stored":457204470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:51.924 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:51.924 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'10987", 'reported_seq': 11063, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:46.762978+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:24:46.762978+0000', 'last_peered': '2024-09-15T21:24:46.762978+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:24:46.762978+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41635840, 'num_objects': 10165, 'num_object_clones': 0, 'num_object_copies': 20330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10987, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6033}, {'shards': '0,2', 'objects': 3721}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:51.924 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:24:51.924 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'10994", 'reported_seq': 11120, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:48.582109+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:24:48.582109+0000', 'last_peered': '2024-09-15T21:24:48.582109+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:24:48.582109+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41414680, 'num_objects': 10112, 'num_object_clones': 0, 'num_object_copies': 20224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10112, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9094, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3828}, {'shards': '2', 'objects': 5845}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:51.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:51.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'10785", 'reported_seq': 10918, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:46.750662+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:24:46.750662+0000', 'last_peered': '2024-09-15T21:24:46.750662+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:24:46.750662+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40346083, 'num_objects': 9851, 'num_object_clones': 0, 'num_object_copies': 19702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9851, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7461, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5775}, {'shards': '0,2', 'objects': 3611}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:51.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:51.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10730", 'reported_seq': 10865, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:46.661436+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:24:46.661436+0000', 'last_peered': '2024-09-15T21:24:46.661436+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:24:46.661436+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40715206, 'num_objects': 9942, 'num_object_clones': 0, 'num_object_copies': 19884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9942, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7041, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5781}, {'shards': '0,2', 'objects': 3769}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:51.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:24:51.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'10743", 'reported_seq': 10870, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:48.687288+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:24:48.687288+0000', 'last_peered': '2024-09-15T21:24:48.687288+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:24:48.687288+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40018403, 'num_objects': 9771, 'num_object_clones': 0, 'num_object_copies': 19542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9771, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8675, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3602}, {'shards': '2', 'objects': 5684}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:51.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:51.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'10672", 'reported_seq': 10803, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:46.693121+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:24:46.693121+0000', 'last_peered': '2024-09-15T21:24:46.693121+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:24:46.693121+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40181760, 'num_objects': 9810, 'num_object_clones': 0, 'num_object_copies': 19620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9810, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7345, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5790}, {'shards': '0,2', 'objects': 3593}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:51.925 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:51.925 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'10804", 'reported_seq': 10923, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:48.649129+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:24:48.649129+0000', 'last_peered': '2024-09-15T21:24:48.649129+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:24:48.649129+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40301027, 'num_objects': 9840, 'num_object_clones': 0, 'num_object_copies': 19680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9840, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8663, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3610}, {'shards': '2', 'objects': 5750}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:51.926 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:51.926 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'10762", 'reported_seq': 10896, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:46.734070+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:24:46.734070+0000', 'last_peered': '2024-09-15T21:24:46.734070+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:24:46.734070+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40648704, 'num_objects': 9924, 'num_object_clones': 0, 'num_object_copies': 19848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9924, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7706, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5830}, {'shards': '0,2', 'objects': 3676}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:51.926 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:51.926 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'10677", 'reported_seq': 10810, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:46.752190+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:24:46.752190+0000', 'last_peered': '2024-09-15T21:24:46.752190+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:24:46.752190+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40603648, 'num_objects': 9913, 'num_object_clones': 0, 'num_object_copies': 19826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9913, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7586, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5774}, {'shards': '0,2', 'objects': 3761}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:51.926 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:51.926 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'10642", 'reported_seq': 13139, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:24:46.724695+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:24:46.724695+0000', 'last_peered': '2024-09-15T21:24:46.724695+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:24:46.724695+0000', 'last_undegraded': '2024-09-15T21:24:46.724695+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39919616, 'num_objects': 9746, 'num_object_clones': 0, 'num_object_copies': 19492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2902, 'num_objects_unfound': 0, 'num_objects_dirty': 9746, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7126, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1180, 'num_bytes_recovered': 4833280, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6844}, {'shards': '0,2', 'objects': 2902}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:51.926 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:24:51.926 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'10872", 'reported_seq': 11011, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:46.712670+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:24:46.712670+0000', 'last_peered': '2024-09-15T21:24:46.712670+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:24:46.712670+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40936390, 'num_objects': 9996, 'num_object_clones': 0, 'num_object_copies': 19992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9996, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7576, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5849}, {'shards': '0,2', 'objects': 3714}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:51.927 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:52.624 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:52.624 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/316512 objects degraded (13.025%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":344,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158256,"data_bytes":648168201,"bytes_used":2253107200,"bytes_avail":287657185280,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":316512,"degraded_ratio":0.1302478263067435,"misplaced_objects":2902,"misplaced_total":316512,"misplaced_ratio":0.0091686887069052682,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":58353,"recovering_keys_per_sec":0,"num_objects_recovered":171,"num_bytes_recovered":700416,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":379},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:52.648 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:52.648 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:53.238 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:53.238 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:53.259 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-15T21:24:52.763944+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":642315017,"num_objects":156827,"num_object_clones":0,"num_object_copies":313654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":156827,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":133632,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5427,"num_bytes_recovered":22216723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160640,"ondisk_log_size":160640,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2206668,"kb_used_data":1299476,"kb_used_omap":12,"kb_used_meta":907123,"kb_avail":280908852,"statfs":{"total":289910292480,"available":287650664448,"internally_reserved":0,"allocated":1330663424,"data_stored":1329792658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":928893955},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,9,9,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24514560,"num_objects":-5985,"num_object_clones":0,"num_object_copies":-11970,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-252,"num_objects_unfound":0,"num_objects_dirty":-5985,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5985,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002879"},"pg_stats":[{"pgid":"2.f","version":"45'11132","reported_seq":11208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.894820+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:51.894820+0000","last_peered":"2024-09-15T21:24:51.894820+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:51.894820+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41041920,"num_objects":10020,"num_object_clones":0,"num_object_copies":20040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11132,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5888},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10815","reported_seq":10920,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:48.673247+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:48.673247+0000","last_peered":"2024-09-15T21:24:48.673247+0000","last_clean":"2024-09-15T21:24:48.673247+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:48.673247+0000","last_undegraded":"2024-09-15T21:24:48.673247+0000","last_fullsized":"2024-09-15T21:24:48.673247+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40383430,"num_objects":9861,"num_object_clones":0,"num_object_copies":19722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9861,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9096,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10994","reported_seq":11120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:48.582109+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:48.582109+0000","last_peered":"2024-09-15T21:24:48.582109+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:48.582109+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41414680,"num_objects":10112,"num_object_clones":0,"num_object_copies":20224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9094,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3828},{"shards":"2","objects":5845}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10637","reported_seq":10708,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:48.678834+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:48.678834+0000","last_peered":"2024-09-15T21:24:48.678834+0000","last_clean":"2024-09-15T21:24:48.678834+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:48.678834+0000","last_undegraded":"2024-09-15T21:24:48.678834+0000","last_fullsized":"2024-09-15T21:24:48.678834+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39636992,"num_objects":9677,"num_object_clones":0,"num_object_copies":19354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10637,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10812","reported_seq":10945,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:51.889974+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:51.889974+0000","last_peered":"2024-09-15T21:24:51.889974+0000","last_clean":"2024-09-15T21:24:51.889974+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:51.889974+0000","last_undegraded":"2024-09-15T21:24:51.889974+0000","last_fullsized":"2024-09-15T21:24:51.889974+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39703011,"num_objects":9694,"num_object_clones":0,"num_object_copies":19388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7267,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10904","reported_seq":11040,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.784559+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:51.784559+0000","last_peered":"2024-09-15T21:24:51.784559+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:51.784559+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39858659,"num_objects":9732,"num_object_clones":0,"num_object_copies":19464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9732,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7580,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5656},{"shards":"0,2","objects":3492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10883","reported_seq":11019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.898812+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:51.898812+0000","last_peered":"2024-09-15T21:24:51.898812+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:51.898812+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40088518,"num_objects":9789,"num_object_clones":0,"num_object_copies":19578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7194,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5628},{"shards":"0,2","objects":3616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11004","reported_seq":11133,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:51.889915+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:51.889915+0000","last_peered":"2024-09-15T21:24:51.889915+0000","last_clean":"2024-09-15T21:24:51.889915+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:51.889915+0000","last_undegraded":"2024-09-15T21:24:51.889915+0000","last_fullsized":"2024-09-15T21:24:51.889915+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40402944,"num_objects":9864,"num_object_clones":0,"num_object_copies":19728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10743","reported_seq":10870,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:48.687288+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:48.687288+0000","last_peered":"2024-09-15T21:24:48.687288+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:48.687288+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40018403,"num_objects":9771,"num_object_clones":0,"num_object_copies":19542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9771,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8675,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3602},{"shards":"2","objects":5684}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10788","reported_seq":10919,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.842657+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:51.842657+0000","last_peered":"2024-09-15T21:24:51.842657+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:51.842657+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39706624,"num_objects":9694,"num_object_clones":0,"num_object_copies":19388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7461,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5674},{"shards":"0,2","objects":3477}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10804","reported_seq":10923,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:48.649129+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:48.649129+0000","last_peered":"2024-09-15T21:24:48.649129+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:48.649129+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40301027,"num_objects":9840,"num_object_clones":0,"num_object_copies":19680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9840,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8663,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3610},{"shards":"2","objects":5750}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10772","reported_seq":19231,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:48.621470+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:48.621470+0000","last_peered":"2024-09-15T21:24:48.621470+0000","last_clean":"2024-09-15T21:24:48.621470+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:48.621470+0000","last_undegraded":"2024-09-15T21:24:48.621470+0000","last_fullsized":"2024-09-15T21:24:48.621470+0000","mapping_epoch":44,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40091648,"num_objects":9788,"num_object_clones":0,"num_object_copies":19576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8465,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10925","reported_seq":11059,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.910768+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:51.910768+0000","last_peered":"2024-09-15T21:24:51.910768+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:51.910768+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7869,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":3513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10838","reported_seq":10971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.825619+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:51.825619+0000","last_peered":"2024-09-15T21:24:51.825619+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:51.825619+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39944192,"num_objects":9752,"num_object_clones":0,"num_object_copies":19504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7747,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5613},{"shards":"0,2","objects":3600}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10767","reported_seq":13426,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:51.883226+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:51.883226+0000","last_peered":"2024-09-15T21:24:51.883226+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:51.883226+0000","last_undegraded":"2024-09-15T21:24:51.883226+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39407616,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7251,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1261,"num_bytes_recovered":5165056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6800},{"shards":"0,2","objects":2821}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11019","reported_seq":11158,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.859075+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:51.859075+0000","last_peered":"2024-09-15T21:24:51.859075+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:51.859075+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40334278,"num_objects":9849,"num_object_clones":0,"num_object_copies":19698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9849,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7723,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5702},{"shards":"0,2","objects":3567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":642314998,"num_objects":156825,"num_object_clones":0,"num_object_copies":313650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":156825,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":133630,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5426,"num_bytes_recovered":22216704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1328304128,"data_stored":1328147851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160637,"ondisk_log_size":160637,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735804,"kb_used_data":447276,"kb_used_omap":4,"kb_used_meta":288507,"kb_avail":93636036,"statfs":{"total":96636764160,"available":95883300864,"internally_reserved":0,"allocated":458010624,"data_stored":457734977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":295431849},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":870264,"kb_used_data":535972,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93501576,"statfs":{"total":96636764160,"available":95745613824,"internally_reserved":0,"allocated":548835328,"data_stored":548559676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600600,"kb_used_data":316228,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93771240,"statfs":{"total":96636764160,"available":96021749760,"internally_reserved":0,"allocated":323817472,"data_stored":323498005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":547729408,"data_stored":547693278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323330048,"data_stored":323250103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":457244672,"data_stored":457204470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:56.262 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:56.832 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:56.877 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:56.877 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-15T21:24:56.764942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638927625,"num_objects":156000,"num_object_clones":0,"num_object_copies":312000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":156000,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":134459,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5427,"num_bytes_recovered":22216723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2206492,"kb_used_data":1295132,"kb_used_omap":12,"kb_used_meta":911283,"kb_avail":280909028,"statfs":{"total":289910292480,"available":287650844672,"internally_reserved":0,"allocated":1326215168,"data_stored":1325344402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":933153795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,7,12,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18579456,"num_objects":-4536,"num_object_clones":0,"num_object_copies":-9072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-163,"num_objects_unfound":0,"num_objects_dirty":-4536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4536,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":667648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002900"},"pg_stats":[{"pgid":"2.f","version":"45'11132","reported_seq":11208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.894820+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:51.894820+0000","last_peered":"2024-09-15T21:24:51.894820+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:51.894820+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41041920,"num_objects":10020,"num_object_clones":0,"num_object_copies":20040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11132,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5888},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10950","reported_seq":11054,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:53.717299+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:53.717299+0000","last_peered":"2024-09-15T21:24:53.717299+0000","last_clean":"2024-09-15T21:24:53.717299+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:53.717299+0000","last_undegraded":"2024-09-15T21:24:53.717299+0000","last_fullsized":"2024-09-15T21:24:53.717299+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39830470,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9231,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11156","reported_seq":11282,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:53.630972+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:53.630972+0000","last_peered":"2024-09-15T21:24:53.630972+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:53.630972+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40751128,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9256,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3666},{"shards":"2","objects":5683}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10780","reported_seq":10851,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:53.665703+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:53.665703+0000","last_peered":"2024-09-15T21:24:53.665703+0000","last_clean":"2024-09-15T21:24:53.665703+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:53.665703+0000","last_undegraded":"2024-09-15T21:24:53.665703+0000","last_fullsized":"2024-09-15T21:24:53.665703+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39051264,"num_objects":9534,"num_object_clones":0,"num_object_copies":19068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10780,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10812","reported_seq":10945,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:51.889974+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:51.889974+0000","last_peered":"2024-09-15T21:24:51.889974+0000","last_clean":"2024-09-15T21:24:51.889974+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:51.889974+0000","last_undegraded":"2024-09-15T21:24:51.889974+0000","last_fullsized":"2024-09-15T21:24:51.889974+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39703011,"num_objects":9694,"num_object_clones":0,"num_object_copies":19388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7267,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10904","reported_seq":11040,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.784559+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:51.784559+0000","last_peered":"2024-09-15T21:24:51.784559+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:51.784559+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39858659,"num_objects":9732,"num_object_clones":0,"num_object_copies":19464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9732,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7580,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5656},{"shards":"0,2","objects":3492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10883","reported_seq":11019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.898812+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:51.898812+0000","last_peered":"2024-09-15T21:24:51.898812+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:51.898812+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40088518,"num_objects":9789,"num_object_clones":0,"num_object_copies":19578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7194,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5628},{"shards":"0,2","objects":3616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11004","reported_seq":11133,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:51.889915+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:51.889915+0000","last_peered":"2024-09-15T21:24:51.889915+0000","last_clean":"2024-09-15T21:24:51.889915+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:51.889915+0000","last_undegraded":"2024-09-15T21:24:51.889915+0000","last_fullsized":"2024-09-15T21:24:51.889915+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40402944,"num_objects":9864,"num_object_clones":0,"num_object_copies":19728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10870","reported_seq":10997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:53.682185+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:53.682185+0000","last_peered":"2024-09-15T21:24:53.682185+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:53.682185+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39498211,"num_objects":9644,"num_object_clones":0,"num_object_copies":19288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9644,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8802,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3475},{"shards":"2","objects":5557}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10788","reported_seq":10919,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.842657+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:51.842657+0000","last_peered":"2024-09-15T21:24:51.842657+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:51.842657+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39706624,"num_objects":9694,"num_object_clones":0,"num_object_copies":19388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7461,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5674},{"shards":"0,2","objects":3477}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10934","reported_seq":11053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:53.516857+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:53.516857+0000","last_peered":"2024-09-15T21:24:53.516857+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:53.516857+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39768547,"num_objects":9710,"num_object_clones":0,"num_object_copies":19420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9710,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8793,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3480},{"shards":"2","objects":5620}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10902","reported_seq":19361,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:53.674427+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:53.674427+0000","last_peered":"2024-09-15T21:24:53.674427+0000","last_clean":"2024-09-15T21:24:53.674427+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:53.674427+0000","last_undegraded":"2024-09-15T21:24:53.674427+0000","last_fullsized":"2024-09-15T21:24:53.674427+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39559168,"num_objects":9658,"num_object_clones":0,"num_object_copies":19316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8595,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10925","reported_seq":11059,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.910768+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:51.910768+0000","last_peered":"2024-09-15T21:24:51.910768+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:51.910768+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7869,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":3513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10838","reported_seq":10971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.825619+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:51.825619+0000","last_peered":"2024-09-15T21:24:51.825619+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:51.825619+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39944192,"num_objects":9752,"num_object_clones":0,"num_object_copies":19504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7747,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5613},{"shards":"0,2","objects":3600}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10767","reported_seq":13426,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:51.883226+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:51.883226+0000","last_peered":"2024-09-15T21:24:51.883226+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:51.883226+0000","last_undegraded":"2024-09-15T21:24:51.883226+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39407616,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7251,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1261,"num_bytes_recovered":5165056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6800},{"shards":"0,2","objects":2821}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11019","reported_seq":11158,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.859075+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:51.859075+0000","last_peered":"2024-09-15T21:24:51.859075+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:51.859075+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40334278,"num_objects":9849,"num_object_clones":0,"num_object_copies":19698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9849,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7723,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5702},{"shards":"0,2","objects":3567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":638927606,"num_objects":155998,"num_object_clones":0,"num_object_copies":311996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":155998,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":134457,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5426,"num_bytes_recovered":22216704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1323819008,"data_stored":1323662731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735628,"kb_used_data":442932,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93636212,"statfs":{"total":96636764160,"available":95883481088,"internally_reserved":0,"allocated":453562368,"data_stored":453286721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":299691689},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":870264,"kb_used_data":535972,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93501576,"statfs":{"total":96636764160,"available":95745613824,"internally_reserved":0,"allocated":548835328,"data_stored":548559676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600600,"kb_used_data":316228,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93771240,"statfs":{"total":96636764160,"available":96021749760,"internally_reserved":0,"allocated":323817472,"data_stored":323498005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":547729408,"data_stored":547693278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323330048,"data_stored":323250103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":452759552,"data_stored":452719350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:56.879 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:57.437 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:57.439 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:57.471 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-15T21:24:56.764942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638927625,"num_objects":156000,"num_object_clones":0,"num_object_copies":312000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":156000,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":134459,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5427,"num_bytes_recovered":22216723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2206492,"kb_used_data":1295132,"kb_used_omap":12,"kb_used_meta":911283,"kb_avail":280909028,"statfs":{"total":289910292480,"available":287650844672,"internally_reserved":0,"allocated":1326215168,"data_stored":1325344402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":933153795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,7,12,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18579456,"num_objects":-4536,"num_object_clones":0,"num_object_copies":-9072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-163,"num_objects_unfound":0,"num_objects_dirty":-4536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4536,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":667648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002900"},"pg_stats":[{"pgid":"2.f","version":"45'11132","reported_seq":11208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.894820+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:51.894820+0000","last_peered":"2024-09-15T21:24:51.894820+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:51.894820+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41041920,"num_objects":10020,"num_object_clones":0,"num_object_copies":20040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11132,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5888},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10950","reported_seq":11054,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:53.717299+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:53.717299+0000","last_peered":"2024-09-15T21:24:53.717299+0000","last_clean":"2024-09-15T21:24:53.717299+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:53.717299+0000","last_undegraded":"2024-09-15T21:24:53.717299+0000","last_fullsized":"2024-09-15T21:24:53.717299+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39830470,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9231,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11156","reported_seq":11282,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:53.630972+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:53.630972+0000","last_peered":"2024-09-15T21:24:53.630972+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:53.630972+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40751128,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9256,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3666},{"shards":"2","objects":5683}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10780","reported_seq":10851,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:53.665703+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:53.665703+0000","last_peered":"2024-09-15T21:24:53.665703+0000","last_clean":"2024-09-15T21:24:53.665703+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:53.665703+0000","last_undegraded":"2024-09-15T21:24:53.665703+0000","last_fullsized":"2024-09-15T21:24:53.665703+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39051264,"num_objects":9534,"num_object_clones":0,"num_object_copies":19068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10780,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10812","reported_seq":10945,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:51.889974+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:51.889974+0000","last_peered":"2024-09-15T21:24:51.889974+0000","last_clean":"2024-09-15T21:24:51.889974+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:51.889974+0000","last_undegraded":"2024-09-15T21:24:51.889974+0000","last_fullsized":"2024-09-15T21:24:51.889974+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39703011,"num_objects":9694,"num_object_clones":0,"num_object_copies":19388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7267,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10904","reported_seq":11040,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.784559+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:51.784559+0000","last_peered":"2024-09-15T21:24:51.784559+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:51.784559+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39858659,"num_objects":9732,"num_object_clones":0,"num_object_copies":19464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9732,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7580,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5656},{"shards":"0,2","objects":3492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10883","reported_seq":11019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.898812+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:51.898812+0000","last_peered":"2024-09-15T21:24:51.898812+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:51.898812+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40088518,"num_objects":9789,"num_object_clones":0,"num_object_copies":19578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7194,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5628},{"shards":"0,2","objects":3616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11004","reported_seq":11133,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:51.889915+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:51.889915+0000","last_peered":"2024-09-15T21:24:51.889915+0000","last_clean":"2024-09-15T21:24:51.889915+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:51.889915+0000","last_undegraded":"2024-09-15T21:24:51.889915+0000","last_fullsized":"2024-09-15T21:24:51.889915+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40402944,"num_objects":9864,"num_object_clones":0,"num_object_copies":19728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10870","reported_seq":10997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:53.682185+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:53.682185+0000","last_peered":"2024-09-15T21:24:53.682185+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:53.682185+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39498211,"num_objects":9644,"num_object_clones":0,"num_object_copies":19288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9644,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8802,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3475},{"shards":"2","objects":5557}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10788","reported_seq":10919,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.842657+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:51.842657+0000","last_peered":"2024-09-15T21:24:51.842657+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:51.842657+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39706624,"num_objects":9694,"num_object_clones":0,"num_object_copies":19388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7461,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5674},{"shards":"0,2","objects":3477}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10934","reported_seq":11053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:53.516857+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:53.516857+0000","last_peered":"2024-09-15T21:24:53.516857+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:53.516857+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39768547,"num_objects":9710,"num_object_clones":0,"num_object_copies":19420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9710,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8793,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3480},{"shards":"2","objects":5620}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10902","reported_seq":19361,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:53.674427+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:53.674427+0000","last_peered":"2024-09-15T21:24:53.674427+0000","last_clean":"2024-09-15T21:24:53.674427+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:53.674427+0000","last_undegraded":"2024-09-15T21:24:53.674427+0000","last_fullsized":"2024-09-15T21:24:53.674427+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39559168,"num_objects":9658,"num_object_clones":0,"num_object_copies":19316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8595,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10925","reported_seq":11059,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.910768+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:51.910768+0000","last_peered":"2024-09-15T21:24:51.910768+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:51.910768+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7869,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":3513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10838","reported_seq":10971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.825619+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:51.825619+0000","last_peered":"2024-09-15T21:24:51.825619+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:51.825619+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39944192,"num_objects":9752,"num_object_clones":0,"num_object_copies":19504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7747,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5613},{"shards":"0,2","objects":3600}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10767","reported_seq":13426,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:51.883226+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:51.883226+0000","last_peered":"2024-09-15T21:24:51.883226+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:51.883226+0000","last_undegraded":"2024-09-15T21:24:51.883226+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39407616,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7251,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1261,"num_bytes_recovered":5165056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6800},{"shards":"0,2","objects":2821}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11019","reported_seq":11158,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.859075+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:51.859075+0000","last_peered":"2024-09-15T21:24:51.859075+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:51.859075+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40334278,"num_objects":9849,"num_object_clones":0,"num_object_copies":19698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9849,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7723,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5702},{"shards":"0,2","objects":3567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":638927606,"num_objects":155998,"num_object_clones":0,"num_object_copies":311996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":155998,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":134457,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5426,"num_bytes_recovered":22216704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1323819008,"data_stored":1323662731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735628,"kb_used_data":442932,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93636212,"statfs":{"total":96636764160,"available":95883481088,"internally_reserved":0,"allocated":453562368,"data_stored":453286721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":299691689},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":870264,"kb_used_data":535972,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93501576,"statfs":{"total":96636764160,"available":95745613824,"internally_reserved":0,"allocated":548835328,"data_stored":548559676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600600,"kb_used_data":316228,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93771240,"statfs":{"total":96636764160,"available":96021749760,"internally_reserved":0,"allocated":323817472,"data_stored":323498005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":547729408,"data_stored":547693278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323330048,"data_stored":323250103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":452759552,"data_stored":452719350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:24:57.472 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:24:57.473 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11132", 'reported_seq': 11208, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:51.894820+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:24:51.894820+0000', 'last_peered': '2024-09-15T21:24:51.894820+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:24:51.894820+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41041920, 'num_objects': 10020, 'num_object_clones': 0, 'num_object_copies': 20040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10020, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11132, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5888}, {'shards': '0,2', 'objects': 3576}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:57.473 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:24:57.473 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'11156", 'reported_seq': 11282, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:53.630972+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:24:53.630972+0000', 'last_peered': '2024-09-15T21:24:53.630972+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:24:53.630972+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40751128, 'num_objects': 9950, 'num_object_clones': 0, 'num_object_copies': 19900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9950, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9256, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3666}, {'shards': '2', 'objects': 5683}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:57.473 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:24:57.473 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'10904", 'reported_seq': 11040, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:51.784559+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:24:51.784559+0000', 'last_peered': '2024-09-15T21:24:51.784559+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:24:51.784559+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39858659, 'num_objects': 9732, 'num_object_clones': 0, 'num_object_copies': 19464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9732, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7580, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5656}, {'shards': '0,2', 'objects': 3492}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:57.473 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:24:57.474 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'10883", 'reported_seq': 11019, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:51.898812+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:24:51.898812+0000', 'last_peered': '2024-09-15T21:24:51.898812+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:24:51.898812+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40088518, 'num_objects': 9789, 'num_object_clones': 0, 'num_object_copies': 19578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9789, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7194, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5628}, {'shards': '0,2', 'objects': 3616}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:57.474 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:24:57.474 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'10870", 'reported_seq': 10997, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:53.682185+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:24:53.682185+0000', 'last_peered': '2024-09-15T21:24:53.682185+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:24:53.682185+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39498211, 'num_objects': 9644, 'num_object_clones': 0, 'num_object_copies': 19288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9644, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8802, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3475}, {'shards': '2', 'objects': 5557}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:57.474 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:24:57.474 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'10788", 'reported_seq': 10919, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:51.842657+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:24:51.842657+0000', 'last_peered': '2024-09-15T21:24:51.842657+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:24:51.842657+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39706624, 'num_objects': 9694, 'num_object_clones': 0, 'num_object_copies': 19388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9694, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7461, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5674}, {'shards': '0,2', 'objects': 3477}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:57.474 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:24:57.474 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'10934", 'reported_seq': 11053, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:53.516857+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:24:53.516857+0000', 'last_peered': '2024-09-15T21:24:53.516857+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:24:53.516857+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39768547, 'num_objects': 9710, 'num_object_clones': 0, 'num_object_copies': 19420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9710, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8793, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3480}, {'shards': '2', 'objects': 5620}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:24:57.474 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:24:57.475 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'10925", 'reported_seq': 11059, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:51.910768+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:24:51.910768+0000', 'last_peered': '2024-09-15T21:24:51.910768+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:24:51.910768+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39981056, 'num_objects': 9761, 'num_object_clones': 0, 'num_object_copies': 19522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9761, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7869, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5667}, {'shards': '0,2', 'objects': 3513}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:57.475 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:24:57.475 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'10838", 'reported_seq': 10971, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:51.825619+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:24:51.825619+0000', 'last_peered': '2024-09-15T21:24:51.825619+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:24:51.825619+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39944192, 'num_objects': 9752, 'num_object_clones': 0, 'num_object_copies': 19504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9752, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7747, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5613}, {'shards': '0,2', 'objects': 3600}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:57.475 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:24:57.475 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'10767", 'reported_seq': 13426, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:24:51.883226+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:24:51.883226+0000', 'last_peered': '2024-09-15T21:24:51.883226+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:24:51.883226+0000', 'last_undegraded': '2024-09-15T21:24:51.883226+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39407616, 'num_objects': 9621, 'num_object_clones': 0, 'num_object_copies': 19242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2821, 'num_objects_unfound': 0, 'num_objects_dirty': 9621, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7251, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1261, 'num_bytes_recovered': 5165056, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6800}, {'shards': '0,2', 'objects': 2821}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:57.475 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:24:57.475 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11019", 'reported_seq': 11158, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:51.859075+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:24:51.859075+0000', 'last_peered': '2024-09-15T21:24:51.859075+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:24:51.859075+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40334278, 'num_objects': 9849, 'num_object_clones': 0, 'num_object_copies': 19698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9849, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7723, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5702}, {'shards': '0,2', 'objects': 3567}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:24:57.476 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:24:58.177 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:58.177 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/312000 objects degraded (13.213%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":349,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156000,"data_bytes":638927625,"bytes_used":2259447808,"bytes_avail":287650844672,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":312000,"degraded_ratio":0.13213141025641026,"misplaced_objects":2821,"misplaced_total":312000,"misplaced_ratio":0.0090416666666666666,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":55623,"recovering_keys_per_sec":0,"num_objects_recovered":163,"num_bytes_recovered":667648,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":377},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:24:58.201 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:24:58.201 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:24:58.749 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:24:58.750 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:24:58.781 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-15T21:24:56.764942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":638927625,"num_objects":156000,"num_object_clones":0,"num_object_copies":312000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":156000,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":134459,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5427,"num_bytes_recovered":22216723,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160667,"ondisk_log_size":160667,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2206492,"kb_used_data":1295132,"kb_used_omap":12,"kb_used_meta":911283,"kb_avail":280909028,"statfs":{"total":289910292480,"available":287650844672,"internally_reserved":0,"allocated":1326215168,"data_stored":1325344402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":933153795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,7,12,10,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18579456,"num_objects":-4536,"num_object_clones":0,"num_object_copies":-9072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-163,"num_objects_unfound":0,"num_objects_dirty":-4536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4536,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":667648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002900"},"pg_stats":[{"pgid":"2.f","version":"45'11132","reported_seq":11208,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.894820+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:51.894820+0000","last_peered":"2024-09-15T21:24:51.894820+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:51.894820+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41041920,"num_objects":10020,"num_object_clones":0,"num_object_copies":20040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11132,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5888},{"shards":"0,2","objects":3576}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10950","reported_seq":11054,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:53.717299+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:53.717299+0000","last_peered":"2024-09-15T21:24:53.717299+0000","last_clean":"2024-09-15T21:24:53.717299+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:53.717299+0000","last_undegraded":"2024-09-15T21:24:53.717299+0000","last_fullsized":"2024-09-15T21:24:53.717299+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39830470,"num_objects":9726,"num_object_clones":0,"num_object_copies":19452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9726,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9231,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11156","reported_seq":11282,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:53.630972+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:53.630972+0000","last_peered":"2024-09-15T21:24:53.630972+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:53.630972+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40751128,"num_objects":9950,"num_object_clones":0,"num_object_copies":19900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9950,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9256,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3666},{"shards":"2","objects":5683}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10780","reported_seq":10851,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:53.665703+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:53.665703+0000","last_peered":"2024-09-15T21:24:53.665703+0000","last_clean":"2024-09-15T21:24:53.665703+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:53.665703+0000","last_undegraded":"2024-09-15T21:24:53.665703+0000","last_fullsized":"2024-09-15T21:24:53.665703+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39051264,"num_objects":9534,"num_object_clones":0,"num_object_copies":19068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10780,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10812","reported_seq":10945,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:51.889974+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:51.889974+0000","last_peered":"2024-09-15T21:24:51.889974+0000","last_clean":"2024-09-15T21:24:51.889974+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:51.889974+0000","last_undegraded":"2024-09-15T21:24:51.889974+0000","last_fullsized":"2024-09-15T21:24:51.889974+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39703011,"num_objects":9694,"num_object_clones":0,"num_object_copies":19388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7267,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10904","reported_seq":11040,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.784559+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:51.784559+0000","last_peered":"2024-09-15T21:24:51.784559+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:51.784559+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39858659,"num_objects":9732,"num_object_clones":0,"num_object_copies":19464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9732,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7580,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5656},{"shards":"0,2","objects":3492}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10883","reported_seq":11019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.898812+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:51.898812+0000","last_peered":"2024-09-15T21:24:51.898812+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:51.898812+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40088518,"num_objects":9789,"num_object_clones":0,"num_object_copies":19578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7194,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5628},{"shards":"0,2","objects":3616}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11004","reported_seq":11133,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:51.889915+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:51.889915+0000","last_peered":"2024-09-15T21:24:51.889915+0000","last_clean":"2024-09-15T21:24:51.889915+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:51.889915+0000","last_undegraded":"2024-09-15T21:24:51.889915+0000","last_fullsized":"2024-09-15T21:24:51.889915+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40402944,"num_objects":9864,"num_object_clones":0,"num_object_copies":19728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10870","reported_seq":10997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:53.682185+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:53.682185+0000","last_peered":"2024-09-15T21:24:53.682185+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:53.682185+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39498211,"num_objects":9644,"num_object_clones":0,"num_object_copies":19288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9644,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8802,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3475},{"shards":"2","objects":5557}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10788","reported_seq":10919,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.842657+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:51.842657+0000","last_peered":"2024-09-15T21:24:51.842657+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:51.842657+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39706624,"num_objects":9694,"num_object_clones":0,"num_object_copies":19388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7461,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5674},{"shards":"0,2","objects":3477}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10934","reported_seq":11053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:53.516857+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:53.516857+0000","last_peered":"2024-09-15T21:24:53.516857+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:53.516857+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39768547,"num_objects":9710,"num_object_clones":0,"num_object_copies":19420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9710,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8793,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3480},{"shards":"2","objects":5620}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10902","reported_seq":19361,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:53.674427+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:53.674427+0000","last_peered":"2024-09-15T21:24:53.674427+0000","last_clean":"2024-09-15T21:24:53.674427+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:53.674427+0000","last_undegraded":"2024-09-15T21:24:53.674427+0000","last_fullsized":"2024-09-15T21:24:53.674427+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39559168,"num_objects":9658,"num_object_clones":0,"num_object_copies":19316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8595,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10925","reported_seq":11059,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.910768+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:51.910768+0000","last_peered":"2024-09-15T21:24:51.910768+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:51.910768+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39981056,"num_objects":9761,"num_object_clones":0,"num_object_copies":19522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7869,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5667},{"shards":"0,2","objects":3513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10838","reported_seq":10971,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.825619+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:51.825619+0000","last_peered":"2024-09-15T21:24:51.825619+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:51.825619+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39944192,"num_objects":9752,"num_object_clones":0,"num_object_copies":19504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7747,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5613},{"shards":"0,2","objects":3600}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10767","reported_seq":13426,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:51.883226+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:51.883226+0000","last_peered":"2024-09-15T21:24:51.883226+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:51.883226+0000","last_undegraded":"2024-09-15T21:24:51.883226+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39407616,"num_objects":9621,"num_object_clones":0,"num_object_copies":19242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":9621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7251,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1261,"num_bytes_recovered":5165056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6800},{"shards":"0,2","objects":2821}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11019","reported_seq":11158,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:51.859075+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:51.859075+0000","last_peered":"2024-09-15T21:24:51.859075+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:51.859075+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40334278,"num_objects":9849,"num_object_clones":0,"num_object_copies":19698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9849,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7723,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5702},{"shards":"0,2","objects":3567}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":638927606,"num_objects":155998,"num_object_clones":0,"num_object_copies":311996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2821,"num_objects_unfound":0,"num_objects_dirty":155998,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":134457,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5426,"num_bytes_recovered":22216704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1323819008,"data_stored":1323662731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160664,"ondisk_log_size":160664,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735628,"kb_used_data":442932,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93636212,"statfs":{"total":96636764160,"available":95883481088,"internally_reserved":0,"allocated":453562368,"data_stored":453286721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":299691689},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":870264,"kb_used_data":535972,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93501576,"statfs":{"total":96636764160,"available":95745613824,"internally_reserved":0,"allocated":548835328,"data_stored":548559676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600600,"kb_used_data":316228,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93771240,"statfs":{"total":96636764160,"available":96021749760,"internally_reserved":0,"allocated":323817472,"data_stored":323498005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":547729408,"data_stored":547693278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":323330048,"data_stored":323250103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":452759552,"data_stored":452719350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:01.783 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:02.355 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:02.356 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:02.379 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":214,"stamp":"2024-09-15T21:25:00.765902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":629633801,"num_objects":153731,"num_object_clones":0,"num_object_copies":307462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2733,"num_objects_unfound":0,"num_objects_dirty":153731,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":136728,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5515,"num_bytes_recovered":22577171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160936,"ondisk_log_size":160936,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2194196,"kb_used_data":1282844,"kb_used_omap":12,"kb_used_meta":911283,"kb_avail":280921324,"statfs":{"total":289910292480,"available":287663435776,"internally_reserved":0,"allocated":1313632256,"data_stored":1312761490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":933153795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,2,8,14,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18534400,"num_objects":-4525,"num_object_clones":0,"num_object_copies":-9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-169,"num_objects_unfound":0,"num_objects_dirty":-4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4525,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002958"},"pg_stats":[{"pgid":"2.f","version":"45'11258","reported_seq":11334,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:56.970640+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:24:56.970640+0000","last_peered":"2024-09-15T21:24:56.970640+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:24:56.970640+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40525824,"num_objects":9894,"num_object_clones":0,"num_object_copies":19788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11258,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5762},{"shards":"0,2","objects":3450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11085","reported_seq":11190,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:58.593462+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:58.593462+0000","last_peered":"2024-09-15T21:24:58.593462+0000","last_clean":"2024-09-15T21:24:58.593462+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:58.593462+0000","last_undegraded":"2024-09-15T21:24:58.593462+0000","last_fullsized":"2024-09-15T21:24:58.593462+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39277510,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9366,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11289","reported_seq":11416,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:58.712441+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:58.712441+0000","last_peered":"2024-09-15T21:24:58.712441+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:58.712441+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40206360,"num_objects":9817,"num_object_clones":0,"num_object_copies":19634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9817,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9389,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3533},{"shards":"2","objects":5550}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10907","reported_seq":10978,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:58.571112+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:58.571112+0000","last_peered":"2024-09-15T21:24:58.571112+0000","last_clean":"2024-09-15T21:24:58.571112+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:58.571112+0000","last_undegraded":"2024-09-15T21:24:58.571112+0000","last_fullsized":"2024-09-15T21:24:58.571112+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38531072,"num_objects":9407,"num_object_clones":0,"num_object_copies":18814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10907,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10945","reported_seq":11078,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:56.885393+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:24:56.885393+0000","last_peered":"2024-09-15T21:24:56.885393+0000","last_clean":"2024-09-15T21:24:56.885393+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:24:56.885393+0000","last_undegraded":"2024-09-15T21:24:56.885393+0000","last_fullsized":"2024-09-15T21:24:56.885393+0000","mapping_epoch":34,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39158243,"num_objects":9561,"num_object_clones":0,"num_object_copies":19122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7400,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11063","reported_seq":11199,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:56.895577+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:24:56.895577+0000","last_peered":"2024-09-15T21:24:56.895577+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:24:56.895577+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39207395,"num_objects":9573,"num_object_clones":0,"num_object_copies":19146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9573,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7739,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5497},{"shards":"0,2","objects":3333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11033","reported_seq":11169,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:56.978700+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:24:56.978700+0000","last_peered":"2024-09-15T21:24:56.978700+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:24:56.978700+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39474118,"num_objects":9639,"num_object_clones":0,"num_object_copies":19278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7344,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5478},{"shards":"0,2","objects":3466}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11159","reported_seq":11288,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:56.962582+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:24:56.962582+0000","last_peered":"2024-09-15T21:24:56.962582+0000","last_clean":"2024-09-15T21:24:56.962582+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:24:56.962582+0000","last_undegraded":"2024-09-15T21:24:56.962582+0000","last_fullsized":"2024-09-15T21:24:56.962582+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39768064,"num_objects":9709,"num_object_clones":0,"num_object_copies":19418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7931,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11028","reported_seq":11155,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:58.670645+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:58.670645+0000","last_peered":"2024-09-15T21:24:58.670645+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:58.670645+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38851043,"num_objects":9486,"num_object_clones":0,"num_object_copies":18972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9486,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8960,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3317},{"shards":"2","objects":5399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10946","reported_seq":11077,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:56.924589+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:24:56.924589+0000","last_peered":"2024-09-15T21:24:56.924589+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:24:56.924589+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39059456,"num_objects":9536,"num_object_clones":0,"num_object_copies":19072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7619,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5516},{"shards":"0,2","objects":3319}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11071","reported_seq":11190,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:58.494937+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:58.494937+0000","last_peered":"2024-09-15T21:24:58.494937+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:58.494937+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39207395,"num_objects":9573,"num_object_clones":0,"num_object_copies":19146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9573,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8930,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3343},{"shards":"2","objects":5483}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11052","reported_seq":19511,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:58.690924+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:58.690924+0000","last_peered":"2024-09-15T21:24:58.690924+0000","last_clean":"2024-09-15T21:24:58.690924+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:58.690924+0000","last_undegraded":"2024-09-15T21:24:58.690924+0000","last_fullsized":"2024-09-15T21:24:58.690924+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38944768,"num_objects":9508,"num_object_clones":0,"num_object_copies":19016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11059","reported_seq":11193,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:56.883776+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:24:56.883776+0000","last_peered":"2024-09-15T21:24:56.883776+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:24:56.883776+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39432192,"num_objects":9627,"num_object_clones":0,"num_object_copies":19254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8003,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5533},{"shards":"0,2","objects":3379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10981","reported_seq":11114,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:56.947938+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:24:56.947938+0000","last_peered":"2024-09-15T21:24:56.947938+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:24:56.947938+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39358464,"num_objects":9609,"num_object_clones":0,"num_object_copies":19218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7890,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5470},{"shards":"0,2","objects":3457}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10897","reported_seq":13729,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:24:56.916758+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:24:56.916758+0000","last_peered":"2024-09-15T21:24:56.916758+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:24:56.916758+0000","last_undegraded":"2024-09-15T21:24:56.916758+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38875136,"num_objects":9491,"num_object_clones":0,"num_object_copies":18982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2733,"num_objects_unfound":0,"num_objects_dirty":9491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7381,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1349,"num_bytes_recovered":5525504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6758},{"shards":"0,2","objects":2733}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11160","reported_seq":11299,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:56.893936+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:24:56.893936+0000","last_peered":"2024-09-15T21:24:56.893936+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:24:56.893936+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39756742,"num_objects":9708,"num_object_clones":0,"num_object_copies":19416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9708,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7864,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5561},{"shards":"0,2","objects":3426}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":629633782,"num_objects":153729,"num_object_clones":0,"num_object_copies":307458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2733,"num_objects_unfound":0,"num_objects_dirty":153729,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":136726,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5514,"num_bytes_recovered":22577152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1311412224,"data_stored":1311255947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160933,"ondisk_log_size":160933,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718544,"kb_used_data":438336,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93653296,"statfs":{"total":96636764160,"available":95900975104,"internally_reserved":0,"allocated":448856064,"data_stored":448580417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":286912169},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182464,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":867604,"kb_used_data":529152,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93504236,"statfs":{"total":96636764160,"available":95748337664,"internally_reserved":0,"allocated":541851648,"data_stored":541575996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608048,"kb_used_data":315356,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93763792,"statfs":{"total":96636764160,"available":96014123008,"internally_reserved":0,"allocated":322924544,"data_stored":322605077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":299691695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,4,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":540758016,"data_stored":540721886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322527232,"data_stored":322447287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448126976,"data_stored":448086774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:02.381 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:02.941 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:02.943 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:02.964 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-15T21:25:02.766332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":623829769,"num_objects":152314,"num_object_clones":0,"num_object_copies":304628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2644,"num_objects_unfound":0,"num_objects_dirty":152314,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":138145,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5604,"num_bytes_recovered":22941715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160953,"ondisk_log_size":160953,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2191780,"kb_used_data":1276272,"kb_used_omap":12,"kb_used_meta":915443,"kb_avail":280923740,"statfs":{"total":289910292480,"available":287665909760,"internally_reserved":0,"allocated":1306902528,"data_stored":1306031762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":937413635},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,11,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24338432,"num_objects":-5942,"num_object_clones":0,"num_object_copies":-11884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-258,"num_objects_unfound":0,"num_objects_dirty":-5942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5942,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":1056768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002850"},"pg_stats":[{"pgid":"2.f","version":"45'11398","reported_seq":11474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.052172+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:02.052172+0000","last_peered":"2024-09-15T21:25:02.052172+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:02.052172+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39952384,"num_objects":9754,"num_object_clones":0,"num_object_copies":19508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11398,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5622},{"shards":"0,2","objects":3310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11085","reported_seq":11190,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:58.593462+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:58.593462+0000","last_peered":"2024-09-15T21:24:58.593462+0000","last_clean":"2024-09-15T21:24:58.593462+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:58.593462+0000","last_undegraded":"2024-09-15T21:24:58.593462+0000","last_fullsized":"2024-09-15T21:24:58.593462+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39277510,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9366,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11289","reported_seq":11416,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:58.712441+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:58.712441+0000","last_peered":"2024-09-15T21:24:58.712441+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:58.712441+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40206360,"num_objects":9817,"num_object_clones":0,"num_object_copies":19634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9817,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9389,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3533},{"shards":"2","objects":5550}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10907","reported_seq":10978,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:58.571112+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:58.571112+0000","last_peered":"2024-09-15T21:24:58.571112+0000","last_clean":"2024-09-15T21:24:58.571112+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:58.571112+0000","last_undegraded":"2024-09-15T21:24:58.571112+0000","last_fullsized":"2024-09-15T21:24:58.571112+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38531072,"num_objects":9407,"num_object_clones":0,"num_object_copies":18814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10907,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11092","reported_seq":11225,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:01.979397+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:01.979397+0000","last_peered":"2024-09-15T21:25:01.979397+0000","last_clean":"2024-09-15T21:25:01.979397+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:01.979397+0000","last_undegraded":"2024-09-15T21:25:01.979397+0000","last_fullsized":"2024-09-15T21:25:01.979397+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38556131,"num_objects":9414,"num_object_clones":0,"num_object_copies":18828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7547,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11206","reported_seq":11342,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:01.999242+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:01.999242+0000","last_peered":"2024-09-15T21:25:01.999242+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:01.999242+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38621667,"num_objects":9430,"num_object_clones":0,"num_object_copies":18860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9430,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7882,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":3190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11195","reported_seq":11331,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:01.921407+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:01.921407+0000","last_peered":"2024-09-15T21:25:01.921407+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:01.921407+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38810566,"num_objects":9477,"num_object_clones":0,"num_object_copies":18954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7506,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5316},{"shards":"0,2","objects":3304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11291","reported_seq":11420,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:01.876247+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:01.876247+0000","last_peered":"2024-09-15T21:25:01.876247+0000","last_clean":"2024-09-15T21:25:01.876247+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:01.876247+0000","last_undegraded":"2024-09-15T21:25:01.876247+0000","last_fullsized":"2024-09-15T21:25:01.876247+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227392,"num_objects":9577,"num_object_clones":0,"num_object_copies":19154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8063,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11028","reported_seq":11155,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:58.670645+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:58.670645+0000","last_peered":"2024-09-15T21:24:58.670645+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:58.670645+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38851043,"num_objects":9486,"num_object_clones":0,"num_object_copies":18972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9486,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8960,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3317},{"shards":"2","objects":5399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11054","reported_seq":11185,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:01.997355+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:01.997355+0000","last_peered":"2024-09-15T21:25:01.997355+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:01.997355+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38617088,"num_objects":9428,"num_object_clones":0,"num_object_copies":18856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7727,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5408},{"shards":"0,2","objects":3211}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11071","reported_seq":11190,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:58.494937+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:58.494937+0000","last_peered":"2024-09-15T21:24:58.494937+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:58.494937+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39207395,"num_objects":9573,"num_object_clones":0,"num_object_copies":19146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9573,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8930,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3343},{"shards":"2","objects":5483}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11052","reported_seq":19511,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:58.690924+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:58.690924+0000","last_peered":"2024-09-15T21:24:58.690924+0000","last_clean":"2024-09-15T21:24:58.690924+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:58.690924+0000","last_undegraded":"2024-09-15T21:24:58.690924+0000","last_fullsized":"2024-09-15T21:24:58.690924+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38944768,"num_objects":9508,"num_object_clones":0,"num_object_copies":19016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11195","reported_seq":11329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.052271+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:02.052271+0000","last_peered":"2024-09-15T21:25:02.052271+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:02.052271+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38875136,"num_objects":9491,"num_object_clones":0,"num_object_copies":18982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8139,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5397},{"shards":"0,2","objects":3243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11137","reported_seq":11270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.052189+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:02.052189+0000","last_peered":"2024-09-15T21:25:02.052189+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:02.052189+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38719488,"num_objects":9453,"num_object_clones":0,"num_object_copies":18906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5314},{"shards":"0,2","objects":3301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11042","reported_seq":14055,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:02.034527+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:02.034527+0000","last_peered":"2024-09-15T21:25:02.034527+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:02.034527+0000","last_undegraded":"2024-09-15T21:25:02.034527+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38281216,"num_objects":9346,"num_object_clones":0,"num_object_copies":18692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2644,"num_objects_unfound":0,"num_objects_dirty":9346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7526,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1438,"num_bytes_recovered":5890048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6702},{"shards":"0,2","objects":2644}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11308","reported_seq":11445,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.036884+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:02.036884+0000","last_peered":"2024-09-15T21:25:02.036884+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:02.036884+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39150534,"num_objects":9560,"num_object_clones":0,"num_object_copies":19120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8012,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5413},{"shards":"0,2","objects":3278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":623829750,"num_objects":152312,"num_object_clones":0,"num_object_copies":304624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2644,"num_objects_unfound":0,"num_objects_dirty":152312,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":138143,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5603,"num_bytes_recovered":22941696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1304457216,"data_stored":1304300939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160950,"ondisk_log_size":160950,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718544,"kb_used_data":438336,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93653296,"statfs":{"total":96636764160,"available":95900975104,"internally_reserved":0,"allocated":448856064,"data_stored":448580417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":286912169},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865188,"kb_used_data":522580,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93506652,"statfs":{"total":96636764160,"available":95750811648,"internally_reserved":0,"allocated":535121920,"data_stored":534846268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608048,"kb_used_data":315356,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93763792,"statfs":{"total":96636764160,"available":96014123008,"internally_reserved":0,"allocated":322924544,"data_stored":322605077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":299691695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,4,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":533803008,"data_stored":533766878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322527232,"data_stored":322447287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448126976,"data_stored":448086774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:02.968 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:02.968 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11398", 'reported_seq': 11474, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:02.052172+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:25:02.052172+0000', 'last_peered': '2024-09-15T21:25:02.052172+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:25:02.052172+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39952384, 'num_objects': 9754, 'num_object_clones': 0, 'num_object_copies': 19508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9754, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11398, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5622}, {'shards': '0,2', 'objects': 3310}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:02.969 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:25:02.969 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'11289", 'reported_seq': 11416, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:58.712441+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:24:58.712441+0000', 'last_peered': '2024-09-15T21:24:58.712441+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:24:58.712441+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40206360, 'num_objects': 9817, 'num_object_clones': 0, 'num_object_copies': 19634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9817, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9389, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3533}, {'shards': '2', 'objects': 5550}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:02.969 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:02.969 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'11206", 'reported_seq': 11342, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:01.999242+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:25:01.999242+0000', 'last_peered': '2024-09-15T21:25:01.999242+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:25:01.999242+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38621667, 'num_objects': 9430, 'num_object_clones': 0, 'num_object_copies': 18860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9430, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7882, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5354}, {'shards': '0,2', 'objects': 3190}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:02.969 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:02.969 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'11195", 'reported_seq': 11331, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:01.921407+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:25:01.921407+0000', 'last_peered': '2024-09-15T21:25:01.921407+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:25:01.921407+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38810566, 'num_objects': 9477, 'num_object_clones': 0, 'num_object_copies': 18954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7506, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5316}, {'shards': '0,2', 'objects': 3304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:02.969 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:25:02.969 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'11028", 'reported_seq': 11155, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:58.670645+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:24:58.670645+0000', 'last_peered': '2024-09-15T21:24:58.670645+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:24:58.670645+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38851043, 'num_objects': 9486, 'num_object_clones': 0, 'num_object_copies': 18972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9486, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8960, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3317}, {'shards': '2', 'objects': 5399}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:02.970 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:02.970 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11054", 'reported_seq': 11185, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:01.997355+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:25:01.997355+0000', 'last_peered': '2024-09-15T21:25:01.997355+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:25:01.997355+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38617088, 'num_objects': 9428, 'num_object_clones': 0, 'num_object_copies': 18856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7727, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5408}, {'shards': '0,2', 'objects': 3211}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:02.970 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:02.970 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'11071", 'reported_seq': 11190, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:24:58.494937+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:24:58.494937+0000', 'last_peered': '2024-09-15T21:24:58.494937+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:24:58.494937+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39207395, 'num_objects': 9573, 'num_object_clones': 0, 'num_object_copies': 19146, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9573, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8930, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3343}, {'shards': '2', 'objects': 5483}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:02.971 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:02.971 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11195", 'reported_seq': 11329, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:02.052271+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:25:02.052271+0000', 'last_peered': '2024-09-15T21:25:02.052271+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:25:02.052271+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38875136, 'num_objects': 9491, 'num_object_clones': 0, 'num_object_copies': 18982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9491, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8139, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5397}, {'shards': '0,2', 'objects': 3243}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:02.971 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:25:02.971 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11137", 'reported_seq': 11270, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:02.052189+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:25:02.052189+0000', 'last_peered': '2024-09-15T21:25:02.052189+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:25:02.052189+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38719488, 'num_objects': 9453, 'num_object_clones': 0, 'num_object_copies': 18906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9453, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8046, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5314}, {'shards': '0,2', 'objects': 3301}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:02.971 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:02.971 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11042", 'reported_seq': 14055, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:25:02.034527+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:25:02.034527+0000', 'last_peered': '2024-09-15T21:25:02.034527+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:25:02.034527+0000', 'last_undegraded': '2024-09-15T21:25:02.034527+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38281216, 'num_objects': 9346, 'num_object_clones': 0, 'num_object_copies': 18692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2644, 'num_objects_unfound': 0, 'num_objects_dirty': 9346, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7526, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1438, 'num_bytes_recovered': 5890048, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6702}, {'shards': '0,2', 'objects': 2644}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:02.972 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:25:02.972 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11308", 'reported_seq': 11445, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:02.036884+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:25:02.036884+0000', 'last_peered': '2024-09-15T21:25:02.036884+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:25:02.036884+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39150534, 'num_objects': 9560, 'num_object_clones': 0, 'num_object_copies': 19120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9560, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8012, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5413}, {'shards': '0,2', 'objects': 3278}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:02.972 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:03.665 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:03.665 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/304628 objects degraded (13.533%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":355,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":152314,"data_bytes":623829769,"bytes_used":2244382720,"bytes_avail":287665909760,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":304628,"degraded_ratio":0.13532899142560761,"misplaced_objects":2644,"misplaced_total":304628,"misplaced_ratio":0.0086794385283033727,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":88043,"recovering_keys_per_sec":0,"num_objects_recovered":258,"num_bytes_recovered":1056768,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":495},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:25:03.689 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:03.689 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:04.254 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:04.256 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:04.282 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-15T21:25:02.766332+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":623829769,"num_objects":152314,"num_object_clones":0,"num_object_copies":304628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2644,"num_objects_unfound":0,"num_objects_dirty":152314,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":138145,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5604,"num_bytes_recovered":22941715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160953,"ondisk_log_size":160953,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2191780,"kb_used_data":1276272,"kb_used_omap":12,"kb_used_meta":915443,"kb_avail":280923740,"statfs":{"total":289910292480,"available":287665909760,"internally_reserved":0,"allocated":1306902528,"data_stored":1306031762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":937413635},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,5,11,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24338432,"num_objects":-5942,"num_object_clones":0,"num_object_copies":-11884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-258,"num_objects_unfound":0,"num_objects_dirty":-5942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5942,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":1056768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002850"},"pg_stats":[{"pgid":"2.f","version":"45'11398","reported_seq":11474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.052172+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:02.052172+0000","last_peered":"2024-09-15T21:25:02.052172+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:02.052172+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39952384,"num_objects":9754,"num_object_clones":0,"num_object_copies":19508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11398,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5622},{"shards":"0,2","objects":3310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11085","reported_seq":11190,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:58.593462+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:24:58.593462+0000","last_peered":"2024-09-15T21:24:58.593462+0000","last_clean":"2024-09-15T21:24:58.593462+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:24:58.593462+0000","last_undegraded":"2024-09-15T21:24:58.593462+0000","last_fullsized":"2024-09-15T21:24:58.593462+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39277510,"num_objects":9591,"num_object_clones":0,"num_object_copies":19182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9591,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9366,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11289","reported_seq":11416,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:58.712441+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:24:58.712441+0000","last_peered":"2024-09-15T21:24:58.712441+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:24:58.712441+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40206360,"num_objects":9817,"num_object_clones":0,"num_object_copies":19634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9817,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9389,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3533},{"shards":"2","objects":5550}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10907","reported_seq":10978,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:58.571112+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:24:58.571112+0000","last_peered":"2024-09-15T21:24:58.571112+0000","last_clean":"2024-09-15T21:24:58.571112+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:24:58.571112+0000","last_undegraded":"2024-09-15T21:24:58.571112+0000","last_fullsized":"2024-09-15T21:24:58.571112+0000","mapping_epoch":35,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38531072,"num_objects":9407,"num_object_clones":0,"num_object_copies":18814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10907,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11092","reported_seq":11225,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:01.979397+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:01.979397+0000","last_peered":"2024-09-15T21:25:01.979397+0000","last_clean":"2024-09-15T21:25:01.979397+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:01.979397+0000","last_undegraded":"2024-09-15T21:25:01.979397+0000","last_fullsized":"2024-09-15T21:25:01.979397+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38556131,"num_objects":9414,"num_object_clones":0,"num_object_copies":18828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7547,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11206","reported_seq":11342,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:01.999242+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:01.999242+0000","last_peered":"2024-09-15T21:25:01.999242+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:01.999242+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38621667,"num_objects":9430,"num_object_clones":0,"num_object_copies":18860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9430,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7882,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":3190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11195","reported_seq":11331,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:01.921407+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:01.921407+0000","last_peered":"2024-09-15T21:25:01.921407+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:01.921407+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38810566,"num_objects":9477,"num_object_clones":0,"num_object_copies":18954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7506,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5316},{"shards":"0,2","objects":3304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11291","reported_seq":11420,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:01.876247+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:01.876247+0000","last_peered":"2024-09-15T21:25:01.876247+0000","last_clean":"2024-09-15T21:25:01.876247+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:01.876247+0000","last_undegraded":"2024-09-15T21:25:01.876247+0000","last_fullsized":"2024-09-15T21:25:01.876247+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227392,"num_objects":9577,"num_object_clones":0,"num_object_copies":19154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8063,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11028","reported_seq":11155,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:58.670645+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:24:58.670645+0000","last_peered":"2024-09-15T21:24:58.670645+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:24:58.670645+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38851043,"num_objects":9486,"num_object_clones":0,"num_object_copies":18972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9486,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8960,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3317},{"shards":"2","objects":5399}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11054","reported_seq":11185,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:01.997355+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:01.997355+0000","last_peered":"2024-09-15T21:25:01.997355+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:01.997355+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38617088,"num_objects":9428,"num_object_clones":0,"num_object_copies":18856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7727,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5408},{"shards":"0,2","objects":3211}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11071","reported_seq":11190,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:24:58.494937+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:24:58.494937+0000","last_peered":"2024-09-15T21:24:58.494937+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:24:58.494937+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39207395,"num_objects":9573,"num_object_clones":0,"num_object_copies":19146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9573,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8930,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3343},{"shards":"2","objects":5483}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11052","reported_seq":19511,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:58.690924+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:24:58.690924+0000","last_peered":"2024-09-15T21:24:58.690924+0000","last_clean":"2024-09-15T21:24:58.690924+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:24:58.690924+0000","last_undegraded":"2024-09-15T21:24:58.690924+0000","last_fullsized":"2024-09-15T21:24:58.690924+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38944768,"num_objects":9508,"num_object_clones":0,"num_object_copies":19016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11195","reported_seq":11329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.052271+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:02.052271+0000","last_peered":"2024-09-15T21:25:02.052271+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:02.052271+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38875136,"num_objects":9491,"num_object_clones":0,"num_object_copies":18982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8139,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5397},{"shards":"0,2","objects":3243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11137","reported_seq":11270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.052189+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:02.052189+0000","last_peered":"2024-09-15T21:25:02.052189+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:02.052189+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38719488,"num_objects":9453,"num_object_clones":0,"num_object_copies":18906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5314},{"shards":"0,2","objects":3301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11042","reported_seq":14055,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:02.034527+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:02.034527+0000","last_peered":"2024-09-15T21:25:02.034527+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:02.034527+0000","last_undegraded":"2024-09-15T21:25:02.034527+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38281216,"num_objects":9346,"num_object_clones":0,"num_object_copies":18692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2644,"num_objects_unfound":0,"num_objects_dirty":9346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7526,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1438,"num_bytes_recovered":5890048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6702},{"shards":"0,2","objects":2644}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11308","reported_seq":11445,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.036884+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:02.036884+0000","last_peered":"2024-09-15T21:25:02.036884+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:02.036884+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39150534,"num_objects":9560,"num_object_clones":0,"num_object_copies":19120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8012,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5413},{"shards":"0,2","objects":3278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":623829750,"num_objects":152312,"num_object_clones":0,"num_object_copies":304624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2644,"num_objects_unfound":0,"num_objects_dirty":152312,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":138143,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5603,"num_bytes_recovered":22941696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1304457216,"data_stored":1304300939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160950,"ondisk_log_size":160950,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718544,"kb_used_data":438336,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93653296,"statfs":{"total":96636764160,"available":95900975104,"internally_reserved":0,"allocated":448856064,"data_stored":448580417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":286912169},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865188,"kb_used_data":522580,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93506652,"statfs":{"total":96636764160,"available":95750811648,"internally_reserved":0,"allocated":535121920,"data_stored":534846268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608048,"kb_used_data":315356,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93763792,"statfs":{"total":96636764160,"available":96014123008,"internally_reserved":0,"allocated":322924544,"data_stored":322605077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":299691695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,4,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":533803008,"data_stored":533766878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":322527232,"data_stored":322447287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":448126976,"data_stored":448086774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:07.285 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:07.864 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:07.864 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:07.896 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-15T21:25:06.767297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620282633,"num_objects":151448,"num_object_clones":0,"num_object_copies":302896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2644,"num_objects_unfound":0,"num_objects_dirty":151448,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":139011,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5604,"num_bytes_recovered":22941715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2178124,"kb_used_data":1270940,"kb_used_omap":12,"kb_used_meta":907123,"kb_avail":280937396,"statfs":{"total":289910292480,"available":287679893504,"internally_reserved":0,"allocated":1301442560,"data_stored":1300571794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":928893955},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,6,14,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18644992,"num_objects":-4552,"num_object_clones":0,"num_object_copies":-9104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-177,"num_objects_unfound":0,"num_objects_dirty":-4552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4552,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":724992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002809"},"pg_stats":[{"pgid":"2.f","version":"45'11398","reported_seq":11474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.052172+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:02.052172+0000","last_peered":"2024-09-15T21:25:02.052172+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:02.052172+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39952384,"num_objects":9754,"num_object_clones":0,"num_object_copies":19508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11398,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5622},{"shards":"0,2","objects":3310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11221","reported_seq":11325,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.726149+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:03.726149+0000","last_peered":"2024-09-15T21:25:03.726149+0000","last_clean":"2024-09-15T21:25:03.726149+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:03.726149+0000","last_undegraded":"2024-09-15T21:25:03.726149+0000","last_fullsized":"2024-09-15T21:25:03.726149+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38720454,"num_objects":9455,"num_object_clones":0,"num_object_copies":18910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9455,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9502,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11448","reported_seq":11572,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:03.699301+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:03.699301+0000","last_peered":"2024-09-15T21:25:03.699301+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:03.699301+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39555096,"num_objects":9658,"num_object_clones":0,"num_object_copies":19316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9548,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3374},{"shards":"2","objects":5391}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11049","reported_seq":11120,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.660199+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:03.660199+0000","last_peered":"2024-09-15T21:25:03.660199+0000","last_clean":"2024-09-15T21:25:03.660199+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:03.660199+0000","last_undegraded":"2024-09-15T21:25:03.660199+0000","last_fullsized":"2024-09-15T21:25:03.660199+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37949440,"num_objects":9265,"num_object_clones":0,"num_object_copies":18530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11049,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11092","reported_seq":11225,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:01.979397+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:01.979397+0000","last_peered":"2024-09-15T21:25:01.979397+0000","last_clean":"2024-09-15T21:25:01.979397+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:01.979397+0000","last_undegraded":"2024-09-15T21:25:01.979397+0000","last_fullsized":"2024-09-15T21:25:01.979397+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38556131,"num_objects":9414,"num_object_clones":0,"num_object_copies":18828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7547,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11206","reported_seq":11342,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:01.999242+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:01.999242+0000","last_peered":"2024-09-15T21:25:01.999242+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:01.999242+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38621667,"num_objects":9430,"num_object_clones":0,"num_object_copies":18860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9430,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7882,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":3190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11195","reported_seq":11331,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:01.921407+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:01.921407+0000","last_peered":"2024-09-15T21:25:01.921407+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:01.921407+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38810566,"num_objects":9477,"num_object_clones":0,"num_object_copies":18954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7506,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5316},{"shards":"0,2","objects":3304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11291","reported_seq":11420,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:01.876247+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:01.876247+0000","last_peered":"2024-09-15T21:25:01.876247+0000","last_clean":"2024-09-15T21:25:01.876247+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:01.876247+0000","last_undegraded":"2024-09-15T21:25:01.876247+0000","last_fullsized":"2024-09-15T21:25:01.876247+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227392,"num_objects":9577,"num_object_clones":0,"num_object_copies":19154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8063,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11160","reported_seq":11287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:03.702298+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:03.702298+0000","last_peered":"2024-09-15T21:25:03.702298+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:03.702298+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38310371,"num_objects":9354,"num_object_clones":0,"num_object_copies":18708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9354,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9092,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3185},{"shards":"2","objects":5267}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11054","reported_seq":11185,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:01.997355+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:01.997355+0000","last_peered":"2024-09-15T21:25:01.997355+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:01.997355+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38617088,"num_objects":9428,"num_object_clones":0,"num_object_copies":18856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7727,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5408},{"shards":"0,2","objects":3211}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11207","reported_seq":11326,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:03.689673+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:03.689673+0000","last_peered":"2024-09-15T21:25:03.689673+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:03.689673+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38650339,"num_objects":9437,"num_object_clones":0,"num_object_copies":18874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9066,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3207},{"shards":"2","objects":5347}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11213","reported_seq":19672,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.610500+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:03.610500+0000","last_peered":"2024-09-15T21:25:03.610500+0000","last_clean":"2024-09-15T21:25:03.610500+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:03.610500+0000","last_undegraded":"2024-09-15T21:25:03.610500+0000","last_fullsized":"2024-09-15T21:25:03.610500+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38285312,"num_objects":9347,"num_object_clones":0,"num_object_copies":18694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8906,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11195","reported_seq":11329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.052271+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:02.052271+0000","last_peered":"2024-09-15T21:25:02.052271+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:02.052271+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38875136,"num_objects":9491,"num_object_clones":0,"num_object_copies":18982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8139,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5397},{"shards":"0,2","objects":3243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11137","reported_seq":11270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.052189+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:02.052189+0000","last_peered":"2024-09-15T21:25:02.052189+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:02.052189+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38719488,"num_objects":9453,"num_object_clones":0,"num_object_copies":18906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5314},{"shards":"0,2","objects":3301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11042","reported_seq":14055,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:02.034527+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:02.034527+0000","last_peered":"2024-09-15T21:25:02.034527+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:02.034527+0000","last_undegraded":"2024-09-15T21:25:02.034527+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38281216,"num_objects":9346,"num_object_clones":0,"num_object_copies":18692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2644,"num_objects_unfound":0,"num_objects_dirty":9346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7526,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1438,"num_bytes_recovered":5890048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6702},{"shards":"0,2","objects":2644}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11308","reported_seq":11445,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.036884+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:02.036884+0000","last_peered":"2024-09-15T21:25:02.036884+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:02.036884+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39150534,"num_objects":9560,"num_object_clones":0,"num_object_copies":19120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8012,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5413},{"shards":"0,2","objects":3278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":620282614,"num_objects":151446,"num_object_clones":0,"num_object_copies":302892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2644,"num_objects_unfound":0,"num_objects_dirty":151446,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":139009,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5603,"num_bytes_recovered":22941696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1298927616,"data_stored":1298771339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160816,"ondisk_log_size":160816,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718196,"kb_used_data":433828,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93653644,"statfs":{"total":96636764160,"available":95901331456,"internally_reserved":0,"allocated":444239872,"data_stored":443964225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":291172009},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865188,"kb_used_data":522580,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93506652,"statfs":{"total":96636764160,"available":95750811648,"internally_reserved":0,"allocated":535121920,"data_stored":534846268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594740,"kb_used_data":314532,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93777100,"statfs":{"total":96636764160,"available":96027750400,"internally_reserved":0,"allocated":322080768,"data_stored":321761301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":533803008,"data_stored":533766878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321675264,"data_stored":321595319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":443449344,"data_stored":443409142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:07.901 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:08.482 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:08.482 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:08.512 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-15T21:25:06.767297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620282633,"num_objects":151448,"num_object_clones":0,"num_object_copies":302896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2644,"num_objects_unfound":0,"num_objects_dirty":151448,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":139011,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5604,"num_bytes_recovered":22941715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2178124,"kb_used_data":1270940,"kb_used_omap":12,"kb_used_meta":907123,"kb_avail":280937396,"statfs":{"total":289910292480,"available":287679893504,"internally_reserved":0,"allocated":1301442560,"data_stored":1300571794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":928893955},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,6,14,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18644992,"num_objects":-4552,"num_object_clones":0,"num_object_copies":-9104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-177,"num_objects_unfound":0,"num_objects_dirty":-4552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4552,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":724992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002809"},"pg_stats":[{"pgid":"2.f","version":"45'11398","reported_seq":11474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.052172+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:02.052172+0000","last_peered":"2024-09-15T21:25:02.052172+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:02.052172+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39952384,"num_objects":9754,"num_object_clones":0,"num_object_copies":19508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11398,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5622},{"shards":"0,2","objects":3310}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11221","reported_seq":11325,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.726149+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:03.726149+0000","last_peered":"2024-09-15T21:25:03.726149+0000","last_clean":"2024-09-15T21:25:03.726149+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:03.726149+0000","last_undegraded":"2024-09-15T21:25:03.726149+0000","last_fullsized":"2024-09-15T21:25:03.726149+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38720454,"num_objects":9455,"num_object_clones":0,"num_object_copies":18910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9455,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9502,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11448","reported_seq":11572,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:03.699301+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:03.699301+0000","last_peered":"2024-09-15T21:25:03.699301+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:03.699301+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39555096,"num_objects":9658,"num_object_clones":0,"num_object_copies":19316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9658,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9548,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3374},{"shards":"2","objects":5391}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11049","reported_seq":11120,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.660199+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:03.660199+0000","last_peered":"2024-09-15T21:25:03.660199+0000","last_clean":"2024-09-15T21:25:03.660199+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:03.660199+0000","last_undegraded":"2024-09-15T21:25:03.660199+0000","last_fullsized":"2024-09-15T21:25:03.660199+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37949440,"num_objects":9265,"num_object_clones":0,"num_object_copies":18530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11049,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11092","reported_seq":11225,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:01.979397+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:01.979397+0000","last_peered":"2024-09-15T21:25:01.979397+0000","last_clean":"2024-09-15T21:25:01.979397+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:01.979397+0000","last_undegraded":"2024-09-15T21:25:01.979397+0000","last_fullsized":"2024-09-15T21:25:01.979397+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38556131,"num_objects":9414,"num_object_clones":0,"num_object_copies":18828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7547,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11206","reported_seq":11342,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:01.999242+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:01.999242+0000","last_peered":"2024-09-15T21:25:01.999242+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:01.999242+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38621667,"num_objects":9430,"num_object_clones":0,"num_object_copies":18860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9430,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7882,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":3190}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11195","reported_seq":11331,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:01.921407+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:01.921407+0000","last_peered":"2024-09-15T21:25:01.921407+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:01.921407+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38810566,"num_objects":9477,"num_object_clones":0,"num_object_copies":18954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7506,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5316},{"shards":"0,2","objects":3304}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11291","reported_seq":11420,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:01.876247+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:01.876247+0000","last_peered":"2024-09-15T21:25:01.876247+0000","last_clean":"2024-09-15T21:25:01.876247+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:01.876247+0000","last_undegraded":"2024-09-15T21:25:01.876247+0000","last_fullsized":"2024-09-15T21:25:01.876247+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39227392,"num_objects":9577,"num_object_clones":0,"num_object_copies":19154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8063,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11160","reported_seq":11287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:03.702298+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:03.702298+0000","last_peered":"2024-09-15T21:25:03.702298+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:03.702298+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38310371,"num_objects":9354,"num_object_clones":0,"num_object_copies":18708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9354,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9092,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3185},{"shards":"2","objects":5267}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11054","reported_seq":11185,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:01.997355+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:01.997355+0000","last_peered":"2024-09-15T21:25:01.997355+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:01.997355+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38617088,"num_objects":9428,"num_object_clones":0,"num_object_copies":18856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7727,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5408},{"shards":"0,2","objects":3211}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11207","reported_seq":11326,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:03.689673+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:03.689673+0000","last_peered":"2024-09-15T21:25:03.689673+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:03.689673+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38650339,"num_objects":9437,"num_object_clones":0,"num_object_copies":18874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9437,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9066,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3207},{"shards":"2","objects":5347}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11213","reported_seq":19672,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:03.610500+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:03.610500+0000","last_peered":"2024-09-15T21:25:03.610500+0000","last_clean":"2024-09-15T21:25:03.610500+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:03.610500+0000","last_undegraded":"2024-09-15T21:25:03.610500+0000","last_fullsized":"2024-09-15T21:25:03.610500+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38285312,"num_objects":9347,"num_object_clones":0,"num_object_copies":18694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8906,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11195","reported_seq":11329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.052271+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:02.052271+0000","last_peered":"2024-09-15T21:25:02.052271+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:02.052271+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38875136,"num_objects":9491,"num_object_clones":0,"num_object_copies":18982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8139,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5397},{"shards":"0,2","objects":3243}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11137","reported_seq":11270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.052189+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:02.052189+0000","last_peered":"2024-09-15T21:25:02.052189+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:02.052189+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38719488,"num_objects":9453,"num_object_clones":0,"num_object_copies":18906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8046,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5314},{"shards":"0,2","objects":3301}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11042","reported_seq":14055,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:02.034527+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:02.034527+0000","last_peered":"2024-09-15T21:25:02.034527+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:02.034527+0000","last_undegraded":"2024-09-15T21:25:02.034527+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38281216,"num_objects":9346,"num_object_clones":0,"num_object_copies":18692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2644,"num_objects_unfound":0,"num_objects_dirty":9346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7526,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1438,"num_bytes_recovered":5890048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6702},{"shards":"0,2","objects":2644}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11308","reported_seq":11445,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:02.036884+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:02.036884+0000","last_peered":"2024-09-15T21:25:02.036884+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:02.036884+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39150534,"num_objects":9560,"num_object_clones":0,"num_object_copies":19120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8012,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5413},{"shards":"0,2","objects":3278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":620282614,"num_objects":151446,"num_object_clones":0,"num_object_copies":302892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2644,"num_objects_unfound":0,"num_objects_dirty":151446,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":139009,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5603,"num_bytes_recovered":22941696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1298927616,"data_stored":1298771339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160816,"ondisk_log_size":160816,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718196,"kb_used_data":433828,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93653644,"statfs":{"total":96636764160,"available":95901331456,"internally_reserved":0,"allocated":444239872,"data_stored":443964225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":291172009},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":865188,"kb_used_data":522580,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93506652,"statfs":{"total":96636764160,"available":95750811648,"internally_reserved":0,"allocated":535121920,"data_stored":534846268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594740,"kb_used_data":314532,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93777100,"statfs":{"total":96636764160,"available":96027750400,"internally_reserved":0,"allocated":322080768,"data_stored":321761301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":533803008,"data_stored":533766878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":321675264,"data_stored":321595319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":443449344,"data_stored":443409142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:08.516 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:08.516 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11398", 'reported_seq': 11474, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:02.052172+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:25:02.052172+0000', 'last_peered': '2024-09-15T21:25:02.052172+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:25:02.052172+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39952384, 'num_objects': 9754, 'num_object_clones': 0, 'num_object_copies': 19508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9754, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11398, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5622}, {'shards': '0,2', 'objects': 3310}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:08.517 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:25:08.517 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'11448", 'reported_seq': 11572, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:03.699301+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:25:03.699301+0000', 'last_peered': '2024-09-15T21:25:03.699301+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:25:03.699301+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39555096, 'num_objects': 9658, 'num_object_clones': 0, 'num_object_copies': 19316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9658, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9548, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3374}, {'shards': '2', 'objects': 5391}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:08.517 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:08.517 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'11206", 'reported_seq': 11342, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:01.999242+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:25:01.999242+0000', 'last_peered': '2024-09-15T21:25:01.999242+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:25:01.999242+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38621667, 'num_objects': 9430, 'num_object_clones': 0, 'num_object_copies': 18860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9430, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 7882, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5354}, {'shards': '0,2', 'objects': 3190}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:08.517 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:08.517 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'11195", 'reported_seq': 11331, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:01.921407+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:25:01.921407+0000', 'last_peered': '2024-09-15T21:25:01.921407+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:25:01.921407+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38810566, 'num_objects': 9477, 'num_object_clones': 0, 'num_object_copies': 18954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7506, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5316}, {'shards': '0,2', 'objects': 3304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:08.517 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:25:08.517 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'11160", 'reported_seq': 11287, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:03.702298+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:25:03.702298+0000', 'last_peered': '2024-09-15T21:25:03.702298+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:25:03.702298+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38310371, 'num_objects': 9354, 'num_object_clones': 0, 'num_object_copies': 18708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9354, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9092, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3185}, {'shards': '2', 'objects': 5267}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:08.518 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:08.518 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11054", 'reported_seq': 11185, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:01.997355+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:25:01.997355+0000', 'last_peered': '2024-09-15T21:25:01.997355+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:25:01.997355+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38617088, 'num_objects': 9428, 'num_object_clones': 0, 'num_object_copies': 18856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7727, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5408}, {'shards': '0,2', 'objects': 3211}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:08.518 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:08.518 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'11207", 'reported_seq': 11326, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:03.689673+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:25:03.689673+0000', 'last_peered': '2024-09-15T21:25:03.689673+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:25:03.689673+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38650339, 'num_objects': 9437, 'num_object_clones': 0, 'num_object_copies': 18874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9437, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9066, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3207}, {'shards': '2', 'objects': 5347}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:08.518 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:08.518 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11195", 'reported_seq': 11329, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:02.052271+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:25:02.052271+0000', 'last_peered': '2024-09-15T21:25:02.052271+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:25:02.052271+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38875136, 'num_objects': 9491, 'num_object_clones': 0, 'num_object_copies': 18982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9491, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8139, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5397}, {'shards': '0,2', 'objects': 3243}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:08.519 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:25:08.519 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11137", 'reported_seq': 11270, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:02.052189+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:25:02.052189+0000', 'last_peered': '2024-09-15T21:25:02.052189+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:25:02.052189+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38719488, 'num_objects': 9453, 'num_object_clones': 0, 'num_object_copies': 18906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9453, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8046, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5314}, {'shards': '0,2', 'objects': 3301}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:08.519 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:08.519 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11042", 'reported_seq': 14055, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:25:02.034527+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:25:02.034527+0000', 'last_peered': '2024-09-15T21:25:02.034527+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:25:02.034527+0000', 'last_undegraded': '2024-09-15T21:25:02.034527+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38281216, 'num_objects': 9346, 'num_object_clones': 0, 'num_object_copies': 18692, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2644, 'num_objects_unfound': 0, 'num_objects_dirty': 9346, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7526, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1438, 'num_bytes_recovered': 5890048, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6702}, {'shards': '0,2', 'objects': 2644}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:08.519 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:25:08.519 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11308", 'reported_seq': 11445, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:02.036884+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:25:02.036884+0000', 'last_peered': '2024-09-15T21:25:02.036884+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:25:02.036884+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39150534, 'num_objects': 9560, 'num_object_clones': 0, 'num_object_copies': 19120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9560, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8012, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5413}, {'shards': '0,2', 'objects': 3278}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:08.519 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:09.246 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:09.246 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/302896 objects degraded (13.610%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":360,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":151448,"data_bytes":620282633,"bytes_used":2230398976,"bytes_avail":287679893504,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":302896,"degraded_ratio":0.13610282077016533,"misplaced_objects":2644,"misplaced_total":302896,"misplaced_ratio":0.0087290687232581474,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":60401,"recovering_keys_per_sec":0,"num_objects_recovered":177,"num_bytes_recovered":724992,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":379},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:25:09.279 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:09.279 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:09.830 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:09.831 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:09.862 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":218,"stamp":"2024-09-15T21:25:08.767722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":611132169,"num_objects":149214,"num_object_clones":0,"num_object_copies":298428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2552,"num_objects_unfound":0,"num_objects_dirty":149214,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":141245,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5696,"num_bytes_recovered":23318547,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2200752,"kb_used_data":1258868,"kb_used_omap":12,"kb_used_meta":941811,"kb_avail":280914768,"statfs":{"total":289910292480,"available":287656722432,"internally_reserved":0,"allocated":1289080832,"data_stored":1288210066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":964414467},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,7,0,4,9,7,10,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27795456,"num_objects":-6786,"num_object_clones":0,"num_object_copies":-13572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-269,"num_objects_unfound":0,"num_objects_dirty":-6786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6786,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":1101824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002779"},"pg_stats":[{"pgid":"2.f","version":"45'11553","reported_seq":11629,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:07.097107+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:07.097107+0000","last_peered":"2024-09-15T21:25:07.097107+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:07.097107+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39317504,"num_objects":9599,"num_object_clones":0,"num_object_copies":19198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11553,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5467},{"shards":"0,2","objects":3155}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11355","reported_seq":11460,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:08.604150+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:08.604150+0000","last_peered":"2024-09-15T21:25:08.604150+0000","last_clean":"2024-09-15T21:25:08.604150+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:08.604150+0000","last_undegraded":"2024-09-15T21:25:08.604150+0000","last_fullsized":"2024-09-15T21:25:08.604150+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38171590,"num_objects":9321,"num_object_clones":0,"num_object_copies":18642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9321,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9636,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11591","reported_seq":11718,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:08.614853+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:08.614853+0000","last_peered":"2024-09-15T21:25:08.614853+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:08.614853+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38969368,"num_objects":9515,"num_object_clones":0,"num_object_copies":19030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9691,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3231},{"shards":"2","objects":5248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11200","reported_seq":11271,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:08.558506+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:08.558506+0000","last_peered":"2024-09-15T21:25:08.558506+0000","last_clean":"2024-09-15T21:25:08.558506+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:08.558506+0000","last_undegraded":"2024-09-15T21:25:08.558506+0000","last_fullsized":"2024-09-15T21:25:08.558506+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37330944,"num_objects":9114,"num_object_clones":0,"num_object_copies":18228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11200,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11244","reported_seq":11377,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:07.006838+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:07.006838+0000","last_peered":"2024-09-15T21:25:07.006838+0000","last_clean":"2024-09-15T21:25:07.006838+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:07.006838+0000","last_undegraded":"2024-09-15T21:25:07.006838+0000","last_fullsized":"2024-09-15T21:25:07.006838+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37933539,"num_objects":9262,"num_object_clones":0,"num_object_copies":18524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9262,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7699,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11337","reported_seq":11473,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:07.079632+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:07.079632+0000","last_peered":"2024-09-15T21:25:07.079632+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:07.079632+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38085091,"num_objects":9299,"num_object_clones":0,"num_object_copies":18598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8013,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5223},{"shards":"0,2","objects":3059}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11322","reported_seq":11458,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:07.057599+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:07.057599+0000","last_peered":"2024-09-15T21:25:07.057599+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:07.057599+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38290374,"num_objects":9350,"num_object_clones":0,"num_object_copies":18700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7633,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5189},{"shards":"0,2","objects":3177}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11432","reported_seq":11561,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:07.059414+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:07.059414+0000","last_peered":"2024-09-15T21:25:07.059414+0000","last_clean":"2024-09-15T21:25:07.059414+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:07.059414+0000","last_undegraded":"2024-09-15T21:25:07.059414+0000","last_fullsized":"2024-09-15T21:25:07.059414+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38649856,"num_objects":9436,"num_object_clones":0,"num_object_copies":18872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8204,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11319","reported_seq":11445,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:08.731461+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:08.731461+0000","last_peered":"2024-09-15T21:25:08.731461+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:08.731461+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37659107,"num_objects":9195,"num_object_clones":0,"num_object_copies":18390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9251,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3026},{"shards":"2","objects":5108}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11184","reported_seq":11315,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:06.996946+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:06.996946+0000","last_peered":"2024-09-15T21:25:06.996946+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:06.996946+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38084608,"num_objects":9298,"num_object_clones":0,"num_object_copies":18596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7857,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5278},{"shards":"0,2","objects":3081}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11342","reported_seq":11461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:08.593539+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:08.593539+0000","last_peered":"2024-09-15T21:25:08.593539+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:08.593539+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38097379,"num_objects":9302,"num_object_clones":0,"num_object_copies":18604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9302,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9201,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3072},{"shards":"2","objects":5212}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11347","reported_seq":19806,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:08.531421+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:08.531421+0000","last_peered":"2024-09-15T21:25:08.531421+0000","last_clean":"2024-09-15T21:25:08.531421+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:08.531421+0000","last_undegraded":"2024-09-15T21:25:08.531421+0000","last_fullsized":"2024-09-15T21:25:08.531421+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37736448,"num_objects":9213,"num_object_clones":0,"num_object_copies":18426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9040,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11323","reported_seq":11457,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:07.030475+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:07.030475+0000","last_peered":"2024-09-15T21:25:07.030475+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:07.030475+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38350848,"num_objects":9363,"num_object_clones":0,"num_object_copies":18726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8267,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5269},{"shards":"0,2","objects":3115}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11285","reported_seq":11418,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:07.127764+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:07.127764+0000","last_peered":"2024-09-15T21:25:07.127764+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:07.127764+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38113280,"num_objects":9305,"num_object_clones":0,"num_object_copies":18610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8194,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5166},{"shards":"0,2","objects":3153}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11169","reported_seq":14366,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:07.124026+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:07.124026+0000","last_peered":"2024-09-15T21:25:07.124026+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:07.124026+0000","last_undegraded":"2024-09-15T21:25:07.124026+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37761024,"num_objects":9219,"num_object_clones":0,"num_object_copies":18438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2552,"num_objects_unfound":0,"num_objects_dirty":9219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7653,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1530,"num_bytes_recovered":6266880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6667},{"shards":"0,2","objects":2552}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11447","reported_seq":11584,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:07.039465+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:07.039465+0000","last_peered":"2024-09-15T21:25:07.039465+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:07.039465+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38581190,"num_objects":9421,"num_object_clones":0,"num_object_copies":18842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9421,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8151,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5274},{"shards":"0,2","objects":3139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":611132150,"num_objects":149212,"num_object_clones":0,"num_object_copies":298424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2552,"num_objects_unfound":0,"num_objects_dirty":149212,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":141243,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5695,"num_bytes_recovered":23318528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1286746112,"data_stored":1286589835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735708,"kb_used_data":429132,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93636132,"statfs":{"total":96636764160,"available":95883399168,"internally_reserved":0,"allocated":439431168,"data_stored":439155521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":866828,"kb_used_data":515888,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93505012,"statfs":{"total":96636764160,"available":95749132288,"internally_reserved":0,"allocated":528269312,"data_stored":527993660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359329451},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,6,2,2,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598216,"kb_used_data":313848,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93773624,"statfs":{"total":96636764160,"available":96024190976,"internally_reserved":0,"allocated":321380352,"data_stored":321060885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":527007744,"data_stored":526971614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320966656,"data_stored":320886711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":438771712,"data_stored":438731510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:12.864 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:13.422 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:13.446 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-15T21:25:12.768743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605418249,"num_objects":147819,"num_object_clones":0,"num_object_copies":295638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2462,"num_objects_unfound":0,"num_objects_dirty":147819,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":142640,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5786,"num_bytes_recovered":23687187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2181712,"kb_used_data":1252320,"kb_used_omap":12,"kb_used_meta":929331,"kb_avail":280933808,"statfs":{"total":289910292480,"available":287676219392,"internally_reserved":0,"allocated":1282375680,"data_stored":1281504914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":951634947},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,6,0,3,14,7,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24215552,"num_objects":-5912,"num_object_clones":0,"num_object_copies":-11824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-271,"num_objects_unfound":0,"num_objects_dirty":-5912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5912,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":1110016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002840"},"pg_stats":[{"pgid":"2.f","version":"45'11709","reported_seq":11785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.157546+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:12.157546+0000","last_peered":"2024-09-15T21:25:12.157546+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:12.157546+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38678528,"num_objects":9443,"num_object_clones":0,"num_object_copies":18886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11709,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5311},{"shards":"0,2","objects":2999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11355","reported_seq":11460,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:08.604150+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:08.604150+0000","last_peered":"2024-09-15T21:25:08.604150+0000","last_clean":"2024-09-15T21:25:08.604150+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:08.604150+0000","last_undegraded":"2024-09-15T21:25:08.604150+0000","last_fullsized":"2024-09-15T21:25:08.604150+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38171590,"num_objects":9321,"num_object_clones":0,"num_object_copies":18642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9321,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9636,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11591","reported_seq":11718,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:08.614853+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:08.614853+0000","last_peered":"2024-09-15T21:25:08.614853+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:08.614853+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38969368,"num_objects":9515,"num_object_clones":0,"num_object_copies":19030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9691,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3231},{"shards":"2","objects":5248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11200","reported_seq":11271,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:08.558506+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:08.558506+0000","last_peered":"2024-09-15T21:25:08.558506+0000","last_clean":"2024-09-15T21:25:08.558506+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:08.558506+0000","last_undegraded":"2024-09-15T21:25:08.558506+0000","last_fullsized":"2024-09-15T21:25:08.558506+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37330944,"num_objects":9114,"num_object_clones":0,"num_object_copies":18228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11200,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11371","reported_seq":11504,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:12.214636+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:12.214636+0000","last_peered":"2024-09-15T21:25:12.214636+0000","last_clean":"2024-09-15T21:25:12.214636+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:12.214636+0000","last_undegraded":"2024-09-15T21:25:12.214636+0000","last_fullsized":"2024-09-15T21:25:12.214636+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37413347,"num_objects":9135,"num_object_clones":0,"num_object_copies":18270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7826,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11481","reported_seq":11615,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.225458+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:12.225458+0000","last_peered":"2024-09-15T21:25:12.225458+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:12.225458+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37495267,"num_objects":9155,"num_object_clones":0,"num_object_copies":18310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8157,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5079},{"shards":"0,2","objects":2915}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11455","reported_seq":11591,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.125018+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:12.125018+0000","last_peered":"2024-09-15T21:25:12.125018+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:12.125018+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37745606,"num_objects":9217,"num_object_clones":0,"num_object_copies":18434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7766,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5056},{"shards":"0,2","objects":3044}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11573","reported_seq":11702,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:12.195551+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:12.195551+0000","last_peered":"2024-09-15T21:25:12.195551+0000","last_clean":"2024-09-15T21:25:12.195551+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:12.195551+0000","last_undegraded":"2024-09-15T21:25:12.195551+0000","last_fullsized":"2024-09-15T21:25:12.195551+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072320,"num_objects":9295,"num_object_clones":0,"num_object_copies":18590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8345,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11319","reported_seq":11445,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:08.731461+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:08.731461+0000","last_peered":"2024-09-15T21:25:08.731461+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:08.731461+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37659107,"num_objects":9195,"num_object_clones":0,"num_object_copies":18390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9251,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3026},{"shards":"2","objects":5108}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11322","reported_seq":11453,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.101055+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:12.101055+0000","last_peered":"2024-09-15T21:25:12.101055+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:12.101055+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37519360,"num_objects":9160,"num_object_clones":0,"num_object_copies":18320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7995,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5140},{"shards":"0,2","objects":2943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11342","reported_seq":11461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:08.593539+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:08.593539+0000","last_peered":"2024-09-15T21:25:08.593539+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:08.593539+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38097379,"num_objects":9302,"num_object_clones":0,"num_object_copies":18604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9302,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9201,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3072},{"shards":"2","objects":5212}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11347","reported_seq":19806,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:08.531421+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:08.531421+0000","last_peered":"2024-09-15T21:25:08.531421+0000","last_clean":"2024-09-15T21:25:08.531421+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:08.531421+0000","last_undegraded":"2024-09-15T21:25:08.531421+0000","last_fullsized":"2024-09-15T21:25:08.531421+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37736448,"num_objects":9213,"num_object_clones":0,"num_object_copies":18426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9040,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11451","reported_seq":11584,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.175618+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:12.175618+0000","last_peered":"2024-09-15T21:25:12.175618+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:12.175618+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37826560,"num_objects":9235,"num_object_clones":0,"num_object_copies":18470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8395,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5141},{"shards":"0,2","objects":2987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11428","reported_seq":11561,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.083206+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:12.083206+0000","last_peered":"2024-09-15T21:25:12.083206+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:12.083206+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37527552,"num_objects":9162,"num_object_clones":0,"num_object_copies":18324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8337,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5023},{"shards":"0,2","objects":3010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11310","reported_seq":14687,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:12.235016+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:12.235016+0000","last_peered":"2024-09-15T21:25:12.235016+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:12.235016+0000","last_undegraded":"2024-09-15T21:25:12.235016+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37183488,"num_objects":9078,"num_object_clones":0,"num_object_copies":18156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2462,"num_objects_unfound":0,"num_objects_dirty":9078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7794,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1620,"num_bytes_recovered":6635520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6616},{"shards":"0,2","objects":2462}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11591","reported_seq":11730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.195878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:12.195878+0000","last_peered":"2024-09-15T21:25:12.195878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:12.195878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37991366,"num_objects":9277,"num_object_clones":0,"num_object_copies":18554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9277,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8295,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5130},{"shards":"0,2","objects":2995}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":605418230,"num_objects":147817,"num_object_clones":0,"num_object_copies":295634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2462,"num_objects_unfound":0,"num_objects_dirty":147817,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":142638,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5785,"num_bytes_recovered":23687168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1279873024,"data_stored":1279716747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735708,"kb_used_data":429132,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93636132,"statfs":{"total":96636764160,"available":95883399168,"internally_reserved":0,"allocated":439431168,"data_stored":439155521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":847788,"kb_used_data":509340,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93524052,"statfs":{"total":96636764160,"available":95768629248,"internally_reserved":0,"allocated":521564160,"data_stored":521288508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,0,1,11,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598216,"kb_used_data":313848,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93773624,"statfs":{"total":96636764160,"available":96024190976,"internally_reserved":0,"allocated":321380352,"data_stored":321060885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":520134656,"data_stored":520098526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320966656,"data_stored":320886711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":438771712,"data_stored":438731510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:13.447 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:14.019 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:14.024 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:14.048 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-15T21:25:12.768743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605418249,"num_objects":147819,"num_object_clones":0,"num_object_copies":295638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2462,"num_objects_unfound":0,"num_objects_dirty":147819,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":142640,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5786,"num_bytes_recovered":23687187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2181712,"kb_used_data":1252320,"kb_used_omap":12,"kb_used_meta":929331,"kb_avail":280933808,"statfs":{"total":289910292480,"available":287676219392,"internally_reserved":0,"allocated":1282375680,"data_stored":1281504914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":951634947},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,6,0,3,14,7,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24215552,"num_objects":-5912,"num_object_clones":0,"num_object_copies":-11824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-271,"num_objects_unfound":0,"num_objects_dirty":-5912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5912,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":1110016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002840"},"pg_stats":[{"pgid":"2.f","version":"45'11709","reported_seq":11785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.157546+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:12.157546+0000","last_peered":"2024-09-15T21:25:12.157546+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:12.157546+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38678528,"num_objects":9443,"num_object_clones":0,"num_object_copies":18886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11709,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5311},{"shards":"0,2","objects":2999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11355","reported_seq":11460,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:08.604150+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:08.604150+0000","last_peered":"2024-09-15T21:25:08.604150+0000","last_clean":"2024-09-15T21:25:08.604150+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:08.604150+0000","last_undegraded":"2024-09-15T21:25:08.604150+0000","last_fullsized":"2024-09-15T21:25:08.604150+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38171590,"num_objects":9321,"num_object_clones":0,"num_object_copies":18642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9321,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9636,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11591","reported_seq":11718,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:08.614853+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:08.614853+0000","last_peered":"2024-09-15T21:25:08.614853+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:08.614853+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38969368,"num_objects":9515,"num_object_clones":0,"num_object_copies":19030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9691,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3231},{"shards":"2","objects":5248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11200","reported_seq":11271,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:08.558506+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:08.558506+0000","last_peered":"2024-09-15T21:25:08.558506+0000","last_clean":"2024-09-15T21:25:08.558506+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:08.558506+0000","last_undegraded":"2024-09-15T21:25:08.558506+0000","last_fullsized":"2024-09-15T21:25:08.558506+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37330944,"num_objects":9114,"num_object_clones":0,"num_object_copies":18228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11200,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11371","reported_seq":11504,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:12.214636+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:12.214636+0000","last_peered":"2024-09-15T21:25:12.214636+0000","last_clean":"2024-09-15T21:25:12.214636+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:12.214636+0000","last_undegraded":"2024-09-15T21:25:12.214636+0000","last_fullsized":"2024-09-15T21:25:12.214636+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37413347,"num_objects":9135,"num_object_clones":0,"num_object_copies":18270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7826,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11481","reported_seq":11615,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.225458+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:12.225458+0000","last_peered":"2024-09-15T21:25:12.225458+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:12.225458+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37495267,"num_objects":9155,"num_object_clones":0,"num_object_copies":18310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8157,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5079},{"shards":"0,2","objects":2915}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11455","reported_seq":11591,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.125018+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:12.125018+0000","last_peered":"2024-09-15T21:25:12.125018+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:12.125018+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37745606,"num_objects":9217,"num_object_clones":0,"num_object_copies":18434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7766,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5056},{"shards":"0,2","objects":3044}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11573","reported_seq":11702,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:12.195551+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:12.195551+0000","last_peered":"2024-09-15T21:25:12.195551+0000","last_clean":"2024-09-15T21:25:12.195551+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:12.195551+0000","last_undegraded":"2024-09-15T21:25:12.195551+0000","last_fullsized":"2024-09-15T21:25:12.195551+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072320,"num_objects":9295,"num_object_clones":0,"num_object_copies":18590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8345,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876912+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876912+0000","last_peered":"2024-09-15T21:23:35.876912+0000","last_clean":"2024-09-15T21:23:35.876912+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876912+0000","last_undegraded":"2024-09-15T21:23:35.876912+0000","last_fullsized":"2024-09-15T21:23:35.876912+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11319","reported_seq":11445,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:08.731461+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:08.731461+0000","last_peered":"2024-09-15T21:25:08.731461+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:08.731461+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37659107,"num_objects":9195,"num_object_clones":0,"num_object_copies":18390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9195,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9251,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3026},{"shards":"2","objects":5108}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11322","reported_seq":11453,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.101055+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:12.101055+0000","last_peered":"2024-09-15T21:25:12.101055+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:12.101055+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37519360,"num_objects":9160,"num_object_clones":0,"num_object_copies":18320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7995,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5140},{"shards":"0,2","objects":2943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11342","reported_seq":11461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:08.593539+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:08.593539+0000","last_peered":"2024-09-15T21:25:08.593539+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:08.593539+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38097379,"num_objects":9302,"num_object_clones":0,"num_object_copies":18604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9302,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9201,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3072},{"shards":"2","objects":5212}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11347","reported_seq":19806,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:08.531421+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:08.531421+0000","last_peered":"2024-09-15T21:25:08.531421+0000","last_clean":"2024-09-15T21:25:08.531421+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:08.531421+0000","last_undegraded":"2024-09-15T21:25:08.531421+0000","last_fullsized":"2024-09-15T21:25:08.531421+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37736448,"num_objects":9213,"num_object_clones":0,"num_object_copies":18426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9040,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11451","reported_seq":11584,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.175618+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:12.175618+0000","last_peered":"2024-09-15T21:25:12.175618+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:12.175618+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37826560,"num_objects":9235,"num_object_clones":0,"num_object_copies":18470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8395,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5141},{"shards":"0,2","objects":2987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11428","reported_seq":11561,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.083206+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:12.083206+0000","last_peered":"2024-09-15T21:25:12.083206+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:12.083206+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37527552,"num_objects":9162,"num_object_clones":0,"num_object_copies":18324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8337,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5023},{"shards":"0,2","objects":3010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11310","reported_seq":14687,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:12.235016+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:12.235016+0000","last_peered":"2024-09-15T21:25:12.235016+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:12.235016+0000","last_undegraded":"2024-09-15T21:25:12.235016+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37183488,"num_objects":9078,"num_object_clones":0,"num_object_copies":18156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2462,"num_objects_unfound":0,"num_objects_dirty":9078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7794,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1620,"num_bytes_recovered":6635520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6616},{"shards":"0,2","objects":2462}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11591","reported_seq":11730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.195878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:12.195878+0000","last_peered":"2024-09-15T21:25:12.195878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:12.195878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37991366,"num_objects":9277,"num_object_clones":0,"num_object_copies":18554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9277,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8295,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5130},{"shards":"0,2","objects":2995}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":605418230,"num_objects":147817,"num_object_clones":0,"num_object_copies":295634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2462,"num_objects_unfound":0,"num_objects_dirty":147817,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":142638,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5785,"num_bytes_recovered":23687168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1279873024,"data_stored":1279716747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735708,"kb_used_data":429132,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93636132,"statfs":{"total":96636764160,"available":95883399168,"internally_reserved":0,"allocated":439431168,"data_stored":439155521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":847788,"kb_used_data":509340,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93524052,"statfs":{"total":96636764160,"available":95768629248,"internally_reserved":0,"allocated":521564160,"data_stored":521288508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,0,1,11,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598216,"kb_used_data":313848,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93773624,"statfs":{"total":96636764160,"available":96024190976,"internally_reserved":0,"allocated":321380352,"data_stored":321060885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,2,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":520134656,"data_stored":520098526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320966656,"data_stored":320886711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":438771712,"data_stored":438731510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:14.049 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:14.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11709", 'reported_seq': 11785, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:12.157546+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:25:12.157546+0000', 'last_peered': '2024-09-15T21:25:12.157546+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:25:12.157546+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38678528, 'num_objects': 9443, 'num_object_clones': 0, 'num_object_copies': 18886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9443, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11709, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5311}, {'shards': '0,2', 'objects': 2999}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:14.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:25:14.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'11591", 'reported_seq': 11718, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:08.614853+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:25:08.614853+0000', 'last_peered': '2024-09-15T21:25:08.614853+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:25:08.614853+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38969368, 'num_objects': 9515, 'num_object_clones': 0, 'num_object_copies': 19030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9515, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9691, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3231}, {'shards': '2', 'objects': 5248}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:14.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:14.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'11481", 'reported_seq': 11615, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:12.225458+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:25:12.225458+0000', 'last_peered': '2024-09-15T21:25:12.225458+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:25:12.225458+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37495267, 'num_objects': 9155, 'num_object_clones': 0, 'num_object_copies': 18310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9155, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8157, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5079}, {'shards': '0,2', 'objects': 2915}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:14.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:14.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'11455", 'reported_seq': 11591, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:12.125018+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:25:12.125018+0000', 'last_peered': '2024-09-15T21:25:12.125018+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:25:12.125018+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37745606, 'num_objects': 9217, 'num_object_clones': 0, 'num_object_copies': 18434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9217, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7766, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5056}, {'shards': '0,2', 'objects': 3044}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:14.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:25:14.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'11319", 'reported_seq': 11445, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:08.731461+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:25:08.731461+0000', 'last_peered': '2024-09-15T21:25:08.731461+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:25:08.731461+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37659107, 'num_objects': 9195, 'num_object_clones': 0, 'num_object_copies': 18390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9195, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9251, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3026}, {'shards': '2', 'objects': 5108}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:14.051 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:14.051 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11322", 'reported_seq': 11453, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:12.101055+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:25:12.101055+0000', 'last_peered': '2024-09-15T21:25:12.101055+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:25:12.101055+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37519360, 'num_objects': 9160, 'num_object_clones': 0, 'num_object_copies': 18320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7995, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5140}, {'shards': '0,2', 'objects': 2943}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:14.051 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:14.051 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'11342", 'reported_seq': 11461, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:08.593539+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:25:08.593539+0000', 'last_peered': '2024-09-15T21:25:08.593539+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:25:08.593539+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38097379, 'num_objects': 9302, 'num_object_clones': 0, 'num_object_copies': 18604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9302, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9201, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3072}, {'shards': '2', 'objects': 5212}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:14.051 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:14.051 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11451", 'reported_seq': 11584, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:12.175618+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:25:12.175618+0000', 'last_peered': '2024-09-15T21:25:12.175618+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:25:12.175618+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37826560, 'num_objects': 9235, 'num_object_clones': 0, 'num_object_copies': 18470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9235, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8395, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5141}, {'shards': '0,2', 'objects': 2987}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:14.051 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:25:14.051 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11428", 'reported_seq': 11561, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:12.083206+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:25:12.083206+0000', 'last_peered': '2024-09-15T21:25:12.083206+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:25:12.083206+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37527552, 'num_objects': 9162, 'num_object_clones': 0, 'num_object_copies': 18324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9162, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8337, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5023}, {'shards': '0,2', 'objects': 3010}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:14.051 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:14.051 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11310", 'reported_seq': 14687, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:25:12.235016+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:25:12.235016+0000', 'last_peered': '2024-09-15T21:25:12.235016+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:25:12.235016+0000', 'last_undegraded': '2024-09-15T21:25:12.235016+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37183488, 'num_objects': 9078, 'num_object_clones': 0, 'num_object_copies': 18156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2462, 'num_objects_unfound': 0, 'num_objects_dirty': 9078, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7794, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1620, 'num_bytes_recovered': 6635520, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6616}, {'shards': '0,2', 'objects': 2462}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:14.052 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:25:14.052 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11591", 'reported_seq': 11730, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:12.195878+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:25:12.195878+0000', 'last_peered': '2024-09-15T21:25:12.195878+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:25:12.195878+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37991366, 'num_objects': 9277, 'num_object_clones': 0, 'num_object_copies': 18554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9277, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8295, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5130}, {'shards': '0,2', 'objects': 2995}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:14.052 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:14.747 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:14.747 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/295638 objects degraded (13.944%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":366,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":147819,"data_bytes":605418249,"bytes_used":2234073088,"bytes_avail":287676219392,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":295638,"degraded_ratio":0.13944418511828655,"misplaced_objects":2462,"misplaced_total":295638,"misplaced_ratio":0.0083277521834134987,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":92479,"recovering_keys_per_sec":0,"num_objects_recovered":271,"num_bytes_recovered":1110016,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":492},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:25:14.796 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:14.796 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:15.364 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:15.366 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:15.386 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":221,"stamp":"2024-09-15T21:25:14.769255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":601965321,"num_objects":146976,"num_object_clones":0,"num_object_copies":293952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2462,"num_objects_unfound":0,"num_objects_dirty":146976,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":143483,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5786,"num_bytes_recovered":23687187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160891,"ondisk_log_size":160891,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2189020,"kb_used_data":1247148,"kb_used_omap":12,"kb_used_meta":941811,"kb_avail":280926500,"statfs":{"total":289910292480,"available":287668736000,"internally_reserved":0,"allocated":1277079552,"data_stored":1276208786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":964414467},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,3,2,5,14,8,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21864448,"num_objects":-5338,"num_object_clones":0,"num_object_copies":-10676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-182,"num_objects_unfound":0,"num_objects_dirty":-5338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5338,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":745472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002922"},"pg_stats":[{"pgid":"2.f","version":"45'11709","reported_seq":11785,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.157546+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:12.157546+0000","last_peered":"2024-09-15T21:25:12.157546+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:12.157546+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38678528,"num_objects":9443,"num_object_clones":0,"num_object_copies":18886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11709,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5311},{"shards":"0,2","objects":2999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11492","reported_seq":11598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:13.758790+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:13.758790+0000","last_peered":"2024-09-15T21:25:13.758790+0000","last_clean":"2024-09-15T21:25:13.758790+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:13.758790+0000","last_undegraded":"2024-09-15T21:25:13.758790+0000","last_fullsized":"2024-09-15T21:25:13.758790+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37610438,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9773,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11733","reported_seq":11858,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:13.743986+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:13.743986+0000","last_peered":"2024-09-15T21:25:13.743986+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:13.743986+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38387736,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9833,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3089},{"shards":"2","objects":5106}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11328","reported_seq":11399,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:13.745727+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:13.745727+0000","last_peered":"2024-09-15T21:25:13.745727+0000","last_clean":"2024-09-15T21:25:13.745727+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:13.745727+0000","last_undegraded":"2024-09-15T21:25:13.745727+0000","last_fullsized":"2024-09-15T21:25:13.745727+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36806656,"num_objects":8986,"num_object_clones":0,"num_object_copies":17972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11328,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11371","reported_seq":11504,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:12.214636+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:12.214636+0000","last_peered":"2024-09-15T21:25:12.214636+0000","last_clean":"2024-09-15T21:25:12.214636+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:12.214636+0000","last_undegraded":"2024-09-15T21:25:12.214636+0000","last_fullsized":"2024-09-15T21:25:12.214636+0000","mapping_epoch":34,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37413347,"num_objects":9135,"num_object_clones":0,"num_object_copies":18270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7826,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11481","reported_seq":11615,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.225458+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:12.225458+0000","last_peered":"2024-09-15T21:25:12.225458+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:12.225458+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37495267,"num_objects":9155,"num_object_clones":0,"num_object_copies":18310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9155,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8157,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5079},{"shards":"0,2","objects":2915}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11455","reported_seq":11591,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.125018+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:12.125018+0000","last_peered":"2024-09-15T21:25:12.125018+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:12.125018+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37745606,"num_objects":9217,"num_object_clones":0,"num_object_copies":18434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9217,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7766,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5056},{"shards":"0,2","objects":3044}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11573","reported_seq":11702,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:12.195551+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:12.195551+0000","last_peered":"2024-09-15T21:25:12.195551+0000","last_clean":"2024-09-15T21:25:12.195551+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:12.195551+0000","last_undegraded":"2024-09-15T21:25:12.195551+0000","last_fullsized":"2024-09-15T21:25:12.195551+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38072320,"num_objects":9295,"num_object_clones":0,"num_object_copies":18590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8345,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11470","reported_seq":11597,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:13.640676+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:13.640676+0000","last_peered":"2024-09-15T21:25:13.640676+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:13.640676+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37040611,"num_objects":9044,"num_object_clones":0,"num_object_copies":18088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9044,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9402,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2875},{"shards":"2","objects":4957}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11322","reported_seq":11453,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.101055+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:12.101055+0000","last_peered":"2024-09-15T21:25:12.101055+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:12.101055+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37519360,"num_objects":9160,"num_object_clones":0,"num_object_copies":18320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7995,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5140},{"shards":"0,2","objects":2943}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11478","reported_seq":11597,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:13.693719+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:13.693719+0000","last_peered":"2024-09-15T21:25:13.693719+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:13.693719+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37540323,"num_objects":9166,"num_object_clones":0,"num_object_copies":18332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9166,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9337,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2936},{"shards":"2","objects":5076}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11496","reported_seq":19955,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:13.696536+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:13.696536+0000","last_peered":"2024-09-15T21:25:13.696536+0000","last_clean":"2024-09-15T21:25:13.696536+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:13.696536+0000","last_undegraded":"2024-09-15T21:25:13.696536+0000","last_fullsized":"2024-09-15T21:25:13.696536+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37126144,"num_objects":9064,"num_object_clones":0,"num_object_copies":18128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9189,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11451","reported_seq":11584,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.175618+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:12.175618+0000","last_peered":"2024-09-15T21:25:12.175618+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:12.175618+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37826560,"num_objects":9235,"num_object_clones":0,"num_object_copies":18470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8395,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5141},{"shards":"0,2","objects":2987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11428","reported_seq":11561,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.083206+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:12.083206+0000","last_peered":"2024-09-15T21:25:12.083206+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:12.083206+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37527552,"num_objects":9162,"num_object_clones":0,"num_object_copies":18324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8337,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5023},{"shards":"0,2","objects":3010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11310","reported_seq":14687,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:12.235016+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:12.235016+0000","last_peered":"2024-09-15T21:25:12.235016+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:12.235016+0000","last_undegraded":"2024-09-15T21:25:12.235016+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37183488,"num_objects":9078,"num_object_clones":0,"num_object_copies":18156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2462,"num_objects_unfound":0,"num_objects_dirty":9078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7794,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1620,"num_bytes_recovered":6635520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6616},{"shards":"0,2","objects":2462}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11591","reported_seq":11730,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:12.195878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:12.195878+0000","last_peered":"2024-09-15T21:25:12.195878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:12.195878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37991366,"num_objects":9277,"num_object_clones":0,"num_object_copies":18554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9277,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8295,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5130},{"shards":"0,2","objects":2995}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":601965302,"num_objects":146974,"num_object_clones":0,"num_object_copies":293948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2462,"num_objects_unfound":0,"num_objects_dirty":146974,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":143481,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5785,"num_bytes_recovered":23687168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1274466304,"data_stored":1274310027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160888,"ondisk_log_size":160888,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735488,"kb_used_data":424752,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93636352,"statfs":{"total":96636764160,"available":95883624448,"internally_reserved":0,"allocated":434946048,"data_stored":434670401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":847788,"kb_used_data":509340,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93524052,"statfs":{"total":96636764160,"available":95768629248,"internally_reserved":0,"allocated":521564160,"data_stored":521288508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,3,0,1,11,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":605744,"kb_used_data":313056,"kb_used_omap":4,"kb_used_meta":292667,"kb_avail":93766096,"statfs":{"total":96636764160,"available":96016482304,"internally_reserved":0,"allocated":320569344,"data_stored":320249877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":299691695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":520134656,"data_stored":520098526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":320131072,"data_stored":320051127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434200576,"data_stored":434160374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:18.388 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:18.937 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:18.938 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:18.968 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-15T21:25:18.770206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596132617,"num_objects":145552,"num_object_clones":0,"num_object_copies":291104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2384,"num_objects_unfound":0,"num_objects_dirty":145552,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":144907,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5864,"num_bytes_recovered":24006675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2209204,"kb_used_data":1239552,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":280906316,"statfs":{"total":289910292480,"available":287648067584,"internally_reserved":0,"allocated":1269301248,"data_stored":1268430482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":992857091},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,8,11,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24150016,"num_objects":-5896,"num_object_clones":0,"num_object_copies":-11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-260,"num_objects_unfound":0,"num_objects_dirty":-5896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5896,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":1064960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002908"},"pg_stats":[{"pgid":"2.f","version":"45'11836","reported_seq":11909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.286654+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:17.286654+0000","last_peered":"2024-09-15T21:25:17.286654+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:17.286654+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38158336,"num_objects":9316,"num_object_clones":0,"num_object_copies":18632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11836,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5184},{"shards":"0,2","objects":2872}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11492","reported_seq":11598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:13.758790+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:13.758790+0000","last_peered":"2024-09-15T21:25:13.758790+0000","last_clean":"2024-09-15T21:25:13.758790+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:13.758790+0000","last_undegraded":"2024-09-15T21:25:13.758790+0000","last_fullsized":"2024-09-15T21:25:13.758790+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37610438,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9773,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11733","reported_seq":11858,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:13.743986+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:13.743986+0000","last_peered":"2024-09-15T21:25:13.743986+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:13.743986+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38387736,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9833,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3089},{"shards":"2","objects":5106}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11328","reported_seq":11399,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:13.745727+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:13.745727+0000","last_peered":"2024-09-15T21:25:13.745727+0000","last_clean":"2024-09-15T21:25:13.745727+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:13.745727+0000","last_undegraded":"2024-09-15T21:25:13.745727+0000","last_fullsized":"2024-09-15T21:25:13.745727+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36806656,"num_objects":8986,"num_object_clones":0,"num_object_copies":17972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11328,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11501","reported_seq":11634,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:17.115859+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:17.115859+0000","last_peered":"2024-09-15T21:25:17.115859+0000","last_clean":"2024-09-15T21:25:17.115859+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:17.115859+0000","last_undegraded":"2024-09-15T21:25:17.115859+0000","last_fullsized":"2024-09-15T21:25:17.115859+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880867,"num_objects":9005,"num_object_clones":0,"num_object_copies":18010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7956,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11619","reported_seq":11755,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.291025+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:17.291025+0000","last_peered":"2024-09-15T21:25:17.291025+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:17.291025+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36930019,"num_objects":9017,"num_object_clones":0,"num_object_copies":18034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9017,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8295,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4941},{"shards":"0,2","objects":2777}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11615","reported_seq":11751,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.265894+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:17.265894+0000","last_peered":"2024-09-15T21:25:17.265894+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:17.265894+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37090246,"num_objects":9057,"num_object_clones":0,"num_object_copies":18114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7926,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4896},{"shards":"0,2","objects":2884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11719","reported_seq":11848,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:17.247217+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:17.247217+0000","last_peered":"2024-09-15T21:25:17.247217+0000","last_clean":"2024-09-15T21:25:17.247217+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:17.247217+0000","last_undegraded":"2024-09-15T21:25:17.247217+0000","last_fullsized":"2024-09-15T21:25:17.247217+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37474304,"num_objects":9149,"num_object_clones":0,"num_object_copies":18298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8491,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11470","reported_seq":11597,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:13.640676+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:13.640676+0000","last_peered":"2024-09-15T21:25:13.640676+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:13.640676+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37040611,"num_objects":9044,"num_object_clones":0,"num_object_copies":18088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9044,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9402,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2875},{"shards":"2","objects":4957}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11467","reported_seq":11598,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.154666+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:17.154666+0000","last_peered":"2024-09-15T21:25:17.154666+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:17.154666+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36925440,"num_objects":9015,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8140,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4995},{"shards":"0,2","objects":2798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11478","reported_seq":11597,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:13.693719+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:13.693719+0000","last_peered":"2024-09-15T21:25:13.693719+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:13.693719+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37540323,"num_objects":9166,"num_object_clones":0,"num_object_copies":18332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9166,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9337,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2936},{"shards":"2","objects":5076}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11496","reported_seq":19955,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:13.696536+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:13.696536+0000","last_peered":"2024-09-15T21:25:13.696536+0000","last_clean":"2024-09-15T21:25:13.696536+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:13.696536+0000","last_undegraded":"2024-09-15T21:25:13.696536+0000","last_fullsized":"2024-09-15T21:25:13.696536+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37126144,"num_objects":9064,"num_object_clones":0,"num_object_copies":18128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9189,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11585","reported_seq":11719,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.181524+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:17.181524+0000","last_peered":"2024-09-15T21:25:17.181524+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:17.181524+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8529,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5007},{"shards":"0,2","objects":2853}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11575","reported_seq":11708,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.292589+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:17.292589+0000","last_peered":"2024-09-15T21:25:17.292589+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:17.292589+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36925440,"num_objects":9015,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8484,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4876},{"shards":"0,2","objects":2863}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11479","reported_seq":15012,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:17.273012+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:17.273012+0000","last_peered":"2024-09-15T21:25:17.273012+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:17.273012+0000","last_undegraded":"2024-09-15T21:25:17.273012+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36491264,"num_objects":8909,"num_object_clones":0,"num_object_copies":17818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2384,"num_objects_unfound":0,"num_objects_dirty":8909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7963,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1698,"num_bytes_recovered":6955008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6525},{"shards":"0,2","objects":2384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11719","reported_seq":11858,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.219692+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:17.219692+0000","last_peered":"2024-09-15T21:25:17.219692+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:17.219692+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37467078,"num_objects":9149,"num_object_clones":0,"num_object_copies":18298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8423,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5002},{"shards":"0,2","objects":2867}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":596132598,"num_objects":145550,"num_object_clones":0,"num_object_copies":291100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2384,"num_objects_unfound":0,"num_objects_dirty":145550,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":144905,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5863,"num_bytes_recovered":24006656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1266581504,"data_stored":1266425227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735488,"kb_used_data":424752,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93636352,"statfs":{"total":96636764160,"available":95883624448,"internally_reserved":0,"allocated":434946048,"data_stored":434670401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":881392,"kb_used_data":502684,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93490448,"statfs":{"total":96636764160,"available":95734218752,"internally_reserved":0,"allocated":514748416,"data_stored":514472764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,7,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592324,"kb_used_data":312116,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93779516,"statfs":{"total":96636764160,"available":96030224384,"internally_reserved":0,"allocated":319606784,"data_stored":319287317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513232896,"data_stored":513196766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319148032,"data_stored":319068087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434200576,"data_stored":434160374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:18.971 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:19.535 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:19.536 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:19.557 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-15T21:25:18.770206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596132617,"num_objects":145552,"num_object_clones":0,"num_object_copies":291104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2384,"num_objects_unfound":0,"num_objects_dirty":145552,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":144907,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5864,"num_bytes_recovered":24006675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2209204,"kb_used_data":1239552,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":280906316,"statfs":{"total":289910292480,"available":287648067584,"internally_reserved":0,"allocated":1269301248,"data_stored":1268430482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":992857091},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,8,11,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24150016,"num_objects":-5896,"num_object_clones":0,"num_object_copies":-11792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-260,"num_objects_unfound":0,"num_objects_dirty":-5896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5896,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":260,"num_bytes_recovered":1064960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002908"},"pg_stats":[{"pgid":"2.f","version":"45'11836","reported_seq":11909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.286654+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:17.286654+0000","last_peered":"2024-09-15T21:25:17.286654+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:17.286654+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38158336,"num_objects":9316,"num_object_clones":0,"num_object_copies":18632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11836,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5184},{"shards":"0,2","objects":2872}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11492","reported_seq":11598,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:13.758790+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:13.758790+0000","last_peered":"2024-09-15T21:25:13.758790+0000","last_clean":"2024-09-15T21:25:13.758790+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:13.758790+0000","last_undegraded":"2024-09-15T21:25:13.758790+0000","last_fullsized":"2024-09-15T21:25:13.758790+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37610438,"num_objects":9184,"num_object_clones":0,"num_object_copies":18368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9184,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9773,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11733","reported_seq":11858,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:13.743986+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:13.743986+0000","last_peered":"2024-09-15T21:25:13.743986+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:13.743986+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38387736,"num_objects":9373,"num_object_clones":0,"num_object_copies":18746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9373,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9833,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3089},{"shards":"2","objects":5106}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11328","reported_seq":11399,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:13.745727+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:13.745727+0000","last_peered":"2024-09-15T21:25:13.745727+0000","last_clean":"2024-09-15T21:25:13.745727+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:13.745727+0000","last_undegraded":"2024-09-15T21:25:13.745727+0000","last_fullsized":"2024-09-15T21:25:13.745727+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36806656,"num_objects":8986,"num_object_clones":0,"num_object_copies":17972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11328,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11501","reported_seq":11634,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:17.115859+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:17.115859+0000","last_peered":"2024-09-15T21:25:17.115859+0000","last_clean":"2024-09-15T21:25:17.115859+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:17.115859+0000","last_undegraded":"2024-09-15T21:25:17.115859+0000","last_fullsized":"2024-09-15T21:25:17.115859+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880867,"num_objects":9005,"num_object_clones":0,"num_object_copies":18010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7956,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11619","reported_seq":11755,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.291025+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:17.291025+0000","last_peered":"2024-09-15T21:25:17.291025+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:17.291025+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36930019,"num_objects":9017,"num_object_clones":0,"num_object_copies":18034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9017,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8295,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4941},{"shards":"0,2","objects":2777}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11615","reported_seq":11751,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.265894+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:17.265894+0000","last_peered":"2024-09-15T21:25:17.265894+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:17.265894+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37090246,"num_objects":9057,"num_object_clones":0,"num_object_copies":18114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7926,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4896},{"shards":"0,2","objects":2884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11719","reported_seq":11848,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:17.247217+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:17.247217+0000","last_peered":"2024-09-15T21:25:17.247217+0000","last_clean":"2024-09-15T21:25:17.247217+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:17.247217+0000","last_undegraded":"2024-09-15T21:25:17.247217+0000","last_fullsized":"2024-09-15T21:25:17.247217+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37474304,"num_objects":9149,"num_object_clones":0,"num_object_copies":18298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8491,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11470","reported_seq":11597,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:13.640676+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:13.640676+0000","last_peered":"2024-09-15T21:25:13.640676+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:13.640676+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37040611,"num_objects":9044,"num_object_clones":0,"num_object_copies":18088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9044,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9402,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2875},{"shards":"2","objects":4957}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11467","reported_seq":11598,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.154666+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:17.154666+0000","last_peered":"2024-09-15T21:25:17.154666+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:17.154666+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36925440,"num_objects":9015,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8140,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4995},{"shards":"0,2","objects":2798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11478","reported_seq":11597,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:13.693719+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:13.693719+0000","last_peered":"2024-09-15T21:25:13.693719+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:13.693719+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37540323,"num_objects":9166,"num_object_clones":0,"num_object_copies":18332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9166,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9337,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2936},{"shards":"2","objects":5076}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11496","reported_seq":19955,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:13.696536+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:13.696536+0000","last_peered":"2024-09-15T21:25:13.696536+0000","last_clean":"2024-09-15T21:25:13.696536+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:13.696536+0000","last_undegraded":"2024-09-15T21:25:13.696536+0000","last_fullsized":"2024-09-15T21:25:13.696536+0000","mapping_epoch":44,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37126144,"num_objects":9064,"num_object_clones":0,"num_object_copies":18128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9189,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11585","reported_seq":11719,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.181524+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:17.181524+0000","last_peered":"2024-09-15T21:25:17.181524+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:17.181524+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8529,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5007},{"shards":"0,2","objects":2853}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11575","reported_seq":11708,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.292589+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:17.292589+0000","last_peered":"2024-09-15T21:25:17.292589+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:17.292589+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36925440,"num_objects":9015,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8484,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4876},{"shards":"0,2","objects":2863}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11479","reported_seq":15012,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:17.273012+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:17.273012+0000","last_peered":"2024-09-15T21:25:17.273012+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:17.273012+0000","last_undegraded":"2024-09-15T21:25:17.273012+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36491264,"num_objects":8909,"num_object_clones":0,"num_object_copies":17818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2384,"num_objects_unfound":0,"num_objects_dirty":8909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7963,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1698,"num_bytes_recovered":6955008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6525},{"shards":"0,2","objects":2384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11719","reported_seq":11858,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.219692+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:17.219692+0000","last_peered":"2024-09-15T21:25:17.219692+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:17.219692+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37467078,"num_objects":9149,"num_object_clones":0,"num_object_copies":18298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8423,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5002},{"shards":"0,2","objects":2867}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":596132598,"num_objects":145550,"num_object_clones":0,"num_object_copies":291100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2384,"num_objects_unfound":0,"num_objects_dirty":145550,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":144905,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5863,"num_bytes_recovered":24006656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1266581504,"data_stored":1266425227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735488,"kb_used_data":424752,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93636352,"statfs":{"total":96636764160,"available":95883624448,"internally_reserved":0,"allocated":434946048,"data_stored":434670401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,1,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":881392,"kb_used_data":502684,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93490448,"statfs":{"total":96636764160,"available":95734218752,"internally_reserved":0,"allocated":514748416,"data_stored":514472764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,7,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592324,"kb_used_data":312116,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93779516,"statfs":{"total":96636764160,"available":96030224384,"internally_reserved":0,"allocated":319606784,"data_stored":319287317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513232896,"data_stored":513196766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319148032,"data_stored":319068087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":434200576,"data_stored":434160374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:19.559 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:19.559 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11836", 'reported_seq': 11909, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:17.286654+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:25:17.286654+0000', 'last_peered': '2024-09-15T21:25:17.286654+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:25:17.286654+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38158336, 'num_objects': 9316, 'num_object_clones': 0, 'num_object_copies': 18632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11836, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5184}, {'shards': '0,2', 'objects': 2872}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.559 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:25:19.559 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'11733", 'reported_seq': 11858, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:13.743986+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:25:13.743986+0000', 'last_peered': '2024-09-15T21:25:13.743986+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:25:13.743986+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38387736, 'num_objects': 9373, 'num_object_clones': 0, 'num_object_copies': 18746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9373, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9833, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3089}, {'shards': '2', 'objects': 5106}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:19.559 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:19.559 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'11619", 'reported_seq': 11755, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:17.291025+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:25:17.291025+0000', 'last_peered': '2024-09-15T21:25:17.291025+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:25:17.291025+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36930019, 'num_objects': 9017, 'num_object_clones': 0, 'num_object_copies': 18034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9017, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8295, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4941}, {'shards': '0,2', 'objects': 2777}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.559 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:19.559 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'11615", 'reported_seq': 11751, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:17.265894+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:25:17.265894+0000', 'last_peered': '2024-09-15T21:25:17.265894+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:25:17.265894+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37090246, 'num_objects': 9057, 'num_object_clones': 0, 'num_object_copies': 18114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9057, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7926, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4896}, {'shards': '0,2', 'objects': 2884}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.560 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:25:19.560 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'11470", 'reported_seq': 11597, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:13.640676+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:25:13.640676+0000', 'last_peered': '2024-09-15T21:25:13.640676+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:25:13.640676+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37040611, 'num_objects': 9044, 'num_object_clones': 0, 'num_object_copies': 18088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9044, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9402, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2875}, {'shards': '2', 'objects': 4957}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:19.560 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:19.560 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11467", 'reported_seq': 11598, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:17.154666+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:25:17.154666+0000', 'last_peered': '2024-09-15T21:25:17.154666+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:25:17.154666+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36925440, 'num_objects': 9015, 'num_object_clones': 0, 'num_object_copies': 18030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9015, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8140, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4995}, {'shards': '0,2', 'objects': 2798}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.560 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:19.561 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'11478", 'reported_seq': 11597, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:13.693719+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:25:13.693719+0000', 'last_peered': '2024-09-15T21:25:13.693719+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:25:13.693719+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37540323, 'num_objects': 9166, 'num_object_clones': 0, 'num_object_copies': 18332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9166, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9337, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2936}, {'shards': '2', 'objects': 5076}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:19.561 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:19.561 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11585", 'reported_seq': 11719, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:17.181524+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:25:17.181524+0000', 'last_peered': '2024-09-15T21:25:17.181524+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:25:17.181524+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37277696, 'num_objects': 9101, 'num_object_clones': 0, 'num_object_copies': 18202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8529, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5007}, {'shards': '0,2', 'objects': 2853}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.561 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:25:19.561 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11575", 'reported_seq': 11708, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:17.292589+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:25:17.292589+0000', 'last_peered': '2024-09-15T21:25:17.292589+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:25:17.292589+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36925440, 'num_objects': 9015, 'num_object_clones': 0, 'num_object_copies': 18030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9015, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8484, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4876}, {'shards': '0,2', 'objects': 2863}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.561 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:19.561 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11479", 'reported_seq': 15012, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:25:17.273012+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:25:17.273012+0000', 'last_peered': '2024-09-15T21:25:17.273012+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:25:17.273012+0000', 'last_undegraded': '2024-09-15T21:25:17.273012+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36491264, 'num_objects': 8909, 'num_object_clones': 0, 'num_object_copies': 17818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2384, 'num_objects_unfound': 0, 'num_objects_dirty': 8909, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7963, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1698, 'num_bytes_recovered': 6955008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6525}, {'shards': '0,2', 'objects': 2384}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.561 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:25:19.562 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11719", 'reported_seq': 11858, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:17.219692+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:25:17.219692+0000', 'last_peered': '2024-09-15T21:25:17.219692+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:25:17.219692+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37467078, 'num_objects': 9149, 'num_object_clones': 0, 'num_object_copies': 18298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9149, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8423, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5002}, {'shards': '0,2', 'objects': 2867}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:19.562 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:20.276 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:20.276 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/291104 objects degraded (14.162%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":372,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":145552,"data_bytes":596132617,"bytes_used":2262224896,"bytes_avail":287648067584,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":291104,"degraded_ratio":0.14161605474332198,"misplaced_objects":2384,"misplaced_total":291104,"misplaced_ratio":0.0081895130262723981,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":88725,"recovering_keys_per_sec":0,"num_objects_recovered":260,"num_bytes_recovered":1064960,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":491},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:25:20.302 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:20.302 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:20.871 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:20.872 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:20.896 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-15T21:25:20.770684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592597769,"num_objects":144689,"num_object_clones":0,"num_object_copies":289378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2384,"num_objects_unfound":0,"num_objects_dirty":144689,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":145770,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5864,"num_bytes_recovered":24006675,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160678,"ondisk_log_size":160678,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2208872,"kb_used_data":1235052,"kb_used_omap":12,"kb_used_meta":973747,"kb_avail":280906648,"statfs":{"total":289910292480,"available":287648407552,"internally_reserved":0,"allocated":1264693248,"data_stored":1263822482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":997116931},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,3,10,11,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18534400,"num_objects":-4525,"num_object_clones":0,"num_object_copies":-9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-168,"num_objects_unfound":0,"num_objects_dirty":-4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4525,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":168,"num_bytes_recovered":688128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002962"},"pg_stats":[{"pgid":"2.f","version":"45'11836","reported_seq":11909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.286654+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:17.286654+0000","last_peered":"2024-09-15T21:25:17.286654+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:17.286654+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38158336,"num_objects":9316,"num_object_clones":0,"num_object_copies":18632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11836,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5184},{"shards":"0,2","objects":2872}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11633","reported_seq":11739,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:18.562859+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:18.562859+0000","last_peered":"2024-09-15T21:25:18.562859+0000","last_clean":"2024-09-15T21:25:18.562859+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:18.562859+0000","last_undegraded":"2024-09-15T21:25:18.562859+0000","last_fullsized":"2024-09-15T21:25:18.562859+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37032902,"num_objects":9043,"num_object_clones":0,"num_object_copies":18086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9043,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9914,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11892","reported_seq":12019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:18.696323+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:18.696323+0000","last_peered":"2024-09-15T21:25:18.696323+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:18.696323+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37736472,"num_objects":9214,"num_object_clones":0,"num_object_copies":18428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9214,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9992,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2930},{"shards":"2","objects":4947}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11467","reported_seq":11538,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:18.686066+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:18.686066+0000","last_peered":"2024-09-15T21:25:18.686066+0000","last_clean":"2024-09-15T21:25:18.686066+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:18.686066+0000","last_undegraded":"2024-09-15T21:25:18.686066+0000","last_fullsized":"2024-09-15T21:25:18.686066+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36237312,"num_objects":8847,"num_object_clones":0,"num_object_copies":17694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11501","reported_seq":11634,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:17.115859+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:17.115859+0000","last_peered":"2024-09-15T21:25:17.115859+0000","last_clean":"2024-09-15T21:25:17.115859+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:17.115859+0000","last_undegraded":"2024-09-15T21:25:17.115859+0000","last_fullsized":"2024-09-15T21:25:17.115859+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36880867,"num_objects":9005,"num_object_clones":0,"num_object_copies":18010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7956,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11619","reported_seq":11755,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.291025+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:17.291025+0000","last_peered":"2024-09-15T21:25:17.291025+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:17.291025+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36930019,"num_objects":9017,"num_object_clones":0,"num_object_copies":18034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9017,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8295,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4941},{"shards":"0,2","objects":2777}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11615","reported_seq":11751,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.265894+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:17.265894+0000","last_peered":"2024-09-15T21:25:17.265894+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:17.265894+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37090246,"num_objects":9057,"num_object_clones":0,"num_object_copies":18114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7926,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4896},{"shards":"0,2","objects":2884}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11719","reported_seq":11848,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:17.247217+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:17.247217+0000","last_peered":"2024-09-15T21:25:17.247217+0000","last_clean":"2024-09-15T21:25:17.247217+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:17.247217+0000","last_undegraded":"2024-09-15T21:25:17.247217+0000","last_fullsized":"2024-09-15T21:25:17.247217+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37474304,"num_objects":9149,"num_object_clones":0,"num_object_copies":18298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8491,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11608","reported_seq":11735,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:18.684637+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:18.684637+0000","last_peered":"2024-09-15T21:25:18.684637+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:18.684637+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36475363,"num_objects":8906,"num_object_clones":0,"num_object_copies":17812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8906,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9540,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2737},{"shards":"2","objects":4819}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11467","reported_seq":11598,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.154666+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:17.154666+0000","last_peered":"2024-09-15T21:25:17.154666+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:17.154666+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36925440,"num_objects":9015,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8140,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4995},{"shards":"0,2","objects":2798}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11618","reported_seq":11737,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:18.605479+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:18.605479+0000","last_peered":"2024-09-15T21:25:18.605479+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:18.605479+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36966883,"num_objects":9026,"num_object_clones":0,"num_object_copies":18052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9477,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2796},{"shards":"2","objects":4936}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11642","reported_seq":20101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:18.772204+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:18.772204+0000","last_peered":"2024-09-15T21:25:18.772204+0000","last_clean":"2024-09-15T21:25:18.772204+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:18.772204+0000","last_undegraded":"2024-09-15T21:25:18.772204+0000","last_fullsized":"2024-09-15T21:25:18.772204+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36528128,"num_objects":8918,"num_object_clones":0,"num_object_copies":17836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9335,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11585","reported_seq":11719,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.181524+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:17.181524+0000","last_peered":"2024-09-15T21:25:17.181524+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:17.181524+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37277696,"num_objects":9101,"num_object_clones":0,"num_object_copies":18202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8529,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5007},{"shards":"0,2","objects":2853}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11575","reported_seq":11708,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.292589+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:17.292589+0000","last_peered":"2024-09-15T21:25:17.292589+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:17.292589+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36925440,"num_objects":9015,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8484,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4876},{"shards":"0,2","objects":2863}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11479","reported_seq":15012,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:17.273012+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:17.273012+0000","last_peered":"2024-09-15T21:25:17.273012+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:17.273012+0000","last_undegraded":"2024-09-15T21:25:17.273012+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36491264,"num_objects":8909,"num_object_clones":0,"num_object_copies":17818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2384,"num_objects_unfound":0,"num_objects_dirty":8909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7963,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1698,"num_bytes_recovered":6955008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6525},{"shards":"0,2","objects":2384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11719","reported_seq":11858,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:17.219692+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:17.219692+0000","last_peered":"2024-09-15T21:25:17.219692+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:17.219692+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37467078,"num_objects":9149,"num_object_clones":0,"num_object_copies":18298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9149,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8423,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5002},{"shards":"0,2","objects":2867}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":592597750,"num_objects":144687,"num_object_clones":0,"num_object_copies":289374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2384,"num_objects_unfound":0,"num_objects_dirty":144687,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":145768,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5863,"num_bytes_recovered":24006656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1261993984,"data_stored":1261837707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160675,"ondisk_log_size":160675,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735156,"kb_used_data":420252,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93636684,"statfs":{"total":96636764160,"available":95883964416,"internally_reserved":0,"allocated":430338048,"data_stored":430062401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":322432681},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":881392,"kb_used_data":502684,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93490448,"statfs":{"total":96636764160,"available":95734218752,"internally_reserved":0,"allocated":514748416,"data_stored":514472764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,7,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592324,"kb_used_data":312116,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93779516,"statfs":{"total":96636764160,"available":96030224384,"internally_reserved":0,"allocated":319606784,"data_stored":319287317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":513232896,"data_stored":513196766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319148032,"data_stored":319068087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":429613056,"data_stored":429572854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:23.898 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:24.442 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:24.443 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:24.468 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-15T21:25:22.771148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":586842889,"num_objects":143284,"num_object_clones":0,"num_object_copies":286568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2299,"num_objects_unfound":0,"num_objects_dirty":143284,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":147175,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5949,"num_bytes_recovered":24354835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2206336,"kb_used_data":1228360,"kb_used_omap":12,"kb_used_meta":977907,"kb_avail":280909184,"statfs":{"total":289910292480,"available":287651004416,"internally_reserved":0,"allocated":1257840640,"data_stored":1256969874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1001376771},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,7,0,6,12,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24289280,"num_objects":-5930,"num_object_clones":0,"num_object_copies":-11860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-253,"num_objects_unfound":0,"num_objects_dirty":-5930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5930,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":1036288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002834"},"pg_stats":[{"pgid":"2.f","version":"45'11983","reported_seq":12059,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.281840+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:22.281840+0000","last_peered":"2024-09-15T21:25:22.281840+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:22.281840+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37556224,"num_objects":9169,"num_object_clones":0,"num_object_copies":18338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11983,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5037},{"shards":"0,2","objects":2725}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11633","reported_seq":11739,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:18.562859+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:18.562859+0000","last_peered":"2024-09-15T21:25:18.562859+0000","last_clean":"2024-09-15T21:25:18.562859+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:18.562859+0000","last_undegraded":"2024-09-15T21:25:18.562859+0000","last_fullsized":"2024-09-15T21:25:18.562859+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37032902,"num_objects":9043,"num_object_clones":0,"num_object_copies":18086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9043,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":9914,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11892","reported_seq":12019,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:18.696323+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:18.696323+0000","last_peered":"2024-09-15T21:25:18.696323+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:18.696323+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37736472,"num_objects":9214,"num_object_clones":0,"num_object_copies":18428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9214,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9992,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2930},{"shards":"2","objects":4947}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11467","reported_seq":11538,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:18.686066+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:18.686066+0000","last_peered":"2024-09-15T21:25:18.686066+0000","last_clean":"2024-09-15T21:25:18.686066+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:18.686066+0000","last_undegraded":"2024-09-15T21:25:18.686066+0000","last_fullsized":"2024-09-15T21:25:18.686066+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36237312,"num_objects":8847,"num_object_clones":0,"num_object_copies":17694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11467,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11632","reported_seq":11765,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:22.349322+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:22.349322+0000","last_peered":"2024-09-15T21:25:22.349322+0000","last_clean":"2024-09-15T21:25:22.349322+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:22.349322+0000","last_undegraded":"2024-09-15T21:25:22.349322+0000","last_fullsized":"2024-09-15T21:25:22.349322+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36344291,"num_objects":8874,"num_object_clones":0,"num_object_copies":17748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8087,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11755","reported_seq":11892,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.332396+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:22.332396+0000","last_peered":"2024-09-15T21:25:22.332396+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:22.332396+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36372963,"num_objects":8881,"num_object_clones":0,"num_object_copies":17762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8431,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4805},{"shards":"0,2","objects":2641}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11772","reported_seq":11906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.328542+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:22.328542+0000","last_peered":"2024-09-15T21:25:22.328542+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:22.328542+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36447174,"num_objects":8900,"num_object_clones":0,"num_object_copies":17800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8083,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4739},{"shards":"0,2","objects":2727}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11853","reported_seq":11981,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:22.355326+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:22.355326+0000","last_peered":"2024-09-15T21:25:22.355326+0000","last_clean":"2024-09-15T21:25:22.355326+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:22.355326+0000","last_undegraded":"2024-09-15T21:25:22.355326+0000","last_fullsized":"2024-09-15T21:25:22.355326+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36925440,"num_objects":9015,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8625,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11608","reported_seq":11735,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:18.684637+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:18.684637+0000","last_peered":"2024-09-15T21:25:18.684637+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:18.684637+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36475363,"num_objects":8906,"num_object_clones":0,"num_object_copies":17812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8906,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9540,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2737},{"shards":"2","objects":4819}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11584","reported_seq":11714,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.321588+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:22.321588+0000","last_peered":"2024-09-15T21:25:22.321588+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:22.321588+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36446208,"num_objects":8898,"num_object_clones":0,"num_object_copies":17796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8257,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4878},{"shards":"0,2","objects":2681}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11618","reported_seq":11737,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:18.605479+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:18.605479+0000","last_peered":"2024-09-15T21:25:18.605479+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:18.605479+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36966883,"num_objects":9026,"num_object_clones":0,"num_object_copies":18052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9477,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2796},{"shards":"2","objects":4936}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877142+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877142+0000","last_peered":"2024-09-15T21:23:35.877142+0000","last_clean":"2024-09-15T21:23:35.877142+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877142+0000","last_undegraded":"2024-09-15T21:23:35.877142+0000","last_fullsized":"2024-09-15T21:23:35.877142+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11642","reported_seq":20101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:18.772204+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:18.772204+0000","last_peered":"2024-09-15T21:25:18.772204+0000","last_clean":"2024-09-15T21:25:18.772204+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:18.772204+0000","last_undegraded":"2024-09-15T21:25:18.772204+0000","last_fullsized":"2024-09-15T21:25:18.772204+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36528128,"num_objects":8918,"num_object_clones":0,"num_object_copies":17836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9335,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11723","reported_seq":11856,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.346766+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:22.346766+0000","last_peered":"2024-09-15T21:25:22.346766+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:22.346766+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8667,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4869},{"shards":"0,2","objects":2715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877348+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877348+0000","last_peered":"2024-09-15T21:23:35.877348+0000","last_clean":"2024-09-15T21:23:35.877348+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877348+0000","last_undegraded":"2024-09-15T21:23:35.877348+0000","last_fullsized":"2024-09-15T21:23:35.877348+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11724","reported_seq":11856,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.228483+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:22.228483+0000","last_peered":"2024-09-15T21:25:22.228483+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:22.228483+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36315136,"num_objects":8866,"num_object_clones":0,"num_object_copies":17732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8633,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4727},{"shards":"0,2","objects":2714}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11623","reported_seq":15325,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:22.357679+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:22.357679+0000","last_peered":"2024-09-15T21:25:22.357679+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:22.357679+0000","last_undegraded":"2024-09-15T21:25:22.357679+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35901440,"num_objects":8765,"num_object_clones":0,"num_object_copies":17530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2299,"num_objects_unfound":0,"num_objects_dirty":8765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8107,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1783,"num_bytes_recovered":7303168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6466},{"shards":"0,2","objects":2299}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877288+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877288+0000","last_peered":"2024-09-15T21:23:35.877288+0000","last_clean":"2024-09-15T21:23:35.877288+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877288+0000","last_undegraded":"2024-09-15T21:23:35.877288+0000","last_fullsized":"2024-09-15T21:23:35.877288+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11871","reported_seq":12010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.334767+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:22.334767+0000","last_peered":"2024-09-15T21:25:22.334767+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:22.334767+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36844486,"num_objects":8997,"num_object_clones":0,"num_object_copies":17994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8997,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8575,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4850},{"shards":"0,2","objects":2715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":586842870,"num_objects":143282,"num_object_clones":0,"num_object_copies":286564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2299,"num_objects_unfound":0,"num_objects_dirty":143282,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":147173,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5948,"num_bytes_recovered":24354816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1255063552,"data_stored":1254907275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":735156,"kb_used_data":420252,"kb_used_omap":4,"kb_used_meta":314875,"kb_avail":93636684,"statfs":{"total":96636764160,"available":95883964416,"internally_reserved":0,"allocated":430338048,"data_stored":430062401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":322432681},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":878856,"kb_used_data":495992,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93492984,"statfs":{"total":96636764160,"available":95736815616,"internally_reserved":0,"allocated":507895808,"data_stored":507620156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592324,"kb_used_data":312116,"kb_used_omap":4,"kb_used_meta":280187,"kb_avail":93779516,"statfs":{"total":96636764160,"available":96030224384,"internally_reserved":0,"allocated":319606784,"data_stored":319287317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286912175},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":506302464,"data_stored":506266334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":319148032,"data_stored":319068087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":429613056,"data_stored":429572854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:24.470 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:25.040 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:25.041 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:25.062 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-15T21:25:24.771552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":583332617,"num_objects":142427,"num_object_clones":0,"num_object_copies":284854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2299,"num_objects_unfound":0,"num_objects_dirty":142427,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":148032,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5949,"num_bytes_recovered":24354835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160840,"ondisk_log_size":160840,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2192832,"kb_used_data":1223184,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":280922688,"statfs":{"total":289910292480,"available":287664832512,"internally_reserved":0,"allocated":1252540416,"data_stored":1251669650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":992857091},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,7,1,8,8,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22085632,"num_objects":-5392,"num_object_clones":0,"num_object_copies":-10784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-163,"num_objects_unfound":0,"num_objects_dirty":-5392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5392,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":667648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002809"},"pg_stats":[{"pgid":"2.f","version":"45'11983","reported_seq":12059,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.281840+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:22.281840+0000","last_peered":"2024-09-15T21:25:22.281840+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:22.281840+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37556224,"num_objects":9169,"num_object_clones":0,"num_object_copies":18338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11983,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5037},{"shards":"0,2","objects":2725}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11779","reported_seq":11885,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:23.734606+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:23.734606+0000","last_peered":"2024-09-15T21:25:23.734606+0000","last_clean":"2024-09-15T21:25:23.734606+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:23.734606+0000","last_undegraded":"2024-09-15T21:25:23.734606+0000","last_fullsized":"2024-09-15T21:25:23.734606+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36434886,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10060,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12054","reported_seq":12181,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:23.749464+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:23.749464+0000","last_peered":"2024-09-15T21:25:23.749464+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:23.749464+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37072920,"num_objects":9052,"num_object_clones":0,"num_object_copies":18104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9052,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10154,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2768},{"shards":"2","objects":4785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11617","reported_seq":11688,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:23.724416+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:23.724416+0000","last_peered":"2024-09-15T21:25:23.724416+0000","last_clean":"2024-09-15T21:25:23.724416+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:23.724416+0000","last_undegraded":"2024-09-15T21:25:23.724416+0000","last_fullsized":"2024-09-15T21:25:23.724416+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35622912,"num_objects":8697,"num_object_clones":0,"num_object_copies":17394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11617,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11632","reported_seq":11765,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:22.349322+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:22.349322+0000","last_peered":"2024-09-15T21:25:22.349322+0000","last_clean":"2024-09-15T21:25:22.349322+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:22.349322+0000","last_undegraded":"2024-09-15T21:25:22.349322+0000","last_fullsized":"2024-09-15T21:25:22.349322+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36344291,"num_objects":8874,"num_object_clones":0,"num_object_copies":17748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8087,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11755","reported_seq":11892,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.332396+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:22.332396+0000","last_peered":"2024-09-15T21:25:22.332396+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:22.332396+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36372963,"num_objects":8881,"num_object_clones":0,"num_object_copies":17762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8431,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4805},{"shards":"0,2","objects":2641}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11772","reported_seq":11906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.328542+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:22.328542+0000","last_peered":"2024-09-15T21:25:22.328542+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:22.328542+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36447174,"num_objects":8900,"num_object_clones":0,"num_object_copies":17800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8083,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4739},{"shards":"0,2","objects":2727}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11853","reported_seq":11981,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:22.355326+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:22.355326+0000","last_peered":"2024-09-15T21:25:22.355326+0000","last_clean":"2024-09-15T21:25:22.355326+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:22.355326+0000","last_undegraded":"2024-09-15T21:25:22.355326+0000","last_fullsized":"2024-09-15T21:25:22.355326+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36925440,"num_objects":9015,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8625,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11741","reported_seq":11868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:23.788557+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:23.788557+0000","last_peered":"2024-09-15T21:25:23.788557+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:23.788557+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35930595,"num_objects":8773,"num_object_clones":0,"num_object_copies":17546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9673,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2604},{"shards":"2","objects":4686}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11584","reported_seq":11714,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.321588+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:22.321588+0000","last_peered":"2024-09-15T21:25:22.321588+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:22.321588+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36446208,"num_objects":8898,"num_object_clones":0,"num_object_copies":17796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8257,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4878},{"shards":"0,2","objects":2681}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11755","reported_seq":11874,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:23.723188+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:23.723188+0000","last_peered":"2024-09-15T21:25:23.723188+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:23.723188+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36405731,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9614,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2659},{"shards":"2","objects":4799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11771","reported_seq":20230,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:23.652486+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:23.652486+0000","last_peered":"2024-09-15T21:25:23.652486+0000","last_clean":"2024-09-15T21:25:23.652486+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:23.652486+0000","last_undegraded":"2024-09-15T21:25:23.652486+0000","last_fullsized":"2024-09-15T21:25:23.652486+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35999744,"num_objects":8789,"num_object_clones":0,"num_object_copies":17578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9464,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11723","reported_seq":11856,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.346766+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:22.346766+0000","last_peered":"2024-09-15T21:25:22.346766+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:22.346766+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8667,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4869},{"shards":"0,2","objects":2715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11724","reported_seq":11856,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.228483+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:22.228483+0000","last_peered":"2024-09-15T21:25:22.228483+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:22.228483+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36315136,"num_objects":8866,"num_object_clones":0,"num_object_copies":17732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8633,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4727},{"shards":"0,2","objects":2714}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11623","reported_seq":15325,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:22.357679+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:22.357679+0000","last_peered":"2024-09-15T21:25:22.357679+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:22.357679+0000","last_undegraded":"2024-09-15T21:25:22.357679+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35901440,"num_objects":8765,"num_object_clones":0,"num_object_copies":17530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2299,"num_objects_unfound":0,"num_objects_dirty":8765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8107,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1783,"num_bytes_recovered":7303168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6466},{"shards":"0,2","objects":2299}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11871","reported_seq":12010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.334767+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:22.334767+0000","last_peered":"2024-09-15T21:25:22.334767+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:22.334767+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36844486,"num_objects":8997,"num_object_clones":0,"num_object_copies":17994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8997,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8575,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4850},{"shards":"0,2","objects":2715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":583332598,"num_objects":142425,"num_object_clones":0,"num_object_copies":284850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2299,"num_objects_unfound":0,"num_objects_dirty":142425,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":148030,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5948,"num_bytes_recovered":24354816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1249652736,"data_stored":1249496459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160837,"ondisk_log_size":160837,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718268,"kb_used_data":415852,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93653572,"statfs":{"total":96636764160,"available":95901257728,"internally_reserved":0,"allocated":425832448,"data_stored":425556801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":878856,"kb_used_data":495992,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93492984,"statfs":{"total":96636764160,"available":95736815616,"internally_reserved":0,"allocated":507895808,"data_stored":507620156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595708,"kb_used_data":311340,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93776132,"statfs":{"total":96636764160,"available":96026759168,"internally_reserved":0,"allocated":318812160,"data_stored":318492693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":506302464,"data_stored":506266334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318394368,"data_stored":318314423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":424955904,"data_stored":424915702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:25.063 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:25.063 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'11983", 'reported_seq': 12059, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:22.281840+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:25:22.281840+0000', 'last_peered': '2024-09-15T21:25:22.281840+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:25:22.281840+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37556224, 'num_objects': 9169, 'num_object_clones': 0, 'num_object_copies': 18338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9169, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11983, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5037}, {'shards': '0,2', 'objects': 2725}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.064 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:25:25.064 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'12054", 'reported_seq': 12181, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:23.749464+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:25:23.749464+0000', 'last_peered': '2024-09-15T21:25:23.749464+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:25:23.749464+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37072920, 'num_objects': 9052, 'num_object_clones': 0, 'num_object_copies': 18104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9052, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10154, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2768}, {'shards': '2', 'objects': 4785}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:25.064 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:25.064 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'11755", 'reported_seq': 11892, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:22.332396+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:25:22.332396+0000', 'last_peered': '2024-09-15T21:25:22.332396+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:25:22.332396+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36372963, 'num_objects': 8881, 'num_object_clones': 0, 'num_object_copies': 17762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8881, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8431, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4805}, {'shards': '0,2', 'objects': 2641}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.064 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:25.064 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'11772", 'reported_seq': 11906, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:22.328542+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:25:22.328542+0000', 'last_peered': '2024-09-15T21:25:22.328542+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:25:22.328542+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36447174, 'num_objects': 8900, 'num_object_clones': 0, 'num_object_copies': 17800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8083, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4739}, {'shards': '0,2', 'objects': 2727}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.064 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:25:25.064 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'11741", 'reported_seq': 11868, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:23.788557+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:25:23.788557+0000', 'last_peered': '2024-09-15T21:25:23.788557+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:25:23.788557+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35930595, 'num_objects': 8773, 'num_object_clones': 0, 'num_object_copies': 17546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8773, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9673, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2604}, {'shards': '2', 'objects': 4686}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:25.064 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:25.064 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11584", 'reported_seq': 11714, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:22.321588+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:25:22.321588+0000', 'last_peered': '2024-09-15T21:25:22.321588+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:25:22.321588+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36446208, 'num_objects': 8898, 'num_object_clones': 0, 'num_object_copies': 17796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8898, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8257, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4878}, {'shards': '0,2', 'objects': 2681}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:25.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'11755", 'reported_seq': 11874, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:23.723188+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:25:23.723188+0000', 'last_peered': '2024-09-15T21:25:23.723188+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:25:23.723188+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36405731, 'num_objects': 8889, 'num_object_clones': 0, 'num_object_copies': 17778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8889, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9614, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2659}, {'shards': '2', 'objects': 4799}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:25.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:25.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11723", 'reported_seq': 11856, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:22.346766+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:25:22.346766+0000', 'last_peered': '2024-09-15T21:25:22.346766+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:25:22.346766+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36712448, 'num_objects': 8963, 'num_object_clones': 0, 'num_object_copies': 17926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8963, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8667, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4869}, {'shards': '0,2', 'objects': 2715}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:25:25.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11724", 'reported_seq': 11856, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:22.228483+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:25:22.228483+0000', 'last_peered': '2024-09-15T21:25:22.228483+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:25:22.228483+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36315136, 'num_objects': 8866, 'num_object_clones': 0, 'num_object_copies': 17732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8866, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8633, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4727}, {'shards': '0,2', 'objects': 2714}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:25.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11623", 'reported_seq': 15325, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:25:22.357679+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:25:22.357679+0000', 'last_peered': '2024-09-15T21:25:22.357679+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:25:22.357679+0000', 'last_undegraded': '2024-09-15T21:25:22.357679+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35901440, 'num_objects': 8765, 'num_object_clones': 0, 'num_object_copies': 17530, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2299, 'num_objects_unfound': 0, 'num_objects_dirty': 8765, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8107, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1783, 'num_bytes_recovered': 7303168, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6466}, {'shards': '0,2', 'objects': 2299}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.065 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:25:25.065 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11871", 'reported_seq': 12010, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:22.334767+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:25:22.334767+0000', 'last_peered': '2024-09-15T21:25:22.334767+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:25:22.334767+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36844486, 'num_objects': 8997, 'num_object_clones': 0, 'num_object_copies': 17994, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8997, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8575, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4850}, {'shards': '0,2', 'objects': 2715}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:25.066 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:25.765 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:25.765 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/284854 objects degraded (14.472%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":377,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":142427,"data_bytes":583332617,"bytes_used":2245459968,"bytes_avail":287664832512,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":284854,"degraded_ratio":0.14472326174110245,"misplaced_objects":2299,"misplaced_total":284854,"misplaced_ratio":0.0080708011823600861,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":55624,"recovering_keys_per_sec":0,"num_objects_recovered":163,"num_bytes_recovered":667648,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":449},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:25:25.790 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:25.791 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:26.360 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:26.360 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:26.381 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-15T21:25:24.771552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":583332617,"num_objects":142427,"num_object_clones":0,"num_object_copies":284854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2299,"num_objects_unfound":0,"num_objects_dirty":142427,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":148032,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5949,"num_bytes_recovered":24354835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160840,"ondisk_log_size":160840,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2192832,"kb_used_data":1223184,"kb_used_omap":12,"kb_used_meta":969587,"kb_avail":280922688,"statfs":{"total":289910292480,"available":287664832512,"internally_reserved":0,"allocated":1252540416,"data_stored":1251669650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":992857091},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,7,1,8,8,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22085632,"num_objects":-5392,"num_object_clones":0,"num_object_copies":-10784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-163,"num_objects_unfound":0,"num_objects_dirty":-5392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5392,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":667648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002809"},"pg_stats":[{"pgid":"2.f","version":"45'11983","reported_seq":12059,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.281840+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:22.281840+0000","last_peered":"2024-09-15T21:25:22.281840+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:22.281840+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37556224,"num_objects":9169,"num_object_clones":0,"num_object_copies":18338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11983,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5037},{"shards":"0,2","objects":2725}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11779","reported_seq":11885,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:23.734606+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:23.734606+0000","last_peered":"2024-09-15T21:25:23.734606+0000","last_clean":"2024-09-15T21:25:23.734606+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:23.734606+0000","last_undegraded":"2024-09-15T21:25:23.734606+0000","last_fullsized":"2024-09-15T21:25:23.734606+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36434886,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10060,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12054","reported_seq":12181,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:23.749464+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:23.749464+0000","last_peered":"2024-09-15T21:25:23.749464+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:23.749464+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37072920,"num_objects":9052,"num_object_clones":0,"num_object_copies":18104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9052,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10154,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2768},{"shards":"2","objects":4785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11617","reported_seq":11688,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:23.724416+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:23.724416+0000","last_peered":"2024-09-15T21:25:23.724416+0000","last_clean":"2024-09-15T21:25:23.724416+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:23.724416+0000","last_undegraded":"2024-09-15T21:25:23.724416+0000","last_fullsized":"2024-09-15T21:25:23.724416+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35622912,"num_objects":8697,"num_object_clones":0,"num_object_copies":17394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11617,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11632","reported_seq":11765,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:22.349322+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:22.349322+0000","last_peered":"2024-09-15T21:25:22.349322+0000","last_clean":"2024-09-15T21:25:22.349322+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:22.349322+0000","last_undegraded":"2024-09-15T21:25:22.349322+0000","last_fullsized":"2024-09-15T21:25:22.349322+0000","mapping_epoch":34,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36344291,"num_objects":8874,"num_object_clones":0,"num_object_copies":17748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8087,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11755","reported_seq":11892,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.332396+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:22.332396+0000","last_peered":"2024-09-15T21:25:22.332396+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:22.332396+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36372963,"num_objects":8881,"num_object_clones":0,"num_object_copies":17762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8431,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4805},{"shards":"0,2","objects":2641}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11772","reported_seq":11906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.328542+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:22.328542+0000","last_peered":"2024-09-15T21:25:22.328542+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:22.328542+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36447174,"num_objects":8900,"num_object_clones":0,"num_object_copies":17800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8083,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4739},{"shards":"0,2","objects":2727}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11853","reported_seq":11981,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:22.355326+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:22.355326+0000","last_peered":"2024-09-15T21:25:22.355326+0000","last_clean":"2024-09-15T21:25:22.355326+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:22.355326+0000","last_undegraded":"2024-09-15T21:25:22.355326+0000","last_fullsized":"2024-09-15T21:25:22.355326+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36925440,"num_objects":9015,"num_object_clones":0,"num_object_copies":18030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8625,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11741","reported_seq":11868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:23.788557+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:23.788557+0000","last_peered":"2024-09-15T21:25:23.788557+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:23.788557+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35930595,"num_objects":8773,"num_object_clones":0,"num_object_copies":17546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9673,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2604},{"shards":"2","objects":4686}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11584","reported_seq":11714,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.321588+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:22.321588+0000","last_peered":"2024-09-15T21:25:22.321588+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:22.321588+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36446208,"num_objects":8898,"num_object_clones":0,"num_object_copies":17796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8257,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4878},{"shards":"0,2","objects":2681}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11755","reported_seq":11874,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:23.723188+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:23.723188+0000","last_peered":"2024-09-15T21:25:23.723188+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:23.723188+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36405731,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9614,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2659},{"shards":"2","objects":4799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11771","reported_seq":20230,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:23.652486+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:23.652486+0000","last_peered":"2024-09-15T21:25:23.652486+0000","last_clean":"2024-09-15T21:25:23.652486+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:23.652486+0000","last_undegraded":"2024-09-15T21:25:23.652486+0000","last_fullsized":"2024-09-15T21:25:23.652486+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35999744,"num_objects":8789,"num_object_clones":0,"num_object_copies":17578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9464,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11723","reported_seq":11856,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.346766+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:22.346766+0000","last_peered":"2024-09-15T21:25:22.346766+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:22.346766+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36712448,"num_objects":8963,"num_object_clones":0,"num_object_copies":17926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8667,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4869},{"shards":"0,2","objects":2715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11724","reported_seq":11856,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.228483+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:22.228483+0000","last_peered":"2024-09-15T21:25:22.228483+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:22.228483+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36315136,"num_objects":8866,"num_object_clones":0,"num_object_copies":17732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8633,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4727},{"shards":"0,2","objects":2714}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11623","reported_seq":15325,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:22.357679+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:22.357679+0000","last_peered":"2024-09-15T21:25:22.357679+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:22.357679+0000","last_undegraded":"2024-09-15T21:25:22.357679+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35901440,"num_objects":8765,"num_object_clones":0,"num_object_copies":17530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2299,"num_objects_unfound":0,"num_objects_dirty":8765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8107,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1783,"num_bytes_recovered":7303168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6466},{"shards":"0,2","objects":2299}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11871","reported_seq":12010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:22.334767+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:22.334767+0000","last_peered":"2024-09-15T21:25:22.334767+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:22.334767+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36844486,"num_objects":8997,"num_object_clones":0,"num_object_copies":17994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8997,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8575,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4850},{"shards":"0,2","objects":2715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":583332598,"num_objects":142425,"num_object_clones":0,"num_object_copies":284850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2299,"num_objects_unfound":0,"num_objects_dirty":142425,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":148030,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5948,"num_bytes_recovered":24354816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1249652736,"data_stored":1249496459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160837,"ondisk_log_size":160837,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718268,"kb_used_data":415852,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93653572,"statfs":{"total":96636764160,"available":95901257728,"internally_reserved":0,"allocated":425832448,"data_stored":425556801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":878856,"kb_used_data":495992,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93492984,"statfs":{"total":96636764160,"available":95736815616,"internally_reserved":0,"allocated":507895808,"data_stored":507620156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":595708,"kb_used_data":311340,"kb_used_omap":4,"kb_used_meta":284347,"kb_avail":93776132,"statfs":{"total":96636764160,"available":96026759168,"internally_reserved":0,"allocated":318812160,"data_stored":318492693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291172015},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":506302464,"data_stored":506266334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318394368,"data_stored":318314423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":424955904,"data_stored":424915702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:29.384 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:29.963 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:29.964 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:29.992 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-15T21:25:28.772497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":577667849,"num_objects":141044,"num_object_clones":0,"num_object_copies":282088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2212,"num_objects_unfound":0,"num_objects_dirty":141044,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":149415,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6036,"num_bytes_recovered":24711187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160823,"ondisk_log_size":160823,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2203432,"kb_used_data":1215736,"kb_used_omap":12,"kb_used_meta":987635,"kb_avail":280912088,"statfs":{"total":289910292480,"available":287653978112,"internally_reserved":0,"allocated":1244913664,"data_stored":1244042898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1011338243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,7,8,13,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24297472,"num_objects":-5932,"num_object_clones":0,"num_object_copies":-11864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-250,"num_objects_unfound":0,"num_objects_dirty":-5932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5932,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":1024000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002714"},"pg_stats":[{"pgid":"2.f","version":"45'12123","reported_seq":12193,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.326886+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:27.326886+0000","last_peered":"2024-09-15T21:25:27.326886+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:27.326886+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36982784,"num_objects":9029,"num_object_clones":0,"num_object_copies":18058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12123,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4897},{"shards":"0,2","objects":2585}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11779","reported_seq":11885,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:23.734606+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:23.734606+0000","last_peered":"2024-09-15T21:25:23.734606+0000","last_clean":"2024-09-15T21:25:23.734606+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:23.734606+0000","last_undegraded":"2024-09-15T21:25:23.734606+0000","last_fullsized":"2024-09-15T21:25:23.734606+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36434886,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10060,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12054","reported_seq":12181,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:23.749464+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:23.749464+0000","last_peered":"2024-09-15T21:25:23.749464+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:23.749464+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37072920,"num_objects":9052,"num_object_clones":0,"num_object_copies":18104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9052,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10154,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2768},{"shards":"2","objects":4785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11617","reported_seq":11688,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:23.724416+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:23.724416+0000","last_peered":"2024-09-15T21:25:23.724416+0000","last_clean":"2024-09-15T21:25:23.724416+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:23.724416+0000","last_undegraded":"2024-09-15T21:25:23.724416+0000","last_fullsized":"2024-09-15T21:25:23.724416+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35622912,"num_objects":8697,"num_object_clones":0,"num_object_copies":17394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11617,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11761","reported_seq":11894,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:27.407166+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:27.407166+0000","last_peered":"2024-09-15T21:25:27.407166+0000","last_clean":"2024-09-15T21:25:27.407166+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:27.407166+0000","last_undegraded":"2024-09-15T21:25:27.407166+0000","last_fullsized":"2024-09-15T21:25:27.407166+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35815907,"num_objects":8745,"num_object_clones":0,"num_object_copies":17490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8216,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11894","reported_seq":12031,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.352473+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:27.352473+0000","last_peered":"2024-09-15T21:25:27.352473+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:27.352473+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35803619,"num_objects":8742,"num_object_clones":0,"num_object_copies":17484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8742,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8570,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4666},{"shards":"0,2","objects":2502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11924","reported_seq":12061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.397851+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:27.397851+0000","last_peered":"2024-09-15T21:25:27.397851+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:27.397851+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35824582,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8235,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4587},{"shards":"0,2","objects":2575}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12016","reported_seq":12145,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:27.345975+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:27.345975+0000","last_peered":"2024-09-15T21:25:27.345975+0000","last_clean":"2024-09-15T21:25:27.345975+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:27.345975+0000","last_undegraded":"2024-09-15T21:25:27.345975+0000","last_fullsized":"2024-09-15T21:25:27.345975+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36257792,"num_objects":8852,"num_object_clones":0,"num_object_copies":17704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8788,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11741","reported_seq":11868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:23.788557+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:23.788557+0000","last_peered":"2024-09-15T21:25:23.788557+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:23.788557+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35930595,"num_objects":8773,"num_object_clones":0,"num_object_copies":17546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9673,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2604},{"shards":"2","objects":4686}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11730","reported_seq":11861,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.276221+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:27.276221+0000","last_peered":"2024-09-15T21:25:27.276221+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:27.276221+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8403,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4732},{"shards":"0,2","objects":2535}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11755","reported_seq":11874,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:23.723188+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:23.723188+0000","last_peered":"2024-09-15T21:25:23.723188+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:23.723188+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36405731,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9614,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2659},{"shards":"2","objects":4799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11771","reported_seq":20230,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:23.652486+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:23.652486+0000","last_peered":"2024-09-15T21:25:23.652486+0000","last_clean":"2024-09-15T21:25:23.652486+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:23.652486+0000","last_undegraded":"2024-09-15T21:25:23.652486+0000","last_fullsized":"2024-09-15T21:25:23.652486+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35999744,"num_objects":8789,"num_object_clones":0,"num_object_copies":17578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9464,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11852","reported_seq":11986,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.222070+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:27.222070+0000","last_peered":"2024-09-15T21:25:27.222070+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:27.222070+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36184064,"num_objects":8834,"num_object_clones":0,"num_object_copies":17668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8796,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4740},{"shards":"0,2","objects":2586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11859","reported_seq":11992,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.329230+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:27.329230+0000","last_peered":"2024-09-15T21:25:27.329230+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:27.329230+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35762176,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8768,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4592},{"shards":"0,2","objects":2579}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11756","reported_seq":15633,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:27.391386+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:27.391386+0000","last_peered":"2024-09-15T21:25:27.391386+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:27.391386+0000","last_undegraded":"2024-09-15T21:25:27.391386+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35356672,"num_objects":8632,"num_object_clones":0,"num_object_copies":17264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2212,"num_objects_unfound":0,"num_objects_dirty":8632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8240,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1870,"num_bytes_recovered":7659520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6420},{"shards":"0,2","objects":2212}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11988","reported_seq":12127,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.342516+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:27.342516+0000","last_peered":"2024-09-15T21:25:27.342516+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:27.342516+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36365254,"num_objects":8880,"num_object_clones":0,"num_object_copies":17760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8880,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8692,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4733},{"shards":"0,2","objects":2598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":577667830,"num_objects":141042,"num_object_clones":0,"num_object_copies":282084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2212,"num_objects_unfound":0,"num_objects_dirty":141042,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":149413,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6035,"num_bytes_recovered":24711168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1241862144,"data_stored":1241705867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160820,"ondisk_log_size":160820,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718268,"kb_used_data":415852,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93653572,"statfs":{"total":96636764160,"available":95901257728,"internally_reserved":0,"allocated":425832448,"data_stored":425556801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863900,"kb_used_data":489356,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93507940,"statfs":{"total":96636764160,"available":95752130560,"internally_reserved":0,"allocated":501100544,"data_stored":500824892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,5,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621264,"kb_used_data":310528,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93750576,"statfs":{"total":96636764160,"available":96000589824,"internally_reserved":0,"allocated":317980672,"data_stored":317661205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":499396608,"data_stored":499360478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317509632,"data_stored":317429687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":424955904,"data_stored":424915702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:29.993 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:30.560 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:30.561 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:30.589 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-09-15T21:25:28.772497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":577667849,"num_objects":141044,"num_object_clones":0,"num_object_copies":282088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2212,"num_objects_unfound":0,"num_objects_dirty":141044,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":149415,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6036,"num_bytes_recovered":24711187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160823,"ondisk_log_size":160823,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2203432,"kb_used_data":1215736,"kb_used_omap":12,"kb_used_meta":987635,"kb_avail":280912088,"statfs":{"total":289910292480,"available":287653978112,"internally_reserved":0,"allocated":1244913664,"data_stored":1244042898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1011338243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,7,8,13,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24297472,"num_objects":-5932,"num_object_clones":0,"num_object_copies":-11864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-250,"num_objects_unfound":0,"num_objects_dirty":-5932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5932,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":1024000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002714"},"pg_stats":[{"pgid":"2.f","version":"45'12123","reported_seq":12193,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.326886+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:27.326886+0000","last_peered":"2024-09-15T21:25:27.326886+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:27.326886+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36982784,"num_objects":9029,"num_object_clones":0,"num_object_copies":18058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12123,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4897},{"shards":"0,2","objects":2585}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11779","reported_seq":11885,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:23.734606+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:23.734606+0000","last_peered":"2024-09-15T21:25:23.734606+0000","last_clean":"2024-09-15T21:25:23.734606+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:23.734606+0000","last_undegraded":"2024-09-15T21:25:23.734606+0000","last_fullsized":"2024-09-15T21:25:23.734606+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36434886,"num_objects":8897,"num_object_clones":0,"num_object_copies":17794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8897,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10060,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12054","reported_seq":12181,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:23.749464+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:23.749464+0000","last_peered":"2024-09-15T21:25:23.749464+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:23.749464+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37072920,"num_objects":9052,"num_object_clones":0,"num_object_copies":18104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9052,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10154,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2768},{"shards":"2","objects":4785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11617","reported_seq":11688,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:23.724416+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:23.724416+0000","last_peered":"2024-09-15T21:25:23.724416+0000","last_clean":"2024-09-15T21:25:23.724416+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:23.724416+0000","last_undegraded":"2024-09-15T21:25:23.724416+0000","last_fullsized":"2024-09-15T21:25:23.724416+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35622912,"num_objects":8697,"num_object_clones":0,"num_object_copies":17394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11617,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11761","reported_seq":11894,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:27.407166+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:27.407166+0000","last_peered":"2024-09-15T21:25:27.407166+0000","last_clean":"2024-09-15T21:25:27.407166+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:27.407166+0000","last_undegraded":"2024-09-15T21:25:27.407166+0000","last_fullsized":"2024-09-15T21:25:27.407166+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35815907,"num_objects":8745,"num_object_clones":0,"num_object_copies":17490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8216,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11894","reported_seq":12031,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.352473+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:27.352473+0000","last_peered":"2024-09-15T21:25:27.352473+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:27.352473+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35803619,"num_objects":8742,"num_object_clones":0,"num_object_copies":17484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8742,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8570,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4666},{"shards":"0,2","objects":2502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11924","reported_seq":12061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.397851+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:27.397851+0000","last_peered":"2024-09-15T21:25:27.397851+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:27.397851+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35824582,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8235,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4587},{"shards":"0,2","objects":2575}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12016","reported_seq":12145,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:27.345975+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:27.345975+0000","last_peered":"2024-09-15T21:25:27.345975+0000","last_clean":"2024-09-15T21:25:27.345975+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:27.345975+0000","last_undegraded":"2024-09-15T21:25:27.345975+0000","last_fullsized":"2024-09-15T21:25:27.345975+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36257792,"num_objects":8852,"num_object_clones":0,"num_object_copies":17704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8788,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11741","reported_seq":11868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:23.788557+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:23.788557+0000","last_peered":"2024-09-15T21:25:23.788557+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:23.788557+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35930595,"num_objects":8773,"num_object_clones":0,"num_object_copies":17546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9673,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2604},{"shards":"2","objects":4686}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11730","reported_seq":11861,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.276221+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:27.276221+0000","last_peered":"2024-09-15T21:25:27.276221+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:27.276221+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8403,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4732},{"shards":"0,2","objects":2535}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11755","reported_seq":11874,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:23.723188+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:23.723188+0000","last_peered":"2024-09-15T21:25:23.723188+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:23.723188+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36405731,"num_objects":8889,"num_object_clones":0,"num_object_copies":17778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8889,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9614,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2659},{"shards":"2","objects":4799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11771","reported_seq":20230,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:23.652486+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:23.652486+0000","last_peered":"2024-09-15T21:25:23.652486+0000","last_clean":"2024-09-15T21:25:23.652486+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:23.652486+0000","last_undegraded":"2024-09-15T21:25:23.652486+0000","last_fullsized":"2024-09-15T21:25:23.652486+0000","mapping_epoch":44,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35999744,"num_objects":8789,"num_object_clones":0,"num_object_copies":17578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8789,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9464,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11852","reported_seq":11986,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.222070+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:27.222070+0000","last_peered":"2024-09-15T21:25:27.222070+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:27.222070+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36184064,"num_objects":8834,"num_object_clones":0,"num_object_copies":17668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8796,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4740},{"shards":"0,2","objects":2586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11859","reported_seq":11992,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.329230+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:27.329230+0000","last_peered":"2024-09-15T21:25:27.329230+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:27.329230+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35762176,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8768,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4592},{"shards":"0,2","objects":2579}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11756","reported_seq":15633,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:27.391386+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:27.391386+0000","last_peered":"2024-09-15T21:25:27.391386+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:27.391386+0000","last_undegraded":"2024-09-15T21:25:27.391386+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35356672,"num_objects":8632,"num_object_clones":0,"num_object_copies":17264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2212,"num_objects_unfound":0,"num_objects_dirty":8632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8240,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1870,"num_bytes_recovered":7659520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6420},{"shards":"0,2","objects":2212}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11988","reported_seq":12127,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.342516+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:27.342516+0000","last_peered":"2024-09-15T21:25:27.342516+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:27.342516+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36365254,"num_objects":8880,"num_object_clones":0,"num_object_copies":17760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8880,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8692,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4733},{"shards":"0,2","objects":2598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":577667830,"num_objects":141042,"num_object_clones":0,"num_object_copies":282084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2212,"num_objects_unfound":0,"num_objects_dirty":141042,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":149413,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6035,"num_bytes_recovered":24711168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1241862144,"data_stored":1241705867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160820,"ondisk_log_size":160820,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":718268,"kb_used_data":415852,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93653572,"statfs":{"total":96636764160,"available":95901257728,"internally_reserved":0,"allocated":425832448,"data_stored":425556801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863900,"kb_used_data":489356,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93507940,"statfs":{"total":96636764160,"available":95752130560,"internally_reserved":0,"allocated":501100544,"data_stored":500824892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,5,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621264,"kb_used_data":310528,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93750576,"statfs":{"total":96636764160,"available":96000589824,"internally_reserved":0,"allocated":317980672,"data_stored":317661205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":499396608,"data_stored":499360478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317509632,"data_stored":317429687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":424955904,"data_stored":424915702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:30.591 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:30.591 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12123", 'reported_seq': 12193, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:27.326886+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:25:27.326886+0000', 'last_peered': '2024-09-15T21:25:27.326886+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:25:27.326886+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36982784, 'num_objects': 9029, 'num_object_clones': 0, 'num_object_copies': 18058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9029, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12123, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4897}, {'shards': '0,2', 'objects': 2585}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:30.591 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:25:30.591 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'12054", 'reported_seq': 12181, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:23.749464+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:25:23.749464+0000', 'last_peered': '2024-09-15T21:25:23.749464+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:25:23.749464+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37072920, 'num_objects': 9052, 'num_object_clones': 0, 'num_object_copies': 18104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9052, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10154, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2768}, {'shards': '2', 'objects': 4785}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:30.591 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:30.591 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'11894", 'reported_seq': 12031, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:27.352473+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:25:27.352473+0000', 'last_peered': '2024-09-15T21:25:27.352473+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:25:27.352473+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35803619, 'num_objects': 8742, 'num_object_clones': 0, 'num_object_copies': 17484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8742, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8570, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4666}, {'shards': '0,2', 'objects': 2502}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:30.591 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:30.592 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'11924", 'reported_seq': 12061, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:27.397851+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:25:27.397851+0000', 'last_peered': '2024-09-15T21:25:27.397851+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:25:27.397851+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35824582, 'num_objects': 8748, 'num_object_clones': 0, 'num_object_copies': 17496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8748, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8235, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4587}, {'shards': '0,2', 'objects': 2575}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:30.592 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:25:30.592 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'11741", 'reported_seq': 11868, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:23.788557+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:25:23.788557+0000', 'last_peered': '2024-09-15T21:25:23.788557+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:25:23.788557+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35930595, 'num_objects': 8773, 'num_object_clones': 0, 'num_object_copies': 17546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8773, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9673, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2604}, {'shards': '2', 'objects': 4686}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:30.592 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:30.592 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11730", 'reported_seq': 11861, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:27.276221+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:25:27.276221+0000', 'last_peered': '2024-09-15T21:25:27.276221+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:25:27.276221+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35848192, 'num_objects': 8752, 'num_object_clones': 0, 'num_object_copies': 17504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8752, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8403, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4732}, {'shards': '0,2', 'objects': 2535}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:30.592 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:30.592 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'11755", 'reported_seq': 11874, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:23.723188+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:25:23.723188+0000', 'last_peered': '2024-09-15T21:25:23.723188+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:25:23.723188+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36405731, 'num_objects': 8889, 'num_object_clones': 0, 'num_object_copies': 17778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8889, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9614, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2659}, {'shards': '2', 'objects': 4799}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:30.592 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:30.592 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'11852", 'reported_seq': 11986, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:27.222070+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:25:27.222070+0000', 'last_peered': '2024-09-15T21:25:27.222070+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:25:27.222070+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36184064, 'num_objects': 8834, 'num_object_clones': 0, 'num_object_copies': 17668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8834, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8796, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4740}, {'shards': '0,2', 'objects': 2586}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:30.593 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:25:30.593 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11859", 'reported_seq': 11992, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:27.329230+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:25:27.329230+0000', 'last_peered': '2024-09-15T21:25:27.329230+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:25:27.329230+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35762176, 'num_objects': 8731, 'num_object_clones': 0, 'num_object_copies': 17462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8731, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8768, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4592}, {'shards': '0,2', 'objects': 2579}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:30.593 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:30.593 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11756", 'reported_seq': 15633, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:25:27.391386+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:25:27.391386+0000', 'last_peered': '2024-09-15T21:25:27.391386+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:25:27.391386+0000', 'last_undegraded': '2024-09-15T21:25:27.391386+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35356672, 'num_objects': 8632, 'num_object_clones': 0, 'num_object_copies': 17264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2212, 'num_objects_unfound': 0, 'num_objects_dirty': 8632, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8240, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1870, 'num_bytes_recovered': 7659520, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6420}, {'shards': '0,2', 'objects': 2212}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:30.593 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:25:30.593 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'11988", 'reported_seq': 12127, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:27.342516+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:25:27.342516+0000', 'last_peered': '2024-09-15T21:25:27.342516+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:25:27.342516+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36365254, 'num_objects': 8880, 'num_object_clones': 0, 'num_object_copies': 17760, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8880, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8692, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4733}, {'shards': '0,2', 'objects': 2598}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:30.593 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:31.310 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:31.310 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/280364 objects degraded (14.704%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":383,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":140182,"data_bytes":574137097,"bytes_used":2255695872,"bytes_avail":287654596608,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":280364,"degraded_ratio":0.14704098957070094,"misplaced_objects":2212,"misplaced_total":280364,"misplaced_ratio":0.0078897433336662347,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":58695,"recovering_keys_per_sec":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":447},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:25:31.341 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:31.341 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:31.910 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:31.911 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:31.932 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-15T21:25:30.773102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":574137097,"num_objects":140182,"num_object_clones":0,"num_object_copies":280364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2212,"num_objects_unfound":0,"num_objects_dirty":140182,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":150277,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6036,"num_bytes_recovered":24711187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160785,"ondisk_log_size":160785,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2202828,"kb_used_data":1210964,"kb_used_omap":12,"kb_used_meta":991795,"kb_avail":280912692,"statfs":{"total":289910292480,"available":287654596608,"internally_reserved":0,"allocated":1240027136,"data_stored":1239156370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1015598083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,7,8,13,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21995520,"num_objects":-5370,"num_object_clones":0,"num_object_copies":-10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-172,"num_objects_unfound":0,"num_objects_dirty":-5370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5370,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002896"},"pg_stats":[{"pgid":"2.f","version":"45'12123","reported_seq":12193,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.326886+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:27.326886+0000","last_peered":"2024-09-15T21:25:27.326886+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:27.326886+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36982784,"num_objects":9029,"num_object_clones":0,"num_object_copies":18058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12123,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4897},{"shards":"0,2","objects":2585}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11915","reported_seq":12021,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:28.740998+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:28.740998+0000","last_peered":"2024-09-15T21:25:28.740998+0000","last_clean":"2024-09-15T21:25:28.740998+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:28.740998+0000","last_undegraded":"2024-09-15T21:25:28.740998+0000","last_fullsized":"2024-09-15T21:25:28.740998+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35877830,"num_objects":8761,"num_object_clones":0,"num_object_copies":17522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8761,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10196,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12184","reported_seq":12311,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:28.658149+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:28.658149+0000","last_peered":"2024-09-15T21:25:28.658149+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:28.658149+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36540440,"num_objects":8922,"num_object_clones":0,"num_object_copies":17844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8922,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10284,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2638},{"shards":"2","objects":4655}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11767","reported_seq":11838,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:28.549196+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:28.549196+0000","last_peered":"2024-09-15T21:25:28.549196+0000","last_clean":"2024-09-15T21:25:28.549196+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:28.549196+0000","last_undegraded":"2024-09-15T21:25:28.549196+0000","last_fullsized":"2024-09-15T21:25:28.549196+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35008512,"num_objects":8547,"num_object_clones":0,"num_object_copies":17094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11767,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11761","reported_seq":11894,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:27.407166+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:27.407166+0000","last_peered":"2024-09-15T21:25:27.407166+0000","last_clean":"2024-09-15T21:25:27.407166+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:27.407166+0000","last_undegraded":"2024-09-15T21:25:27.407166+0000","last_fullsized":"2024-09-15T21:25:27.407166+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35815907,"num_objects":8745,"num_object_clones":0,"num_object_copies":17490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8216,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11894","reported_seq":12031,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.352473+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:27.352473+0000","last_peered":"2024-09-15T21:25:27.352473+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:27.352473+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35803619,"num_objects":8742,"num_object_clones":0,"num_object_copies":17484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8742,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8570,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4666},{"shards":"0,2","objects":2502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11924","reported_seq":12061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.397851+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:27.397851+0000","last_peered":"2024-09-15T21:25:27.397851+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:27.397851+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35824582,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8235,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4587},{"shards":"0,2","objects":2575}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12016","reported_seq":12145,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:27.345975+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:27.345975+0000","last_peered":"2024-09-15T21:25:27.345975+0000","last_clean":"2024-09-15T21:25:27.345975+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:27.345975+0000","last_undegraded":"2024-09-15T21:25:27.345975+0000","last_fullsized":"2024-09-15T21:25:27.345975+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36257792,"num_objects":8852,"num_object_clones":0,"num_object_copies":17704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8788,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11875","reported_seq":12002,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:28.709866+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:28.709866+0000","last_peered":"2024-09-15T21:25:28.709866+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:28.709866+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35381731,"num_objects":8639,"num_object_clones":0,"num_object_copies":17278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8639,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9807,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2470},{"shards":"2","objects":4552}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11730","reported_seq":11861,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.276221+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:27.276221+0000","last_peered":"2024-09-15T21:25:27.276221+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:27.276221+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35848192,"num_objects":8752,"num_object_clones":0,"num_object_copies":17504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8752,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8403,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4732},{"shards":"0,2","objects":2535}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11906","reported_seq":12025,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:28.721033+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:28.721033+0000","last_peered":"2024-09-15T21:25:28.721033+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:28.721033+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35787235,"num_objects":8738,"num_object_clones":0,"num_object_copies":17476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9765,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2508},{"shards":"2","objects":4648}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11932","reported_seq":20391,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:28.722711+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:28.722711+0000","last_peered":"2024-09-15T21:25:28.722711+0000","last_clean":"2024-09-15T21:25:28.722711+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:28.722711+0000","last_undegraded":"2024-09-15T21:25:28.722711+0000","last_fullsized":"2024-09-15T21:25:28.722711+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35340288,"num_objects":8628,"num_object_clones":0,"num_object_copies":17256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9625,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11852","reported_seq":11986,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.222070+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:27.222070+0000","last_peered":"2024-09-15T21:25:27.222070+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:27.222070+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36184064,"num_objects":8834,"num_object_clones":0,"num_object_copies":17668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8796,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4740},{"shards":"0,2","objects":2586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11859","reported_seq":11992,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.329230+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:27.329230+0000","last_peered":"2024-09-15T21:25:27.329230+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:27.329230+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35762176,"num_objects":8731,"num_object_clones":0,"num_object_copies":17462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8768,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4592},{"shards":"0,2","objects":2579}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11756","reported_seq":15633,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:27.391386+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:27.391386+0000","last_peered":"2024-09-15T21:25:27.391386+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:27.391386+0000","last_undegraded":"2024-09-15T21:25:27.391386+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35356672,"num_objects":8632,"num_object_clones":0,"num_object_copies":17264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2212,"num_objects_unfound":0,"num_objects_dirty":8632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8240,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1870,"num_bytes_recovered":7659520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6420},{"shards":"0,2","objects":2212}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877303+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877303+0000","last_peered":"2024-09-15T21:23:35.877303+0000","last_clean":"2024-09-15T21:23:35.877303+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877303+0000","last_undegraded":"2024-09-15T21:23:35.877303+0000","last_fullsized":"2024-09-15T21:23:35.877303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11988","reported_seq":12127,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:27.342516+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:27.342516+0000","last_peered":"2024-09-15T21:25:27.342516+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:27.342516+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36365254,"num_objects":8880,"num_object_clones":0,"num_object_copies":17760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8880,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8692,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4733},{"shards":"0,2","objects":2598}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":574137078,"num_objects":140180,"num_object_clones":0,"num_object_copies":280360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2212,"num_objects_unfound":0,"num_objects_dirty":140180,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":150275,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6035,"num_bytes_recovered":24711168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1237123072,"data_stored":1236966795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160782,"ondisk_log_size":160782,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717664,"kb_used_data":411080,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93654176,"statfs":{"total":96636764160,"available":95901876224,"internally_reserved":0,"allocated":420945920,"data_stored":420670273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":863900,"kb_used_data":489356,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93507940,"statfs":{"total":96636764160,"available":95752130560,"internally_reserved":0,"allocated":501100544,"data_stored":500824892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,4,2,5,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621264,"kb_used_data":310528,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93750576,"statfs":{"total":96636764160,"available":96000589824,"internally_reserved":0,"allocated":317980672,"data_stored":317661205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":499396608,"data_stored":499360478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317509632,"data_stored":317429687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":420216832,"data_stored":420176630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:34.935 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:35.495 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:35.496 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:35.520 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-15T21:25:34.774037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":564986633,"num_objects":137948,"num_object_clones":0,"num_object_copies":275896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2112,"num_objects_unfound":0,"num_objects_dirty":137948,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":152511,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6136,"num_bytes_recovered":25120787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160919,"ondisk_log_size":160919,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2182692,"kb_used_data":1199156,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":280932828,"statfs":{"total":289910292480,"available":287675215872,"internally_reserved":0,"allocated":1227935744,"data_stored":1227064978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1007078403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,3,4,3,14,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21856256,"num_objects":-5336,"num_object_clones":0,"num_object_copies":-10672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-187,"num_objects_unfound":0,"num_objects_dirty":-5336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5336,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":765952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002889"},"pg_stats":[{"pgid":"2.f","version":"45'12284","reported_seq":12360,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.415800+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:32.415800+0000","last_peered":"2024-09-15T21:25:32.415800+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:32.415800+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36323328,"num_objects":8868,"num_object_clones":0,"num_object_copies":17736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12284,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4736},{"shards":"0,2","objects":2424}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12047","reported_seq":12153,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:33.738047+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:33.738047+0000","last_peered":"2024-09-15T21:25:33.738047+0000","last_clean":"2024-09-15T21:25:33.738047+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:33.738047+0000","last_undegraded":"2024-09-15T21:25:33.738047+0000","last_fullsized":"2024-09-15T21:25:33.738047+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35337158,"num_objects":8629,"num_object_clones":0,"num_object_copies":17258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8629,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10328,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12325","reported_seq":12452,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:33.650144+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:33.650144+0000","last_peered":"2024-09-15T21:25:33.650144+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:33.650144+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962904,"num_objects":8781,"num_object_clones":0,"num_object_copies":17562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10425,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2497},{"shards":"2","objects":4514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11899","reported_seq":11970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:33.762040+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:33.762040+0000","last_peered":"2024-09-15T21:25:33.762040+0000","last_clean":"2024-09-15T21:25:33.762040+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:33.762040+0000","last_undegraded":"2024-09-15T21:25:33.762040+0000","last_fullsized":"2024-09-15T21:25:33.762040+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11899,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11899","reported_seq":12032,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:32.203007+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:32.203007+0000","last_peered":"2024-09-15T21:25:32.203007+0000","last_clean":"2024-09-15T21:25:32.203007+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:32.203007+0000","last_undegraded":"2024-09-15T21:25:32.203007+0000","last_fullsized":"2024-09-15T21:25:32.203007+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35250659,"num_objects":8607,"num_object_clones":0,"num_object_copies":17214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8354,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12027","reported_seq":12164,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.374077+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:32.374077+0000","last_peered":"2024-09-15T21:25:32.374077+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:32.374077+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35258851,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8703,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4533},{"shards":"0,2","objects":2369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12075","reported_seq":12212,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.418911+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:32.418911+0000","last_peered":"2024-09-15T21:25:32.418911+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:32.418911+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35206086,"num_objects":8597,"num_object_clones":0,"num_object_copies":17194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4436},{"shards":"0,2","objects":2424}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12147","reported_seq":12276,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:32.350275+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:32.350275+0000","last_peered":"2024-09-15T21:25:32.350275+0000","last_clean":"2024-09-15T21:25:32.350275+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:32.350275+0000","last_undegraded":"2024-09-15T21:25:32.350275+0000","last_fullsized":"2024-09-15T21:25:32.350275+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8919,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12012","reported_seq":12139,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:33.764784+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:33.764784+0000","last_peered":"2024-09-15T21:25:33.764784+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:33.764784+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34820579,"num_objects":8502,"num_object_clones":0,"num_object_copies":17004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9944,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2333},{"shards":"2","objects":4415}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11856","reported_seq":11987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.338343+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:32.338343+0000","last_peered":"2024-09-15T21:25:32.338343+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:32.338343+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332096,"num_objects":8626,"num_object_clones":0,"num_object_copies":17252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8529,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4606},{"shards":"0,2","objects":2409}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12032","reported_seq":12151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:33.765453+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:33.765453+0000","last_peered":"2024-09-15T21:25:33.765453+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:33.765453+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35271139,"num_objects":8612,"num_object_clones":0,"num_object_copies":17224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8612,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9891,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2382},{"shards":"2","objects":4522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12071","reported_seq":20530,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:33.710521+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:33.710521+0000","last_peered":"2024-09-15T21:25:33.710521+0000","last_clean":"2024-09-15T21:25:33.710521+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:33.710521+0000","last_undegraded":"2024-09-15T21:25:33.710521+0000","last_fullsized":"2024-09-15T21:25:33.710521+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34770944,"num_objects":8489,"num_object_clones":0,"num_object_copies":16978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9764,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12004","reported_seq":12138,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.409448+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:32.409448+0000","last_peered":"2024-09-15T21:25:32.409448+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:32.409448+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35561472,"num_objects":8682,"num_object_clones":0,"num_object_copies":17364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4588},{"shards":"0,2","objects":2434}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11997","reported_seq":12130,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.449873+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:32.449873+0000","last_peered":"2024-09-15T21:25:32.449873+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:32.449873+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35196928,"num_objects":8593,"num_object_clones":0,"num_object_copies":17186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8906,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4454},{"shards":"0,2","objects":2441}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11898","reported_seq":15975,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:32.399429+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:32.399429+0000","last_peered":"2024-09-15T21:25:32.399429+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:32.399429+0000","last_undegraded":"2024-09-15T21:25:32.399429+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34775040,"num_objects":8490,"num_object_clones":0,"num_object_copies":16980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2112,"num_objects_unfound":0,"num_objects_dirty":8490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8382,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1970,"num_bytes_recovered":8069120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6378},{"shards":"0,2","objects":2112}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12143","reported_seq":12282,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.369395+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:32.369395+0000","last_peered":"2024-09-15T21:25:32.369395+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:32.369395+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35730374,"num_objects":8725,"num_object_clones":0,"num_object_copies":17450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8725,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8847,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4578},{"shards":"0,2","objects":2443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":564986614,"num_objects":137946,"num_object_clones":0,"num_object_copies":275892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2112,"num_objects_unfound":0,"num_objects_dirty":137946,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":152509,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6135,"num_bytes_recovered":25120768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1225084928,"data_stored":1224928651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713392,"kb_used_data":406816,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93658448,"statfs":{"total":96636764160,"available":95906250752,"internally_reserved":0,"allocated":416579584,"data_stored":416303937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,0,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":861272,"kb_used_data":482568,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93510568,"statfs":{"total":96636764160,"available":95754821632,"internally_reserved":0,"allocated":494149632,"data_stored":493873980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608028,"kb_used_data":309772,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93763812,"statfs":{"total":96636764160,"available":96014143488,"internally_reserved":0,"allocated":317206528,"data_stored":316887061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305393327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":492552192,"data_stored":492516062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316755968,"data_stored":316676023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":415776768,"data_stored":415736566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:35.521 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:36.074 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:36.080 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:36.105 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-09-15T21:25:34.774037+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":564986633,"num_objects":137948,"num_object_clones":0,"num_object_copies":275896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2112,"num_objects_unfound":0,"num_objects_dirty":137948,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":152511,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6136,"num_bytes_recovered":25120787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160919,"ondisk_log_size":160919,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2182692,"kb_used_data":1199156,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":280932828,"statfs":{"total":289910292480,"available":287675215872,"internally_reserved":0,"allocated":1227935744,"data_stored":1227064978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1007078403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,3,4,3,14,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21856256,"num_objects":-5336,"num_object_clones":0,"num_object_copies":-10672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-187,"num_objects_unfound":0,"num_objects_dirty":-5336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5336,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":765952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002889"},"pg_stats":[{"pgid":"2.f","version":"45'12284","reported_seq":12360,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.415800+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:32.415800+0000","last_peered":"2024-09-15T21:25:32.415800+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:32.415800+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36323328,"num_objects":8868,"num_object_clones":0,"num_object_copies":17736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12284,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4736},{"shards":"0,2","objects":2424}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12047","reported_seq":12153,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:33.738047+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:33.738047+0000","last_peered":"2024-09-15T21:25:33.738047+0000","last_clean":"2024-09-15T21:25:33.738047+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:33.738047+0000","last_undegraded":"2024-09-15T21:25:33.738047+0000","last_fullsized":"2024-09-15T21:25:33.738047+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35337158,"num_objects":8629,"num_object_clones":0,"num_object_copies":17258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8629,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10328,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12325","reported_seq":12452,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:33.650144+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:33.650144+0000","last_peered":"2024-09-15T21:25:33.650144+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:33.650144+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962904,"num_objects":8781,"num_object_clones":0,"num_object_copies":17562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10425,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2497},{"shards":"2","objects":4514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11899","reported_seq":11970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:33.762040+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:33.762040+0000","last_peered":"2024-09-15T21:25:33.762040+0000","last_clean":"2024-09-15T21:25:33.762040+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:33.762040+0000","last_undegraded":"2024-09-15T21:25:33.762040+0000","last_fullsized":"2024-09-15T21:25:33.762040+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11899,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11899","reported_seq":12032,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:32.203007+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:32.203007+0000","last_peered":"2024-09-15T21:25:32.203007+0000","last_clean":"2024-09-15T21:25:32.203007+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:32.203007+0000","last_undegraded":"2024-09-15T21:25:32.203007+0000","last_fullsized":"2024-09-15T21:25:32.203007+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35250659,"num_objects":8607,"num_object_clones":0,"num_object_copies":17214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8354,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12027","reported_seq":12164,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.374077+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:32.374077+0000","last_peered":"2024-09-15T21:25:32.374077+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:32.374077+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35258851,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8703,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4533},{"shards":"0,2","objects":2369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12075","reported_seq":12212,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.418911+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:32.418911+0000","last_peered":"2024-09-15T21:25:32.418911+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:32.418911+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35206086,"num_objects":8597,"num_object_clones":0,"num_object_copies":17194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4436},{"shards":"0,2","objects":2424}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12147","reported_seq":12276,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:32.350275+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:32.350275+0000","last_peered":"2024-09-15T21:25:32.350275+0000","last_clean":"2024-09-15T21:25:32.350275+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:32.350275+0000","last_undegraded":"2024-09-15T21:25:32.350275+0000","last_fullsized":"2024-09-15T21:25:32.350275+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8919,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12012","reported_seq":12139,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:33.764784+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:33.764784+0000","last_peered":"2024-09-15T21:25:33.764784+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:33.764784+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34820579,"num_objects":8502,"num_object_clones":0,"num_object_copies":17004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9944,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2333},{"shards":"2","objects":4415}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11856","reported_seq":11987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.338343+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:32.338343+0000","last_peered":"2024-09-15T21:25:32.338343+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:32.338343+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332096,"num_objects":8626,"num_object_clones":0,"num_object_copies":17252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8529,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4606},{"shards":"0,2","objects":2409}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12032","reported_seq":12151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:33.765453+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:33.765453+0000","last_peered":"2024-09-15T21:25:33.765453+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:33.765453+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35271139,"num_objects":8612,"num_object_clones":0,"num_object_copies":17224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8612,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9891,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2382},{"shards":"2","objects":4522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12071","reported_seq":20530,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:33.710521+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:33.710521+0000","last_peered":"2024-09-15T21:25:33.710521+0000","last_clean":"2024-09-15T21:25:33.710521+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:33.710521+0000","last_undegraded":"2024-09-15T21:25:33.710521+0000","last_fullsized":"2024-09-15T21:25:33.710521+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34770944,"num_objects":8489,"num_object_clones":0,"num_object_copies":16978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9764,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12004","reported_seq":12138,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.409448+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:32.409448+0000","last_peered":"2024-09-15T21:25:32.409448+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:32.409448+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35561472,"num_objects":8682,"num_object_clones":0,"num_object_copies":17364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4588},{"shards":"0,2","objects":2434}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11997","reported_seq":12130,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.449873+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:32.449873+0000","last_peered":"2024-09-15T21:25:32.449873+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:32.449873+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35196928,"num_objects":8593,"num_object_clones":0,"num_object_copies":17186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8906,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4454},{"shards":"0,2","objects":2441}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11898","reported_seq":15975,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:32.399429+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:32.399429+0000","last_peered":"2024-09-15T21:25:32.399429+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:32.399429+0000","last_undegraded":"2024-09-15T21:25:32.399429+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34775040,"num_objects":8490,"num_object_clones":0,"num_object_copies":16980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2112,"num_objects_unfound":0,"num_objects_dirty":8490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8382,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1970,"num_bytes_recovered":8069120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6378},{"shards":"0,2","objects":2112}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12143","reported_seq":12282,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.369395+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:32.369395+0000","last_peered":"2024-09-15T21:25:32.369395+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:32.369395+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35730374,"num_objects":8725,"num_object_clones":0,"num_object_copies":17450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8725,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8847,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4578},{"shards":"0,2","objects":2443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":564986614,"num_objects":137946,"num_object_clones":0,"num_object_copies":275892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2112,"num_objects_unfound":0,"num_objects_dirty":137946,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":152509,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6135,"num_bytes_recovered":25120768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1225084928,"data_stored":1224928651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713392,"kb_used_data":406816,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93658448,"statfs":{"total":96636764160,"available":95906250752,"internally_reserved":0,"allocated":416579584,"data_stored":416303937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,0,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":861272,"kb_used_data":482568,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93510568,"statfs":{"total":96636764160,"available":95754821632,"internally_reserved":0,"allocated":494149632,"data_stored":493873980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608028,"kb_used_data":309772,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93763812,"statfs":{"total":96636764160,"available":96014143488,"internally_reserved":0,"allocated":317206528,"data_stored":316887061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305393327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":492552192,"data_stored":492516062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316755968,"data_stored":316676023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":415776768,"data_stored":415736566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:36.106 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:36.107 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12284", 'reported_seq': 12360, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:32.415800+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:25:32.415800+0000', 'last_peered': '2024-09-15T21:25:32.415800+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:25:32.415800+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36323328, 'num_objects': 8868, 'num_object_clones': 0, 'num_object_copies': 17736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12284, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4736}, {'shards': '0,2', 'objects': 2424}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:36.107 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:25:36.107 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'12325", 'reported_seq': 12452, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:33.650144+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:25:33.650144+0000', 'last_peered': '2024-09-15T21:25:33.650144+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:25:33.650144+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35962904, 'num_objects': 8781, 'num_object_clones': 0, 'num_object_copies': 17562, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8781, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10425, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2497}, {'shards': '2', 'objects': 4514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:36.107 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:36.107 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12027", 'reported_seq': 12164, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:32.374077+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:25:32.374077+0000', 'last_peered': '2024-09-15T21:25:32.374077+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:25:32.374077+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35258851, 'num_objects': 8609, 'num_object_clones': 0, 'num_object_copies': 17218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8609, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8703, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4533}, {'shards': '0,2', 'objects': 2369}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:36.107 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:36.107 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12075", 'reported_seq': 12212, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:32.418911+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:25:32.418911+0000', 'last_peered': '2024-09-15T21:25:32.418911+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:25:32.418911+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35206086, 'num_objects': 8597, 'num_object_clones': 0, 'num_object_copies': 17194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8597, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8386, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4436}, {'shards': '0,2', 'objects': 2424}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:36.107 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:25:36.107 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12012", 'reported_seq': 12139, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:33.764784+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:25:33.764784+0000', 'last_peered': '2024-09-15T21:25:33.764784+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:25:33.764784+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34820579, 'num_objects': 8502, 'num_object_clones': 0, 'num_object_copies': 17004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8502, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9944, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2333}, {'shards': '2', 'objects': 4415}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:36.107 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:36.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11856", 'reported_seq': 11987, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:32.338343+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:25:32.338343+0000', 'last_peered': '2024-09-15T21:25:32.338343+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:25:32.338343+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35332096, 'num_objects': 8626, 'num_object_clones': 0, 'num_object_copies': 17252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8529, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4606}, {'shards': '0,2', 'objects': 2409}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:36.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:36.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12032", 'reported_seq': 12151, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:33.765453+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:25:33.765453+0000', 'last_peered': '2024-09-15T21:25:33.765453+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:25:33.765453+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35271139, 'num_objects': 8612, 'num_object_clones': 0, 'num_object_copies': 17224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8612, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9891, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2382}, {'shards': '2', 'objects': 4522}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:36.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:36.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12004", 'reported_seq': 12138, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:32.409448+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:25:32.409448+0000', 'last_peered': '2024-09-15T21:25:32.409448+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:25:32.409448+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35561472, 'num_objects': 8682, 'num_object_clones': 0, 'num_object_copies': 17364, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8682, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8948, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4588}, {'shards': '0,2', 'objects': 2434}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:36.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:25:36.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'11997", 'reported_seq': 12130, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:32.449873+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:25:32.449873+0000', 'last_peered': '2024-09-15T21:25:32.449873+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:25:32.449873+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35196928, 'num_objects': 8593, 'num_object_clones': 0, 'num_object_copies': 17186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8593, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8906, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4454}, {'shards': '0,2', 'objects': 2441}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:36.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:36.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'11898", 'reported_seq': 15975, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:25:32.399429+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:25:32.399429+0000', 'last_peered': '2024-09-15T21:25:32.399429+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:25:32.399429+0000', 'last_undegraded': '2024-09-15T21:25:32.399429+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34775040, 'num_objects': 8490, 'num_object_clones': 0, 'num_object_copies': 16980, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2112, 'num_objects_unfound': 0, 'num_objects_dirty': 8490, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8382, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1970, 'num_bytes_recovered': 8069120, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6378}, {'shards': '0,2', 'objects': 2112}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:36.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:25:36.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'12143", 'reported_seq': 12282, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:32.369395+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:25:32.369395+0000', 'last_peered': '2024-09-15T21:25:32.369395+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:25:32.369395+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35730374, 'num_objects': 8725, 'num_object_clones': 0, 'num_object_copies': 17450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8725, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8847, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4578}, {'shards': '0,2', 'objects': 2443}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:36.109 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:36.827 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/275896 objects degraded (14.942%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":388,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":137948,"data_bytes":564986633,"bytes_used":2235076608,"bytes_avail":287675215872,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":275896,"degraded_ratio":0.14942224606373417,"misplaced_objects":2112,"misplaced_total":275896,"misplaced_ratio":0.0076550584278133778,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":63813,"recovering_keys_per_sec":0,"num_objects_recovered":187,"num_bytes_recovered":765952,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":444},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:25:36.849 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:36.849 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:37.414 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:37.440 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-09-15T21:25:36.774649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":564986633,"num_objects":137948,"num_object_clones":0,"num_object_copies":275896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2112,"num_objects_unfound":0,"num_objects_dirty":137948,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":152511,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6136,"num_bytes_recovered":25120787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160919,"ondisk_log_size":160919,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2182692,"kb_used_data":1199156,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":280932828,"statfs":{"total":289910292480,"available":287675215872,"internally_reserved":0,"allocated":1227935744,"data_stored":1227064978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1007078403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,3,4,3,14,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18345984,"num_objects":-4479,"num_object_clones":0,"num_object_copies":-8958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-187,"num_objects_unfound":0,"num_objects_dirty":-4479,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4479,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":765952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003096"},"pg_stats":[{"pgid":"2.f","version":"45'12284","reported_seq":12360,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.415800+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:32.415800+0000","last_peered":"2024-09-15T21:25:32.415800+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:32.415800+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36323328,"num_objects":8868,"num_object_clones":0,"num_object_copies":17736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12284,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4736},{"shards":"0,2","objects":2424}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12047","reported_seq":12153,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:33.738047+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:33.738047+0000","last_peered":"2024-09-15T21:25:33.738047+0000","last_clean":"2024-09-15T21:25:33.738047+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:33.738047+0000","last_undegraded":"2024-09-15T21:25:33.738047+0000","last_fullsized":"2024-09-15T21:25:33.738047+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35337158,"num_objects":8629,"num_object_clones":0,"num_object_copies":17258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8629,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10328,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12325","reported_seq":12452,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:33.650144+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:33.650144+0000","last_peered":"2024-09-15T21:25:33.650144+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:33.650144+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35962904,"num_objects":8781,"num_object_clones":0,"num_object_copies":17562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8781,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10425,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2497},{"shards":"2","objects":4514}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11899","reported_seq":11970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:33.762040+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:33.762040+0000","last_peered":"2024-09-15T21:25:33.762040+0000","last_clean":"2024-09-15T21:25:33.762040+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:33.762040+0000","last_undegraded":"2024-09-15T21:25:33.762040+0000","last_fullsized":"2024-09-15T21:25:33.762040+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34467840,"num_objects":8415,"num_object_clones":0,"num_object_copies":16830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11899,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11899","reported_seq":12032,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:32.203007+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:32.203007+0000","last_peered":"2024-09-15T21:25:32.203007+0000","last_clean":"2024-09-15T21:25:32.203007+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:32.203007+0000","last_undegraded":"2024-09-15T21:25:32.203007+0000","last_fullsized":"2024-09-15T21:25:32.203007+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35250659,"num_objects":8607,"num_object_clones":0,"num_object_copies":17214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8354,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12027","reported_seq":12164,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.374077+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:32.374077+0000","last_peered":"2024-09-15T21:25:32.374077+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:32.374077+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35258851,"num_objects":8609,"num_object_clones":0,"num_object_copies":17218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8609,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8703,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4533},{"shards":"0,2","objects":2369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12075","reported_seq":12212,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.418911+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:32.418911+0000","last_peered":"2024-09-15T21:25:32.418911+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:32.418911+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35206086,"num_objects":8597,"num_object_clones":0,"num_object_copies":17194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8597,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8386,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4436},{"shards":"0,2","objects":2424}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12147","reported_seq":12276,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:32.350275+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:32.350275+0000","last_peered":"2024-09-15T21:25:32.350275+0000","last_clean":"2024-09-15T21:25:32.350275+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:32.350275+0000","last_undegraded":"2024-09-15T21:25:32.350275+0000","last_fullsized":"2024-09-15T21:25:32.350275+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8919,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12012","reported_seq":12139,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:33.764784+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:33.764784+0000","last_peered":"2024-09-15T21:25:33.764784+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:33.764784+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34820579,"num_objects":8502,"num_object_clones":0,"num_object_copies":17004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9944,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2333},{"shards":"2","objects":4415}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11856","reported_seq":11987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.338343+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:32.338343+0000","last_peered":"2024-09-15T21:25:32.338343+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:32.338343+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35332096,"num_objects":8626,"num_object_clones":0,"num_object_copies":17252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8529,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4606},{"shards":"0,2","objects":2409}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12032","reported_seq":12151,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:33.765453+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:33.765453+0000","last_peered":"2024-09-15T21:25:33.765453+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:33.765453+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35271139,"num_objects":8612,"num_object_clones":0,"num_object_copies":17224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8612,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9891,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2382},{"shards":"2","objects":4522}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12071","reported_seq":20530,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:33.710521+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:33.710521+0000","last_peered":"2024-09-15T21:25:33.710521+0000","last_clean":"2024-09-15T21:25:33.710521+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:33.710521+0000","last_undegraded":"2024-09-15T21:25:33.710521+0000","last_fullsized":"2024-09-15T21:25:33.710521+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34770944,"num_objects":8489,"num_object_clones":0,"num_object_copies":16978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9764,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12004","reported_seq":12138,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.409448+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:32.409448+0000","last_peered":"2024-09-15T21:25:32.409448+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:32.409448+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35561472,"num_objects":8682,"num_object_clones":0,"num_object_copies":17364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8682,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8948,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4588},{"shards":"0,2","objects":2434}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11997","reported_seq":12130,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.449873+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:32.449873+0000","last_peered":"2024-09-15T21:25:32.449873+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:32.449873+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35196928,"num_objects":8593,"num_object_clones":0,"num_object_copies":17186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8906,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4454},{"shards":"0,2","objects":2441}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11898","reported_seq":15975,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:32.399429+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:32.399429+0000","last_peered":"2024-09-15T21:25:32.399429+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:32.399429+0000","last_undegraded":"2024-09-15T21:25:32.399429+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34775040,"num_objects":8490,"num_object_clones":0,"num_object_copies":16980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2112,"num_objects_unfound":0,"num_objects_dirty":8490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8382,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1970,"num_bytes_recovered":8069120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6378},{"shards":"0,2","objects":2112}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12143","reported_seq":12282,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:32.369395+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:32.369395+0000","last_peered":"2024-09-15T21:25:32.369395+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:32.369395+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35730374,"num_objects":8725,"num_object_clones":0,"num_object_copies":17450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8725,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8847,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4578},{"shards":"0,2","objects":2443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":564986614,"num_objects":137946,"num_object_clones":0,"num_object_copies":275892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2112,"num_objects_unfound":0,"num_objects_dirty":137946,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":152509,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6135,"num_bytes_recovered":25120768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1225084928,"data_stored":1224928651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713392,"kb_used_data":406816,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93658448,"statfs":{"total":96636764160,"available":95906250752,"internally_reserved":0,"allocated":416579584,"data_stored":416303937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,0,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":861272,"kb_used_data":482568,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93510568,"statfs":{"total":96636764160,"available":95754821632,"internally_reserved":0,"allocated":494149632,"data_stored":493873980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608028,"kb_used_data":309772,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93763812,"statfs":{"total":96636764160,"available":96014143488,"internally_reserved":0,"allocated":317206528,"data_stored":316887061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305393327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":492552192,"data_stored":492516062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316755968,"data_stored":316676023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":415776768,"data_stored":415736566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:40.442 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:40.996 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:40.999 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:41.017 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-15T21:25:40.775881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":555799305,"num_objects":135705,"num_object_clones":0,"num_object_copies":271410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2019,"num_objects_unfound":0,"num_objects_dirty":135705,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":154754,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6229,"num_bytes_recovered":25501715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160962,"ondisk_log_size":160962,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2183288,"kb_used_data":1187272,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":280932232,"statfs":{"total":289910292480,"available":287674605568,"internally_reserved":0,"allocated":1215766528,"data_stored":1214895762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1019857923},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,7,10,6,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21868544,"num_objects":-5339,"num_object_clones":0,"num_object_copies":-10678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-193,"num_objects_unfound":0,"num_objects_dirty":-5339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5339,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":790528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003383"},"pg_stats":[{"pgid":"2.f","version":"45'12429","reported_seq":12505,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.452568+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:37.452568+0000","last_peered":"2024-09-15T21:25:37.452568+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:37.452568+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35729408,"num_objects":8723,"num_object_clones":0,"num_object_copies":17446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12429,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4591},{"shards":"0,2","objects":2279}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12171","reported_seq":12277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:38.817378+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:38.817378+0000","last_peered":"2024-09-15T21:25:38.817378+0000","last_clean":"2024-09-15T21:25:38.817378+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:38.817378+0000","last_undegraded":"2024-09-15T21:25:38.817378+0000","last_fullsized":"2024-09-15T21:25:38.817378+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34829254,"num_objects":8505,"num_object_clones":0,"num_object_copies":17010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8505,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10452,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12474","reported_seq":12601,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:38.795332+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:38.795332+0000","last_peered":"2024-09-15T21:25:38.795332+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:38.795332+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35352600,"num_objects":8632,"num_object_clones":0,"num_object_copies":17264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8632,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10574,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2348},{"shards":"2","objects":4365}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12021","reported_seq":12091,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:38.783162+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:38.783162+0000","last_peered":"2024-09-15T21:25:38.783162+0000","last_clean":"2024-09-15T21:25:38.783162+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:38.783162+0000","last_undegraded":"2024-09-15T21:25:38.783162+0000","last_fullsized":"2024-09-15T21:25:38.783162+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33968128,"num_objects":8293,"num_object_clones":0,"num_object_copies":16586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12021,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12054","reported_seq":12187,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:37.415706+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:37.415706+0000","last_peered":"2024-09-15T21:25:37.415706+0000","last_clean":"2024-09-15T21:25:37.415706+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:37.415706+0000","last_undegraded":"2024-09-15T21:25:37.415706+0000","last_fullsized":"2024-09-15T21:25:37.415706+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34615779,"num_objects":8452,"num_object_clones":0,"num_object_copies":16904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8509,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12171","reported_seq":12308,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.455337+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:37.455337+0000","last_peered":"2024-09-15T21:25:37.455337+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:37.455337+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34669027,"num_objects":8465,"num_object_clones":0,"num_object_copies":16930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8465,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8847,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4389},{"shards":"0,2","objects":2225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12215","reported_seq":12352,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.447429+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:37.447429+0000","last_peered":"2024-09-15T21:25:37.447429+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:37.447429+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34632646,"num_objects":8457,"num_object_clones":0,"num_object_copies":16914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8526,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4296},{"shards":"0,2","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12300","reported_seq":12429,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:37.449769+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:37.449769+0000","last_peered":"2024-09-15T21:25:37.449769+0000","last_clean":"2024-09-15T21:25:37.449769+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:37.449769+0000","last_undegraded":"2024-09-15T21:25:37.449769+0000","last_fullsized":"2024-09-15T21:25:37.449769+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35094528,"num_objects":8568,"num_object_clones":0,"num_object_copies":17136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9072,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12153","reported_seq":12279,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:38.798098+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:38.798098+0000","last_peered":"2024-09-15T21:25:38.798098+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:38.798098+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34243043,"num_objects":8361,"num_object_clones":0,"num_object_copies":16722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10085,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2192},{"shards":"2","objects":4274}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11985","reported_seq":12116,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.454091+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:37.454091+0000","last_peered":"2024-09-15T21:25:37.454091+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:37.454091+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34803712,"num_objects":8497,"num_object_clones":0,"num_object_copies":16994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8658,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4477},{"shards":"0,2","objects":2280}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12183","reported_seq":12302,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:38.754216+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:38.754216+0000","last_peered":"2024-09-15T21:25:38.754216+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:38.754216+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34652643,"num_objects":8461,"num_object_clones":0,"num_object_copies":16922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10042,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2231},{"shards":"2","objects":4371}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12196","reported_seq":20655,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:38.735474+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:38.735474+0000","last_peered":"2024-09-15T21:25:38.735474+0000","last_clean":"2024-09-15T21:25:38.735474+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:38.735474+0000","last_undegraded":"2024-09-15T21:25:38.735474+0000","last_fullsized":"2024-09-15T21:25:38.735474+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34258944,"num_objects":8364,"num_object_clones":0,"num_object_copies":16728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12154","reported_seq":12288,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.408063+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:37.408063+0000","last_peered":"2024-09-15T21:25:37.408063+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:37.408063+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34947072,"num_objects":8532,"num_object_clones":0,"num_object_copies":17064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9098,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4438},{"shards":"0,2","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12143","reported_seq":12274,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.404802+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:37.404802+0000","last_peered":"2024-09-15T21:25:37.404802+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:37.404802+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34598912,"num_objects":8447,"num_object_clones":0,"num_object_copies":16894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9052,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4308},{"shards":"0,2","objects":2295}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12031","reported_seq":16294,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:37.406840+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:37.406840+0000","last_peered":"2024-09-15T21:25:37.406840+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:37.406840+0000","last_undegraded":"2024-09-15T21:25:37.406840+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34230272,"num_objects":8357,"num_object_clones":0,"num_object_copies":16714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2019,"num_objects_unfound":0,"num_objects_dirty":8357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8515,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2063,"num_bytes_recovered":8450048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6338},{"shards":"0,2","objects":2019}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12279","reported_seq":12418,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.469998+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:37.469998+0000","last_peered":"2024-09-15T21:25:37.469998+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:37.469998+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35173318,"num_objects":8589,"num_object_clones":0,"num_object_copies":17178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8589,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8983,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":2307}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":555799286,"num_objects":135703,"num_object_clones":0,"num_object_copies":271406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2019,"num_objects_unfound":0,"num_objects_dirty":135703,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":154752,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6228,"num_bytes_recovered":25501696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1212870656,"data_stored":1212714379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160959,"ondisk_log_size":160959,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713208,"kb_used_data":402472,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93658632,"statfs":{"total":96636764160,"available":95906439168,"internally_reserved":0,"allocated":412131328,"data_stored":411855681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":858540,"kb_used_data":475676,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93513300,"statfs":{"total":96636764160,"available":95757619200,"internally_reserved":0,"allocated":487092224,"data_stored":486816572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,5,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611540,"kb_used_data":309124,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93760300,"statfs":{"total":96636764160,"available":96010547200,"internally_reserved":0,"allocated":316542976,"data_stored":316223509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309653167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":485531648,"data_stored":485495518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316080128,"data_stored":316000183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411258880,"data_stored":411218678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:41.019 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:41.590 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:41.590 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:41.611 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-09-15T21:25:40.775881+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":555799305,"num_objects":135705,"num_object_clones":0,"num_object_copies":271410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2019,"num_objects_unfound":0,"num_objects_dirty":135705,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":154754,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6229,"num_bytes_recovered":25501715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160962,"ondisk_log_size":160962,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2183288,"kb_used_data":1187272,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":280932232,"statfs":{"total":289910292480,"available":287674605568,"internally_reserved":0,"allocated":1215766528,"data_stored":1214895762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1019857923},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,7,10,6,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21868544,"num_objects":-5339,"num_object_clones":0,"num_object_copies":-10678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-193,"num_objects_unfound":0,"num_objects_dirty":-5339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5339,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":790528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003383"},"pg_stats":[{"pgid":"2.f","version":"45'12429","reported_seq":12505,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.452568+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:37.452568+0000","last_peered":"2024-09-15T21:25:37.452568+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:37.452568+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35729408,"num_objects":8723,"num_object_clones":0,"num_object_copies":17446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12429,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4591},{"shards":"0,2","objects":2279}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12171","reported_seq":12277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:38.817378+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:38.817378+0000","last_peered":"2024-09-15T21:25:38.817378+0000","last_clean":"2024-09-15T21:25:38.817378+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:38.817378+0000","last_undegraded":"2024-09-15T21:25:38.817378+0000","last_fullsized":"2024-09-15T21:25:38.817378+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34829254,"num_objects":8505,"num_object_clones":0,"num_object_copies":17010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8505,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10452,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12474","reported_seq":12601,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:38.795332+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:38.795332+0000","last_peered":"2024-09-15T21:25:38.795332+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:38.795332+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35352600,"num_objects":8632,"num_object_clones":0,"num_object_copies":17264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8632,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10574,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2348},{"shards":"2","objects":4365}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12021","reported_seq":12091,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:38.783162+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:38.783162+0000","last_peered":"2024-09-15T21:25:38.783162+0000","last_clean":"2024-09-15T21:25:38.783162+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:38.783162+0000","last_undegraded":"2024-09-15T21:25:38.783162+0000","last_fullsized":"2024-09-15T21:25:38.783162+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33968128,"num_objects":8293,"num_object_clones":0,"num_object_copies":16586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12021,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12054","reported_seq":12187,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:37.415706+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:37.415706+0000","last_peered":"2024-09-15T21:25:37.415706+0000","last_clean":"2024-09-15T21:25:37.415706+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:37.415706+0000","last_undegraded":"2024-09-15T21:25:37.415706+0000","last_fullsized":"2024-09-15T21:25:37.415706+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34615779,"num_objects":8452,"num_object_clones":0,"num_object_copies":16904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8509,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12171","reported_seq":12308,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.455337+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:37.455337+0000","last_peered":"2024-09-15T21:25:37.455337+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:37.455337+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34669027,"num_objects":8465,"num_object_clones":0,"num_object_copies":16930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8465,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8847,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4389},{"shards":"0,2","objects":2225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12215","reported_seq":12352,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.447429+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:37.447429+0000","last_peered":"2024-09-15T21:25:37.447429+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:37.447429+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34632646,"num_objects":8457,"num_object_clones":0,"num_object_copies":16914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8526,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4296},{"shards":"0,2","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12300","reported_seq":12429,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:37.449769+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:37.449769+0000","last_peered":"2024-09-15T21:25:37.449769+0000","last_clean":"2024-09-15T21:25:37.449769+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:37.449769+0000","last_undegraded":"2024-09-15T21:25:37.449769+0000","last_fullsized":"2024-09-15T21:25:37.449769+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35094528,"num_objects":8568,"num_object_clones":0,"num_object_copies":17136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9072,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12153","reported_seq":12279,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:38.798098+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:38.798098+0000","last_peered":"2024-09-15T21:25:38.798098+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:38.798098+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34243043,"num_objects":8361,"num_object_clones":0,"num_object_copies":16722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10085,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2192},{"shards":"2","objects":4274}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11985","reported_seq":12116,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.454091+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:37.454091+0000","last_peered":"2024-09-15T21:25:37.454091+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:37.454091+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34803712,"num_objects":8497,"num_object_clones":0,"num_object_copies":16994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8658,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4477},{"shards":"0,2","objects":2280}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12183","reported_seq":12302,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:38.754216+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:38.754216+0000","last_peered":"2024-09-15T21:25:38.754216+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:38.754216+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34652643,"num_objects":8461,"num_object_clones":0,"num_object_copies":16922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10042,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2231},{"shards":"2","objects":4371}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12196","reported_seq":20655,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:38.735474+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:38.735474+0000","last_peered":"2024-09-15T21:25:38.735474+0000","last_clean":"2024-09-15T21:25:38.735474+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:38.735474+0000","last_undegraded":"2024-09-15T21:25:38.735474+0000","last_fullsized":"2024-09-15T21:25:38.735474+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34258944,"num_objects":8364,"num_object_clones":0,"num_object_copies":16728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12154","reported_seq":12288,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.408063+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:37.408063+0000","last_peered":"2024-09-15T21:25:37.408063+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:37.408063+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34947072,"num_objects":8532,"num_object_clones":0,"num_object_copies":17064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9098,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4438},{"shards":"0,2","objects":2284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12143","reported_seq":12274,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.404802+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:37.404802+0000","last_peered":"2024-09-15T21:25:37.404802+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:37.404802+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34598912,"num_objects":8447,"num_object_clones":0,"num_object_copies":16894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9052,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4308},{"shards":"0,2","objects":2295}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12031","reported_seq":16294,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:37.406840+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:37.406840+0000","last_peered":"2024-09-15T21:25:37.406840+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:37.406840+0000","last_undegraded":"2024-09-15T21:25:37.406840+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34230272,"num_objects":8357,"num_object_clones":0,"num_object_copies":16714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2019,"num_objects_unfound":0,"num_objects_dirty":8357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8515,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2063,"num_bytes_recovered":8450048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6338},{"shards":"0,2","objects":2019}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12279","reported_seq":12418,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:37.469998+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:37.469998+0000","last_peered":"2024-09-15T21:25:37.469998+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:37.469998+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35173318,"num_objects":8589,"num_object_clones":0,"num_object_copies":17178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8589,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8983,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":2307}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":555799286,"num_objects":135703,"num_object_clones":0,"num_object_copies":271406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":2019,"num_objects_unfound":0,"num_objects_dirty":135703,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":154752,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6228,"num_bytes_recovered":25501696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1212870656,"data_stored":1212714379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160959,"ondisk_log_size":160959,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713208,"kb_used_data":402472,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93658632,"statfs":{"total":96636764160,"available":95906439168,"internally_reserved":0,"allocated":412131328,"data_stored":411855681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":858540,"kb_used_data":475676,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93513300,"statfs":{"total":96636764160,"available":95757619200,"internally_reserved":0,"allocated":487092224,"data_stored":486816572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,5,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611540,"kb_used_data":309124,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93760300,"statfs":{"total":96636764160,"available":96010547200,"internally_reserved":0,"allocated":316542976,"data_stored":316223509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309653167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":485531648,"data_stored":485495518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316080128,"data_stored":316000183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411258880,"data_stored":411218678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:41.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:41.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12429", 'reported_seq': 12505, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:37.452568+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:25:37.452568+0000', 'last_peered': '2024-09-15T21:25:37.452568+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:25:37.452568+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35729408, 'num_objects': 8723, 'num_object_clones': 0, 'num_object_copies': 17446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8723, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12429, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4591}, {'shards': '0,2', 'objects': 2279}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:41.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:25:41.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'12474", 'reported_seq': 12601, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:38.795332+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:25:38.795332+0000', 'last_peered': '2024-09-15T21:25:38.795332+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:25:38.795332+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35352600, 'num_objects': 8632, 'num_object_clones': 0, 'num_object_copies': 17264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8632, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10574, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2348}, {'shards': '2', 'objects': 4365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:41.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:41.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12171", 'reported_seq': 12308, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:37.455337+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:25:37.455337+0000', 'last_peered': '2024-09-15T21:25:37.455337+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:25:37.455337+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34669027, 'num_objects': 8465, 'num_object_clones': 0, 'num_object_copies': 16930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8465, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8847, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4389}, {'shards': '0,2', 'objects': 2225}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:41.613 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:41.613 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12215", 'reported_seq': 12352, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:37.447429+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:25:37.447429+0000', 'last_peered': '2024-09-15T21:25:37.447429+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:25:37.447429+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34632646, 'num_objects': 8457, 'num_object_clones': 0, 'num_object_copies': 16914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8457, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8526, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4296}, {'shards': '0,2', 'objects': 2284}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:41.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:25:41.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12153", 'reported_seq': 12279, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:38.798098+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:25:38.798098+0000', 'last_peered': '2024-09-15T21:25:38.798098+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:25:38.798098+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34243043, 'num_objects': 8361, 'num_object_clones': 0, 'num_object_copies': 16722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8361, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10085, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2192}, {'shards': '2', 'objects': 4274}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:41.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:41.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'11985", 'reported_seq': 12116, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:37.454091+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:25:37.454091+0000', 'last_peered': '2024-09-15T21:25:37.454091+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:25:37.454091+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34803712, 'num_objects': 8497, 'num_object_clones': 0, 'num_object_copies': 16994, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8497, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8658, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4477}, {'shards': '0,2', 'objects': 2280}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:41.614 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:41.614 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12183", 'reported_seq': 12302, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:38.754216+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:25:38.754216+0000', 'last_peered': '2024-09-15T21:25:38.754216+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:25:38.754216+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34652643, 'num_objects': 8461, 'num_object_clones': 0, 'num_object_copies': 16922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8461, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10042, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2231}, {'shards': '2', 'objects': 4371}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:41.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:41.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12154", 'reported_seq': 12288, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:37.408063+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:25:37.408063+0000', 'last_peered': '2024-09-15T21:25:37.408063+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:25:37.408063+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34947072, 'num_objects': 8532, 'num_object_clones': 0, 'num_object_copies': 17064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8532, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9098, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4438}, {'shards': '0,2', 'objects': 2284}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:41.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:25:41.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12143", 'reported_seq': 12274, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:37.404802+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:25:37.404802+0000', 'last_peered': '2024-09-15T21:25:37.404802+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:25:37.404802+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34598912, 'num_objects': 8447, 'num_object_clones': 0, 'num_object_copies': 16894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8447, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9052, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4308}, {'shards': '0,2', 'objects': 2295}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:41.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:41.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12031", 'reported_seq': 16294, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:25:37.406840+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:25:37.406840+0000', 'last_peered': '2024-09-15T21:25:37.406840+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:25:37.406840+0000', 'last_undegraded': '2024-09-15T21:25:37.406840+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34230272, 'num_objects': 8357, 'num_object_clones': 0, 'num_object_copies': 16714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2019, 'num_objects_unfound': 0, 'num_objects_dirty': 8357, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8515, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2063, 'num_bytes_recovered': 8450048, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6338}, {'shards': '0,2', 'objects': 2019}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:41.615 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:25:41.615 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'12279", 'reported_seq': 12418, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:37.469998+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:25:37.469998+0000', 'last_peered': '2024-09-15T21:25:37.469998+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:25:37.469998+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35173318, 'num_objects': 8589, 'num_object_clones': 0, 'num_object_copies': 17178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8589, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8983, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4442}, {'shards': '0,2', 'objects': 2307}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:41.616 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:42.319 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:42.319 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/271410 objects degraded (15.189%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":394,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":135705,"data_bytes":555799305,"bytes_used":2235686912,"bytes_avail":287674605568,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":271410,"degraded_ratio":0.15189197155594855,"misplaced_objects":2019,"misplaced_total":271410,"misplaced_ratio":0.0074389300320548248,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":65858,"recovering_keys_per_sec":0,"num_objects_recovered":193,"num_bytes_recovered":790528,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":444},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:25:42.348 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:42.348 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:42.916 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:42.917 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:42.945 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-15T21:25:42.776374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":549946121,"num_objects":134276,"num_object_clones":0,"num_object_copies":268552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1919,"num_objects_unfound":0,"num_objects_dirty":134276,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":156183,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6329,"num_bytes_recovered":25911315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160791,"ondisk_log_size":160791,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2168400,"kb_used_data":1180704,"kb_used_omap":12,"kb_used_meta":987635,"kb_avail":280947120,"statfs":{"total":289910292480,"available":287689850880,"internally_reserved":0,"allocated":1209040896,"data_stored":1208170130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1011338243},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,6,10,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24190976,"num_objects":-5906,"num_object_clones":0,"num_object_copies":-11812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-293,"num_objects_unfound":0,"num_objects_dirty":-5906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5906,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":1200128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003271"},"pg_stats":[{"pgid":"2.f","version":"45'12563","reported_seq":12639,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.484042+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:42.484042+0000","last_peered":"2024-09-15T21:25:42.484042+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:42.484042+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35180544,"num_objects":8589,"num_object_clones":0,"num_object_copies":17178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12563,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4457},{"shards":"0,2","objects":2145}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12171","reported_seq":12277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:38.817378+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:38.817378+0000","last_peered":"2024-09-15T21:25:38.817378+0000","last_clean":"2024-09-15T21:25:38.817378+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:38.817378+0000","last_undegraded":"2024-09-15T21:25:38.817378+0000","last_fullsized":"2024-09-15T21:25:38.817378+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34829254,"num_objects":8505,"num_object_clones":0,"num_object_copies":17010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8505,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10452,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12474","reported_seq":12601,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:38.795332+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:38.795332+0000","last_peered":"2024-09-15T21:25:38.795332+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:38.795332+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35352600,"num_objects":8632,"num_object_clones":0,"num_object_copies":17264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8632,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10574,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2348},{"shards":"2","objects":4365}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12021","reported_seq":12091,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:38.783162+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:38.783162+0000","last_peered":"2024-09-15T21:25:38.783162+0000","last_clean":"2024-09-15T21:25:38.783162+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:38.783162+0000","last_undegraded":"2024-09-15T21:25:38.783162+0000","last_fullsized":"2024-09-15T21:25:38.783162+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33968128,"num_objects":8293,"num_object_clones":0,"num_object_copies":16586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12021,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12178","reported_seq":12311,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:42.356200+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:42.356200+0000","last_peered":"2024-09-15T21:25:42.356200+0000","last_clean":"2024-09-15T21:25:42.356200+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:42.356200+0000","last_undegraded":"2024-09-15T21:25:42.356200+0000","last_fullsized":"2024-09-15T21:25:42.356200+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34107875,"num_objects":8328,"num_object_clones":0,"num_object_copies":16656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8633,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12313","reported_seq":12450,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.421828+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:42.421828+0000","last_peered":"2024-09-15T21:25:42.421828+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:42.421828+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34087395,"num_objects":8323,"num_object_clones":0,"num_object_copies":16646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8323,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8989,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4247},{"shards":"0,2","objects":2083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12354","reported_seq":12489,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.488022+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:42.488022+0000","last_peered":"2024-09-15T21:25:42.488022+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:42.488022+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34063302,"num_objects":8318,"num_object_clones":0,"num_object_copies":16636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8665,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4157},{"shards":"0,2","objects":2145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12463","reported_seq":12592,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:42.396913+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:42.396913+0000","last_peered":"2024-09-15T21:25:42.396913+0000","last_clean":"2024-09-15T21:25:42.396913+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:42.396913+0000","last_undegraded":"2024-09-15T21:25:42.396913+0000","last_fullsized":"2024-09-15T21:25:42.396913+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34426880,"num_objects":8405,"num_object_clones":0,"num_object_copies":16810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9235,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12153","reported_seq":12279,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:38.798098+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:38.798098+0000","last_peered":"2024-09-15T21:25:38.798098+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:38.798098+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34243043,"num_objects":8361,"num_object_clones":0,"num_object_copies":16722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8361,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10085,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2192},{"shards":"2","objects":4274}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12127","reported_seq":12258,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.445841+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:42.445841+0000","last_peered":"2024-09-15T21:25:42.445841+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:42.445841+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34222080,"num_objects":8355,"num_object_clones":0,"num_object_copies":16710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8800,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4335},{"shards":"0,2","objects":2138}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12183","reported_seq":12302,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:38.754216+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:38.754216+0000","last_peered":"2024-09-15T21:25:38.754216+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:38.754216+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34652643,"num_objects":8461,"num_object_clones":0,"num_object_copies":16922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8461,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10042,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2231},{"shards":"2","objects":4371}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12196","reported_seq":20655,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:38.735474+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:38.735474+0000","last_peered":"2024-09-15T21:25:38.735474+0000","last_clean":"2024-09-15T21:25:38.735474+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:38.735474+0000","last_undegraded":"2024-09-15T21:25:38.735474+0000","last_fullsized":"2024-09-15T21:25:38.735474+0000","mapping_epoch":44,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34258944,"num_objects":8364,"num_object_clones":0,"num_object_copies":16728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9889,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12302","reported_seq":12436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.433338+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:42.433338+0000","last_peered":"2024-09-15T21:25:42.433338+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:42.433338+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34340864,"num_objects":8384,"num_object_clones":0,"num_object_copies":16768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9246,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4290},{"shards":"0,2","objects":2136}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12307","reported_seq":12436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.454916+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:42.454916+0000","last_peered":"2024-09-15T21:25:42.454916+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:42.454916+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33927168,"num_objects":8283,"num_object_clones":0,"num_object_copies":16566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4144},{"shards":"0,2","objects":2131}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12157","reported_seq":16620,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:42.455045+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:42.455045+0000","last_peered":"2024-09-15T21:25:42.455045+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:42.455045+0000","last_undegraded":"2024-09-15T21:25:42.455045+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33714176,"num_objects":8231,"num_object_clones":0,"num_object_copies":16462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1919,"num_objects_unfound":0,"num_objects_dirty":8231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8641,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2163,"num_bytes_recovered":8859648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6312},{"shards":"0,2","objects":1919}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12426","reported_seq":12565,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.446569+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:42.446569+0000","last_peered":"2024-09-15T21:25:42.446569+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:42.446569+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34571206,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9130,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4295},{"shards":"0,2","objects":2160}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":549946102,"num_objects":134274,"num_object_clones":0,"num_object_copies":268548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1919,"num_objects_unfound":0,"num_objects_dirty":134274,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":156181,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6328,"num_bytes_recovered":25911296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1206034432,"data_stored":1205878155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160788,"ondisk_log_size":160788,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":713208,"kb_used_data":402472,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93658632,"statfs":{"total":96636764160,"available":95906439168,"internally_reserved":0,"allocated":412131328,"data_stored":411855681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843652,"kb_used_data":469108,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93528188,"statfs":{"total":96636764160,"available":95772864512,"internally_reserved":0,"allocated":480366592,"data_stored":480090940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611540,"kb_used_data":309124,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93760300,"statfs":{"total":96636764160,"available":96010547200,"internally_reserved":0,"allocated":316542976,"data_stored":316223509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309653167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":478695424,"data_stored":478659294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316080128,"data_stored":316000183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":411258880,"data_stored":411218678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:45.947 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:46.469 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:46.496 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-15T21:25:44.776936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":546472713,"num_objects":133428,"num_object_clones":0,"num_object_copies":266856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1919,"num_objects_unfound":0,"num_objects_dirty":133428,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":157031,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6329,"num_bytes_recovered":25911315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2159032,"kb_used_data":1175496,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":280956488,"statfs":{"total":289910292480,"available":287699443712,"internally_reserved":0,"allocated":1203707904,"data_stored":1202837138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1007078403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,5,12,10,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21819392,"num_objects":-5327,"num_object_clones":0,"num_object_copies":-10654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-193,"num_objects_unfound":0,"num_objects_dirty":-5327,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5327,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":790528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003387"},"pg_stats":[{"pgid":"2.f","version":"45'12563","reported_seq":12639,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.484042+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:42.484042+0000","last_peered":"2024-09-15T21:25:42.484042+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:42.484042+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35180544,"num_objects":8589,"num_object_clones":0,"num_object_copies":17178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12563,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4457},{"shards":"0,2","objects":2145}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12296","reported_seq":12402,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:43.778406+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:43.778406+0000","last_peered":"2024-09-15T21:25:43.778406+0000","last_clean":"2024-09-15T21:25:43.778406+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:43.778406+0000","last_undegraded":"2024-09-15T21:25:43.778406+0000","last_fullsized":"2024-09-15T21:25:43.778406+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34317254,"num_objects":8380,"num_object_clones":0,"num_object_copies":16760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8380,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10577,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12633","reported_seq":12759,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:43.832243+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:43.832243+0000","last_peered":"2024-09-15T21:25:43.832243+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:43.832243+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34701336,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10733,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2189},{"shards":"2","objects":4206}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12162","reported_seq":12233,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:43.849046+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:43.849046+0000","last_peered":"2024-09-15T21:25:43.849046+0000","last_clean":"2024-09-15T21:25:43.849046+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:43.849046+0000","last_undegraded":"2024-09-15T21:25:43.849046+0000","last_fullsized":"2024-09-15T21:25:43.849046+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33390592,"num_objects":8152,"num_object_clones":0,"num_object_copies":16304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12162,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12178","reported_seq":12311,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:42.356200+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:42.356200+0000","last_peered":"2024-09-15T21:25:42.356200+0000","last_clean":"2024-09-15T21:25:42.356200+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:42.356200+0000","last_undegraded":"2024-09-15T21:25:42.356200+0000","last_fullsized":"2024-09-15T21:25:42.356200+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34107875,"num_objects":8328,"num_object_clones":0,"num_object_copies":16656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8633,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12313","reported_seq":12450,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.421828+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:42.421828+0000","last_peered":"2024-09-15T21:25:42.421828+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:42.421828+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34087395,"num_objects":8323,"num_object_clones":0,"num_object_copies":16646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8323,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8989,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4247},{"shards":"0,2","objects":2083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12354","reported_seq":12489,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.488022+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:42.488022+0000","last_peered":"2024-09-15T21:25:42.488022+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:42.488022+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34063302,"num_objects":8318,"num_object_clones":0,"num_object_copies":16636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8665,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4157},{"shards":"0,2","objects":2145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12463","reported_seq":12592,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:42.396913+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:42.396913+0000","last_peered":"2024-09-15T21:25:42.396913+0000","last_clean":"2024-09-15T21:25:42.396913+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:42.396913+0000","last_undegraded":"2024-09-15T21:25:42.396913+0000","last_fullsized":"2024-09-15T21:25:42.396913+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34426880,"num_objects":8405,"num_object_clones":0,"num_object_copies":16810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9235,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12284","reported_seq":12411,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:43.832557+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:43.832557+0000","last_peered":"2024-09-15T21:25:43.832557+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:43.832557+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33706467,"num_objects":8230,"num_object_clones":0,"num_object_copies":16460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8230,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10216,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2061},{"shards":"2","objects":4143}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12127","reported_seq":12258,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.445841+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:42.445841+0000","last_peered":"2024-09-15T21:25:42.445841+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:42.445841+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34222080,"num_objects":8355,"num_object_clones":0,"num_object_copies":16710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8800,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4335},{"shards":"0,2","objects":2138}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12322","reported_seq":12440,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:43.850536+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:43.850536+0000","last_peered":"2024-09-15T21:25:43.850536+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:43.850536+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34083299,"num_objects":8322,"num_object_clones":0,"num_object_copies":16644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10181,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2092},{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12349","reported_seq":20808,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:43.816455+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:43.816455+0000","last_peered":"2024-09-15T21:25:43.816455+0000","last_clean":"2024-09-15T21:25:43.816455+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:43.816455+0000","last_undegraded":"2024-09-15T21:25:43.816455+0000","last_fullsized":"2024-09-15T21:25:43.816455+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33632256,"num_objects":8211,"num_object_clones":0,"num_object_copies":16422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10042,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12302","reported_seq":12436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.433338+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:42.433338+0000","last_peered":"2024-09-15T21:25:42.433338+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:42.433338+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34340864,"num_objects":8384,"num_object_clones":0,"num_object_copies":16768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9246,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4290},{"shards":"0,2","objects":2136}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12307","reported_seq":12436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.454916+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:42.454916+0000","last_peered":"2024-09-15T21:25:42.454916+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:42.454916+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33927168,"num_objects":8283,"num_object_clones":0,"num_object_copies":16566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4144},{"shards":"0,2","objects":2131}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12157","reported_seq":16620,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:42.455045+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:42.455045+0000","last_peered":"2024-09-15T21:25:42.455045+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:42.455045+0000","last_undegraded":"2024-09-15T21:25:42.455045+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33714176,"num_objects":8231,"num_object_clones":0,"num_object_copies":16462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1919,"num_objects_unfound":0,"num_objects_dirty":8231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8641,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2163,"num_bytes_recovered":8859648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6312},{"shards":"0,2","objects":1919}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12426","reported_seq":12565,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.446569+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:42.446569+0000","last_peered":"2024-09-15T21:25:42.446569+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:42.446569+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34571206,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9130,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4295},{"shards":"0,2","objects":2160}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":546472694,"num_objects":133426,"num_object_clones":0,"num_object_copies":266852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1919,"num_objects_unfound":0,"num_objects_dirty":133426,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":157029,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6328,"num_bytes_recovered":25911296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1200648192,"data_stored":1200491915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696216,"kb_used_data":397960,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93675624,"statfs":{"total":96636764160,"available":95923838976,"internally_reserved":0,"allocated":407511040,"data_stored":407235393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843652,"kb_used_data":469108,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93528188,"statfs":{"total":96636764160,"available":95772864512,"internally_reserved":0,"allocated":480366592,"data_stored":480090940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619164,"kb_used_data":308428,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93752676,"statfs":{"total":96636764160,"available":96002740224,"internally_reserved":0,"allocated":315830272,"data_stored":315510805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,6,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":478695424,"data_stored":478659294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315314176,"data_stored":315234231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406638592,"data_stored":406598390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:46.497 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:47.053 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:47.054 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:47.081 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-15T21:25:46.777404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":546472713,"num_objects":133428,"num_object_clones":0,"num_object_copies":266856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1919,"num_objects_unfound":0,"num_objects_dirty":133428,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":157031,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6329,"num_bytes_recovered":25911315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2159032,"kb_used_data":1175496,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":280956488,"statfs":{"total":289910292480,"available":287699443712,"internally_reserved":0,"allocated":1203707904,"data_stored":1202837138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1007078403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,5,12,10,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18513920,"num_objects":-4520,"num_object_clones":0,"num_object_copies":-9040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-193,"num_objects_unfound":0,"num_objects_dirty":-4520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4520,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":790528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003366"},"pg_stats":[{"pgid":"2.f","version":"45'12563","reported_seq":12639,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.484042+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:42.484042+0000","last_peered":"2024-09-15T21:25:42.484042+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:42.484042+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35180544,"num_objects":8589,"num_object_clones":0,"num_object_copies":17178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12563,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4457},{"shards":"0,2","objects":2145}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12296","reported_seq":12402,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:43.778406+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:43.778406+0000","last_peered":"2024-09-15T21:25:43.778406+0000","last_clean":"2024-09-15T21:25:43.778406+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:43.778406+0000","last_undegraded":"2024-09-15T21:25:43.778406+0000","last_fullsized":"2024-09-15T21:25:43.778406+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34317254,"num_objects":8380,"num_object_clones":0,"num_object_copies":16760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8380,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10577,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12633","reported_seq":12759,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:43.832243+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:43.832243+0000","last_peered":"2024-09-15T21:25:43.832243+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:43.832243+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34701336,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10733,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2189},{"shards":"2","objects":4206}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12162","reported_seq":12233,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:43.849046+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:43.849046+0000","last_peered":"2024-09-15T21:25:43.849046+0000","last_clean":"2024-09-15T21:25:43.849046+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:43.849046+0000","last_undegraded":"2024-09-15T21:25:43.849046+0000","last_fullsized":"2024-09-15T21:25:43.849046+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33390592,"num_objects":8152,"num_object_clones":0,"num_object_copies":16304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12162,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12178","reported_seq":12311,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:42.356200+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:42.356200+0000","last_peered":"2024-09-15T21:25:42.356200+0000","last_clean":"2024-09-15T21:25:42.356200+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:42.356200+0000","last_undegraded":"2024-09-15T21:25:42.356200+0000","last_fullsized":"2024-09-15T21:25:42.356200+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34107875,"num_objects":8328,"num_object_clones":0,"num_object_copies":16656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8633,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12313","reported_seq":12450,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.421828+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:42.421828+0000","last_peered":"2024-09-15T21:25:42.421828+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:42.421828+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34087395,"num_objects":8323,"num_object_clones":0,"num_object_copies":16646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8323,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8989,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4247},{"shards":"0,2","objects":2083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12354","reported_seq":12489,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.488022+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:42.488022+0000","last_peered":"2024-09-15T21:25:42.488022+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:42.488022+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34063302,"num_objects":8318,"num_object_clones":0,"num_object_copies":16636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8665,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4157},{"shards":"0,2","objects":2145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12463","reported_seq":12592,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:42.396913+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:42.396913+0000","last_peered":"2024-09-15T21:25:42.396913+0000","last_clean":"2024-09-15T21:25:42.396913+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:42.396913+0000","last_undegraded":"2024-09-15T21:25:42.396913+0000","last_fullsized":"2024-09-15T21:25:42.396913+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34426880,"num_objects":8405,"num_object_clones":0,"num_object_copies":16810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9235,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12284","reported_seq":12411,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:43.832557+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:43.832557+0000","last_peered":"2024-09-15T21:25:43.832557+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:43.832557+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33706467,"num_objects":8230,"num_object_clones":0,"num_object_copies":16460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8230,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10216,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2061},{"shards":"2","objects":4143}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12127","reported_seq":12258,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.445841+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:42.445841+0000","last_peered":"2024-09-15T21:25:42.445841+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:42.445841+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34222080,"num_objects":8355,"num_object_clones":0,"num_object_copies":16710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8800,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4335},{"shards":"0,2","objects":2138}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12322","reported_seq":12440,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:43.850536+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:43.850536+0000","last_peered":"2024-09-15T21:25:43.850536+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:43.850536+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34083299,"num_objects":8322,"num_object_clones":0,"num_object_copies":16644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10181,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2092},{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12349","reported_seq":20808,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:43.816455+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:43.816455+0000","last_peered":"2024-09-15T21:25:43.816455+0000","last_clean":"2024-09-15T21:25:43.816455+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:43.816455+0000","last_undegraded":"2024-09-15T21:25:43.816455+0000","last_fullsized":"2024-09-15T21:25:43.816455+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33632256,"num_objects":8211,"num_object_clones":0,"num_object_copies":16422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10042,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12302","reported_seq":12436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.433338+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:42.433338+0000","last_peered":"2024-09-15T21:25:42.433338+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:42.433338+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34340864,"num_objects":8384,"num_object_clones":0,"num_object_copies":16768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9246,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4290},{"shards":"0,2","objects":2136}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12307","reported_seq":12436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.454916+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:42.454916+0000","last_peered":"2024-09-15T21:25:42.454916+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:42.454916+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33927168,"num_objects":8283,"num_object_clones":0,"num_object_copies":16566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4144},{"shards":"0,2","objects":2131}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12157","reported_seq":16620,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:42.455045+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:42.455045+0000","last_peered":"2024-09-15T21:25:42.455045+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:42.455045+0000","last_undegraded":"2024-09-15T21:25:42.455045+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33714176,"num_objects":8231,"num_object_clones":0,"num_object_copies":16462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1919,"num_objects_unfound":0,"num_objects_dirty":8231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8641,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2163,"num_bytes_recovered":8859648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6312},{"shards":"0,2","objects":1919}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12426","reported_seq":12565,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.446569+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:42.446569+0000","last_peered":"2024-09-15T21:25:42.446569+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:42.446569+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34571206,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9130,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4295},{"shards":"0,2","objects":2160}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":546472694,"num_objects":133426,"num_object_clones":0,"num_object_copies":266852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1919,"num_objects_unfound":0,"num_objects_dirty":133426,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":157029,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6328,"num_bytes_recovered":25911296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1200648192,"data_stored":1200491915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696216,"kb_used_data":397960,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93675624,"statfs":{"total":96636764160,"available":95923838976,"internally_reserved":0,"allocated":407511040,"data_stored":407235393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843652,"kb_used_data":469108,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93528188,"statfs":{"total":96636764160,"available":95772864512,"internally_reserved":0,"allocated":480366592,"data_stored":480090940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619164,"kb_used_data":308428,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93752676,"statfs":{"total":96636764160,"available":96002740224,"internally_reserved":0,"allocated":315830272,"data_stored":315510805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,6,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":478695424,"data_stored":478659294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315314176,"data_stored":315234231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406638592,"data_stored":406598390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:47.083 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:47.083 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12563", 'reported_seq': 12639, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:42.484042+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:25:42.484042+0000', 'last_peered': '2024-09-15T21:25:42.484042+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:25:42.484042+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35180544, 'num_objects': 8589, 'num_object_clones': 0, 'num_object_copies': 17178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8589, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12563, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4457}, {'shards': '0,2', 'objects': 2145}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:47.083 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:25:47.083 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'12633", 'reported_seq': 12759, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:43.832243+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:25:43.832243+0000', 'last_peered': '2024-09-15T21:25:43.832243+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:25:43.832243+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34701336, 'num_objects': 8473, 'num_object_clones': 0, 'num_object_copies': 16946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8473, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10733, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2189}, {'shards': '2', 'objects': 4206}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:47.083 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:47.083 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12313", 'reported_seq': 12450, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:42.421828+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:25:42.421828+0000', 'last_peered': '2024-09-15T21:25:42.421828+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:25:42.421828+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34087395, 'num_objects': 8323, 'num_object_clones': 0, 'num_object_copies': 16646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8323, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8989, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4247}, {'shards': '0,2', 'objects': 2083}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:47.084 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:47.084 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12354", 'reported_seq': 12489, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:42.488022+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:25:42.488022+0000', 'last_peered': '2024-09-15T21:25:42.488022+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:25:42.488022+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34063302, 'num_objects': 8318, 'num_object_clones': 0, 'num_object_copies': 16636, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8318, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8665, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4157}, {'shards': '0,2', 'objects': 2145}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:47.084 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:25:47.084 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12284", 'reported_seq': 12411, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:43.832557+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:25:43.832557+0000', 'last_peered': '2024-09-15T21:25:43.832557+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:25:43.832557+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33706467, 'num_objects': 8230, 'num_object_clones': 0, 'num_object_copies': 16460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8230, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10216, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2061}, {'shards': '2', 'objects': 4143}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:47.084 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:47.084 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12127", 'reported_seq': 12258, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:42.445841+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:25:42.445841+0000', 'last_peered': '2024-09-15T21:25:42.445841+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:25:42.445841+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34222080, 'num_objects': 8355, 'num_object_clones': 0, 'num_object_copies': 16710, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8355, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8800, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4335}, {'shards': '0,2', 'objects': 2138}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:47.084 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:47.084 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12322", 'reported_seq': 12440, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:43.850536+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:25:43.850536+0000', 'last_peered': '2024-09-15T21:25:43.850536+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:25:43.850536+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34083299, 'num_objects': 8322, 'num_object_clones': 0, 'num_object_copies': 16644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8322, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10181, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2092}, {'shards': '2', 'objects': 4232}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:47.084 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:47.084 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12302", 'reported_seq': 12436, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:42.433338+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:25:42.433338+0000', 'last_peered': '2024-09-15T21:25:42.433338+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:25:42.433338+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34340864, 'num_objects': 8384, 'num_object_clones': 0, 'num_object_copies': 16768, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8384, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9246, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4290}, {'shards': '0,2', 'objects': 2136}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:47.084 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:25:47.085 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12307", 'reported_seq': 12436, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:42.454916+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:25:42.454916+0000', 'last_peered': '2024-09-15T21:25:42.454916+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:25:42.454916+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33927168, 'num_objects': 8283, 'num_object_clones': 0, 'num_object_copies': 16566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8283, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9216, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4144}, {'shards': '0,2', 'objects': 2131}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:47.085 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:47.085 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12157", 'reported_seq': 16620, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:25:42.455045+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:25:42.455045+0000', 'last_peered': '2024-09-15T21:25:42.455045+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:25:42.455045+0000', 'last_undegraded': '2024-09-15T21:25:42.455045+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33714176, 'num_objects': 8231, 'num_object_clones': 0, 'num_object_copies': 16462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1919, 'num_objects_unfound': 0, 'num_objects_dirty': 8231, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8641, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2163, 'num_bytes_recovered': 8859648, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6312}, {'shards': '0,2', 'objects': 1919}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:47.085 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:25:47.085 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'12426", 'reported_seq': 12565, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:42.446569+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:25:42.446569+0000', 'last_peered': '2024-09-15T21:25:42.446569+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:25:42.446569+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34571206, 'num_objects': 8442, 'num_object_clones': 0, 'num_object_copies': 16884, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8442, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9130, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4295}, {'shards': '0,2', 'objects': 2160}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:47.085 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:47.778 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:47.779 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/266856 objects degraded (15.448%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":399,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":133428,"data_bytes":546472713,"bytes_used":2210848768,"bytes_avail":287699443712,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":266856,"degraded_ratio":0.15448406631291783,"misplaced_objects":1919,"misplaced_total":266856,"misplaced_ratio":0.0071911442875558351,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":65858,"recovering_keys_per_sec":0,"num_objects_recovered":193,"num_bytes_recovered":790528,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":376},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:25:47.802 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:47.802 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:48.372 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:48.374 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:48.395 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-15T21:25:46.777404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":546472713,"num_objects":133428,"num_object_clones":0,"num_object_copies":266856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1919,"num_objects_unfound":0,"num_objects_dirty":133428,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":157031,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6329,"num_bytes_recovered":25911315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2159032,"kb_used_data":1175496,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":280956488,"statfs":{"total":289910292480,"available":287699443712,"internally_reserved":0,"allocated":1203707904,"data_stored":1202837138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1007078403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,5,12,10,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18513920,"num_objects":-4520,"num_object_clones":0,"num_object_copies":-9040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-193,"num_objects_unfound":0,"num_objects_dirty":-4520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4520,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":790528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003366"},"pg_stats":[{"pgid":"2.f","version":"45'12563","reported_seq":12639,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.484042+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:42.484042+0000","last_peered":"2024-09-15T21:25:42.484042+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:42.484042+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35180544,"num_objects":8589,"num_object_clones":0,"num_object_copies":17178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12563,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4457},{"shards":"0,2","objects":2145}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12296","reported_seq":12402,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:43.778406+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:43.778406+0000","last_peered":"2024-09-15T21:25:43.778406+0000","last_clean":"2024-09-15T21:25:43.778406+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:43.778406+0000","last_undegraded":"2024-09-15T21:25:43.778406+0000","last_fullsized":"2024-09-15T21:25:43.778406+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34317254,"num_objects":8380,"num_object_clones":0,"num_object_copies":16760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8380,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10577,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12633","reported_seq":12759,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:43.832243+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:43.832243+0000","last_peered":"2024-09-15T21:25:43.832243+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:43.832243+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34701336,"num_objects":8473,"num_object_clones":0,"num_object_copies":16946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8473,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10733,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2189},{"shards":"2","objects":4206}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12162","reported_seq":12233,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:43.849046+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:43.849046+0000","last_peered":"2024-09-15T21:25:43.849046+0000","last_clean":"2024-09-15T21:25:43.849046+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:43.849046+0000","last_undegraded":"2024-09-15T21:25:43.849046+0000","last_fullsized":"2024-09-15T21:25:43.849046+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33390592,"num_objects":8152,"num_object_clones":0,"num_object_copies":16304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12162,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12178","reported_seq":12311,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:42.356200+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:42.356200+0000","last_peered":"2024-09-15T21:25:42.356200+0000","last_clean":"2024-09-15T21:25:42.356200+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:42.356200+0000","last_undegraded":"2024-09-15T21:25:42.356200+0000","last_fullsized":"2024-09-15T21:25:42.356200+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34107875,"num_objects":8328,"num_object_clones":0,"num_object_copies":16656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8633,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12313","reported_seq":12450,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.421828+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:42.421828+0000","last_peered":"2024-09-15T21:25:42.421828+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:42.421828+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34087395,"num_objects":8323,"num_object_clones":0,"num_object_copies":16646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8323,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8989,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4247},{"shards":"0,2","objects":2083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12354","reported_seq":12489,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.488022+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:42.488022+0000","last_peered":"2024-09-15T21:25:42.488022+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:42.488022+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34063302,"num_objects":8318,"num_object_clones":0,"num_object_copies":16636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8665,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4157},{"shards":"0,2","objects":2145}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12463","reported_seq":12592,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:42.396913+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:42.396913+0000","last_peered":"2024-09-15T21:25:42.396913+0000","last_clean":"2024-09-15T21:25:42.396913+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:42.396913+0000","last_undegraded":"2024-09-15T21:25:42.396913+0000","last_fullsized":"2024-09-15T21:25:42.396913+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34426880,"num_objects":8405,"num_object_clones":0,"num_object_copies":16810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9235,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12284","reported_seq":12411,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:43.832557+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:43.832557+0000","last_peered":"2024-09-15T21:25:43.832557+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:43.832557+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33706467,"num_objects":8230,"num_object_clones":0,"num_object_copies":16460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8230,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10216,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2061},{"shards":"2","objects":4143}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12127","reported_seq":12258,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.445841+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:42.445841+0000","last_peered":"2024-09-15T21:25:42.445841+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:42.445841+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34222080,"num_objects":8355,"num_object_clones":0,"num_object_copies":16710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8800,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4335},{"shards":"0,2","objects":2138}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12322","reported_seq":12440,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:43.850536+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:43.850536+0000","last_peered":"2024-09-15T21:25:43.850536+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:43.850536+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34083299,"num_objects":8322,"num_object_clones":0,"num_object_copies":16644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10181,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2092},{"shards":"2","objects":4232}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12349","reported_seq":20808,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:43.816455+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:43.816455+0000","last_peered":"2024-09-15T21:25:43.816455+0000","last_clean":"2024-09-15T21:25:43.816455+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:43.816455+0000","last_undegraded":"2024-09-15T21:25:43.816455+0000","last_fullsized":"2024-09-15T21:25:43.816455+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33632256,"num_objects":8211,"num_object_clones":0,"num_object_copies":16422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10042,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12302","reported_seq":12436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.433338+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:42.433338+0000","last_peered":"2024-09-15T21:25:42.433338+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:42.433338+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34340864,"num_objects":8384,"num_object_clones":0,"num_object_copies":16768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9246,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4290},{"shards":"0,2","objects":2136}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12307","reported_seq":12436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.454916+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:42.454916+0000","last_peered":"2024-09-15T21:25:42.454916+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:42.454916+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33927168,"num_objects":8283,"num_object_clones":0,"num_object_copies":16566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4144},{"shards":"0,2","objects":2131}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12157","reported_seq":16620,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:42.455045+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:42.455045+0000","last_peered":"2024-09-15T21:25:42.455045+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:42.455045+0000","last_undegraded":"2024-09-15T21:25:42.455045+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33714176,"num_objects":8231,"num_object_clones":0,"num_object_copies":16462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1919,"num_objects_unfound":0,"num_objects_dirty":8231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8641,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2163,"num_bytes_recovered":8859648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6312},{"shards":"0,2","objects":1919}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12426","reported_seq":12565,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:42.446569+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:42.446569+0000","last_peered":"2024-09-15T21:25:42.446569+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:42.446569+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34571206,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9130,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4295},{"shards":"0,2","objects":2160}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":546472694,"num_objects":133426,"num_object_clones":0,"num_object_copies":266852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1919,"num_objects_unfound":0,"num_objects_dirty":133426,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":157029,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6328,"num_bytes_recovered":25911296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1200648192,"data_stored":1200491915,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":696216,"kb_used_data":397960,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93675624,"statfs":{"total":96636764160,"available":95923838976,"internally_reserved":0,"allocated":407511040,"data_stored":407235393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":843652,"kb_used_data":469108,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93528188,"statfs":{"total":96636764160,"available":95772864512,"internally_reserved":0,"allocated":480366592,"data_stored":480090940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619164,"kb_used_data":308428,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93752676,"statfs":{"total":96636764160,"available":96002740224,"internally_reserved":0,"allocated":315830272,"data_stored":315510805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,6,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":478695424,"data_stored":478659294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315314176,"data_stored":315234231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":406638592,"data_stored":406598390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:51.397 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:51.948 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:51.950 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:51.975 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-15T21:25:50.779078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":537178889,"num_objects":131159,"num_object_clones":0,"num_object_copies":262318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1825,"num_objects_unfound":0,"num_objects_dirty":131159,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":159300,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6425,"num_bytes_recovered":26296339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160808,"ondisk_log_size":160808,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2141452,"kb_used_data":1162072,"kb_used_omap":12,"kb_used_meta":979315,"kb_avail":280974068,"statfs":{"total":289910292480,"available":287717445632,"internally_reserved":0,"allocated":1189961728,"data_stored":1189098188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1002818563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,4,14,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21946368,"num_objects":-5358,"num_object_clones":0,"num_object_copies":-10716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-194,"num_objects_unfound":0,"num_objects_dirty":-5358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5358,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":794624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003810"},"pg_stats":[{"pgid":"2.f","version":"45'12688","reported_seq":12763,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.440491+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:47.440491+0000","last_peered":"2024-09-15T21:25:47.440491+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:47.440491+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34668544,"num_objects":8464,"num_object_clones":0,"num_object_copies":16928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12688,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4332},{"shards":"0,2","objects":2020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12428","reported_seq":12533,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:48.893834+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:48.893834+0000","last_peered":"2024-09-15T21:25:48.893834+0000","last_clean":"2024-09-15T21:25:48.893834+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:48.893834+0000","last_undegraded":"2024-09-15T21:25:48.893834+0000","last_fullsized":"2024-09-15T21:25:48.893834+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33776582,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10709,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12790","reported_seq":12914,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:48.823264+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:48.823264+0000","last_peered":"2024-09-15T21:25:48.823264+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:48.823264+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34058264,"num_objects":8316,"num_object_clones":0,"num_object_copies":16632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10890,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2032},{"shards":"2","objects":4049}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12311","reported_seq":12382,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:48.853130+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:48.853130+0000","last_peered":"2024-09-15T21:25:48.853130+0000","last_clean":"2024-09-15T21:25:48.853130+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:48.853130+0000","last_undegraded":"2024-09-15T21:25:48.853130+0000","last_fullsized":"2024-09-15T21:25:48.853130+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32780288,"num_objects":8003,"num_object_clones":0,"num_object_copies":16006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12311,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12325","reported_seq":12458,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:47.530582+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:47.530582+0000","last_peered":"2024-09-15T21:25:47.530582+0000","last_clean":"2024-09-15T21:25:47.530582+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:47.530582+0000","last_undegraded":"2024-09-15T21:25:47.530582+0000","last_fullsized":"2024-09-15T21:25:47.530582+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505763,"num_objects":8181,"num_object_clones":0,"num_object_copies":16362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8780,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12447","reported_seq":12581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.509311+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:47.509311+0000","last_peered":"2024-09-15T21:25:47.509311+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:47.509311+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538531,"num_objects":8189,"num_object_clones":0,"num_object_copies":16378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8189,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9123,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4113},{"shards":"0,2","objects":1949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12494","reported_seq":12631,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.496012+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:47.496012+0000","last_peered":"2024-09-15T21:25:47.496012+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:47.496012+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33489862,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8805,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4017},{"shards":"0,2","objects":2005}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12608","reported_seq":12737,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:47.547359+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:47.547359+0000","last_peered":"2024-09-15T21:25:47.547359+0000","last_clean":"2024-09-15T21:25:47.547359+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:47.547359+0000","last_undegraded":"2024-09-15T21:25:47.547359+0000","last_fullsized":"2024-09-15T21:25:47.547359+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33832960,"num_objects":8260,"num_object_clones":0,"num_object_copies":16520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9380,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12406","reported_seq":12533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:48.720685+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:48.720685+0000","last_peered":"2024-09-15T21:25:48.720685+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:48.720685+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33206755,"num_objects":8108,"num_object_clones":0,"num_object_copies":16216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8108,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10338,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1939},{"shards":"2","objects":4021}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12276","reported_seq":12407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.353317+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:47.353317+0000","last_peered":"2024-09-15T21:25:47.353317+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:47.353317+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33611776,"num_objects":8206,"num_object_clones":0,"num_object_copies":16412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8949,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4186},{"shards":"0,2","objects":1989}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12463","reported_seq":12581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:48.879555+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:48.879555+0000","last_peered":"2024-09-15T21:25:48.879555+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:48.879555+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505763,"num_objects":8181,"num_object_clones":0,"num_object_copies":16362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10322,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1951},{"shards":"2","objects":4091}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12479","reported_seq":20938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:48.858377+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:48.858377+0000","last_peered":"2024-09-15T21:25:48.858377+0000","last_clean":"2024-09-15T21:25:48.858377+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:48.858377+0000","last_undegraded":"2024-09-15T21:25:48.858377+0000","last_fullsized":"2024-09-15T21:25:48.858377+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33099776,"num_objects":8081,"num_object_clones":0,"num_object_copies":16162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10172,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12449","reported_seq":12583,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.432144+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:47.432144+0000","last_peered":"2024-09-15T21:25:47.432144+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:47.432144+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33738752,"num_objects":8237,"num_object_clones":0,"num_object_copies":16474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9393,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4143},{"shards":"0,2","objects":1989}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12447","reported_seq":12580,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.492820+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:47.492820+0000","last_peered":"2024-09-15T21:25:47.492820+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:47.492820+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33353728,"num_objects":8143,"num_object_clones":0,"num_object_copies":16286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9356,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4004},{"shards":"0,2","objects":1991}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12316","reported_seq":16969,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:47.514917+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:47.514917+0000","last_peered":"2024-09-15T21:25:47.514917+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:47.514917+0000","last_undegraded":"2024-09-15T21:25:47.514917+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33062912,"num_objects":8072,"num_object_clones":0,"num_object_copies":16144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1825,"num_objects_unfound":0,"num_objects_dirty":8072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8800,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2259,"num_bytes_recovered":9244672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6247},{"shards":"0,2","objects":1825}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12578","reported_seq":12717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.522263+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:47.522263+0000","last_peered":"2024-09-15T21:25:47.522263+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:47.522263+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33948614,"num_objects":8290,"num_object_clones":0,"num_object_copies":16580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8290,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9282,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4143},{"shards":"0,2","objects":2008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":537178870,"num_objects":131157,"num_object_clones":0,"num_object_copies":262314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1825,"num_objects_unfound":0,"num_objects_dirty":131157,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":159298,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6424,"num_bytes_recovered":26296320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1188233216,"data_stored":1188084165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695896,"kb_used_data":393476,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93675944,"statfs":{"total":96636764160,"available":95924166656,"internally_reserved":0,"allocated":402919424,"data_stored":402643777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182474,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":839672,"kb_used_data":460968,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93532168,"statfs":{"total":96636764160,"available":95776940032,"internally_reserved":0,"allocated":472031232,"data_stored":471755580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,8,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":605884,"kb_used_data":307628,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93765956,"statfs":{"total":96636764160,"available":96016338944,"internally_reserved":0,"allocated":315011072,"data_stored":314698831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305393327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,3,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":471633920,"data_stored":471597790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314540032,"data_stored":314467313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402059264,"data_stored":402019062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:51.982 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:52.562 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:52.562 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:52.585 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-15T21:25:50.779078+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":537178889,"num_objects":131159,"num_object_clones":0,"num_object_copies":262318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1825,"num_objects_unfound":0,"num_objects_dirty":131159,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":159300,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6425,"num_bytes_recovered":26296339,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160808,"ondisk_log_size":160808,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2141452,"kb_used_data":1162072,"kb_used_omap":12,"kb_used_meta":979315,"kb_avail":280974068,"statfs":{"total":289910292480,"available":287717445632,"internally_reserved":0,"allocated":1189961728,"data_stored":1189098188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1002818563},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,4,14,4,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21946368,"num_objects":-5358,"num_object_clones":0,"num_object_copies":-10716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-194,"num_objects_unfound":0,"num_objects_dirty":-5358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5358,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":794624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003810"},"pg_stats":[{"pgid":"2.f","version":"45'12688","reported_seq":12763,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.440491+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:47.440491+0000","last_peered":"2024-09-15T21:25:47.440491+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:47.440491+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34668544,"num_objects":8464,"num_object_clones":0,"num_object_copies":16928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12688,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4332},{"shards":"0,2","objects":2020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12428","reported_seq":12533,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:48.893834+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:48.893834+0000","last_peered":"2024-09-15T21:25:48.893834+0000","last_clean":"2024-09-15T21:25:48.893834+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:48.893834+0000","last_undegraded":"2024-09-15T21:25:48.893834+0000","last_fullsized":"2024-09-15T21:25:48.893834+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33776582,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10709,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12790","reported_seq":12914,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:48.823264+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:48.823264+0000","last_peered":"2024-09-15T21:25:48.823264+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:48.823264+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34058264,"num_objects":8316,"num_object_clones":0,"num_object_copies":16632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10890,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2032},{"shards":"2","objects":4049}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12311","reported_seq":12382,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:48.853130+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:48.853130+0000","last_peered":"2024-09-15T21:25:48.853130+0000","last_clean":"2024-09-15T21:25:48.853130+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:48.853130+0000","last_undegraded":"2024-09-15T21:25:48.853130+0000","last_fullsized":"2024-09-15T21:25:48.853130+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32780288,"num_objects":8003,"num_object_clones":0,"num_object_copies":16006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12311,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12325","reported_seq":12458,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:47.530582+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:47.530582+0000","last_peered":"2024-09-15T21:25:47.530582+0000","last_clean":"2024-09-15T21:25:47.530582+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:47.530582+0000","last_undegraded":"2024-09-15T21:25:47.530582+0000","last_fullsized":"2024-09-15T21:25:47.530582+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505763,"num_objects":8181,"num_object_clones":0,"num_object_copies":16362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8780,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12447","reported_seq":12581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.509311+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:47.509311+0000","last_peered":"2024-09-15T21:25:47.509311+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:47.509311+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33538531,"num_objects":8189,"num_object_clones":0,"num_object_copies":16378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8189,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9123,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4113},{"shards":"0,2","objects":1949}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12494","reported_seq":12631,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.496012+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:47.496012+0000","last_peered":"2024-09-15T21:25:47.496012+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:47.496012+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33489862,"num_objects":8178,"num_object_clones":0,"num_object_copies":16356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8805,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4017},{"shards":"0,2","objects":2005}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12608","reported_seq":12737,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:47.547359+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:47.547359+0000","last_peered":"2024-09-15T21:25:47.547359+0000","last_clean":"2024-09-15T21:25:47.547359+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:47.547359+0000","last_undegraded":"2024-09-15T21:25:47.547359+0000","last_fullsized":"2024-09-15T21:25:47.547359+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33832960,"num_objects":8260,"num_object_clones":0,"num_object_copies":16520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9380,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12406","reported_seq":12533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:48.720685+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:48.720685+0000","last_peered":"2024-09-15T21:25:48.720685+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:48.720685+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33206755,"num_objects":8108,"num_object_clones":0,"num_object_copies":16216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8108,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10338,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1939},{"shards":"2","objects":4021}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12276","reported_seq":12407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.353317+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:47.353317+0000","last_peered":"2024-09-15T21:25:47.353317+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:47.353317+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33611776,"num_objects":8206,"num_object_clones":0,"num_object_copies":16412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8949,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4186},{"shards":"0,2","objects":1989}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12463","reported_seq":12581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:48.879555+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:48.879555+0000","last_peered":"2024-09-15T21:25:48.879555+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:48.879555+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505763,"num_objects":8181,"num_object_clones":0,"num_object_copies":16362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10322,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1951},{"shards":"2","objects":4091}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12479","reported_seq":20938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:48.858377+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:48.858377+0000","last_peered":"2024-09-15T21:25:48.858377+0000","last_clean":"2024-09-15T21:25:48.858377+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:48.858377+0000","last_undegraded":"2024-09-15T21:25:48.858377+0000","last_fullsized":"2024-09-15T21:25:48.858377+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33099776,"num_objects":8081,"num_object_clones":0,"num_object_copies":16162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10172,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12449","reported_seq":12583,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.432144+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:47.432144+0000","last_peered":"2024-09-15T21:25:47.432144+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:47.432144+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33738752,"num_objects":8237,"num_object_clones":0,"num_object_copies":16474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9393,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4143},{"shards":"0,2","objects":1989}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12447","reported_seq":12580,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.492820+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:47.492820+0000","last_peered":"2024-09-15T21:25:47.492820+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:47.492820+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33353728,"num_objects":8143,"num_object_clones":0,"num_object_copies":16286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9356,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4004},{"shards":"0,2","objects":1991}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12316","reported_seq":16969,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:47.514917+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:47.514917+0000","last_peered":"2024-09-15T21:25:47.514917+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:47.514917+0000","last_undegraded":"2024-09-15T21:25:47.514917+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33062912,"num_objects":8072,"num_object_clones":0,"num_object_copies":16144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1825,"num_objects_unfound":0,"num_objects_dirty":8072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8800,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2259,"num_bytes_recovered":9244672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6247},{"shards":"0,2","objects":1825}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12578","reported_seq":12717,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:47.522263+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:47.522263+0000","last_peered":"2024-09-15T21:25:47.522263+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:47.522263+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33948614,"num_objects":8290,"num_object_clones":0,"num_object_copies":16580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8290,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9282,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4143},{"shards":"0,2","objects":2008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":537178870,"num_objects":131157,"num_object_clones":0,"num_object_copies":262314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1825,"num_objects_unfound":0,"num_objects_dirty":131157,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":159298,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6424,"num_bytes_recovered":26296320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1188233216,"data_stored":1188084165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160805,"ondisk_log_size":160805,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695896,"kb_used_data":393476,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93675944,"statfs":{"total":96636764160,"available":95924166656,"internally_reserved":0,"allocated":402919424,"data_stored":402643777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182474,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":839672,"kb_used_data":460968,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93532168,"statfs":{"total":96636764160,"available":95776940032,"internally_reserved":0,"allocated":472031232,"data_stored":471755580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,2,8,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":605884,"kb_used_data":307628,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93765956,"statfs":{"total":96636764160,"available":96016338944,"internally_reserved":0,"allocated":315011072,"data_stored":314698831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305393327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,3,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":471633920,"data_stored":471597790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314540032,"data_stored":314467313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402059264,"data_stored":402019062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:52.587 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:52.587 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12688", 'reported_seq': 12763, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:47.440491+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:25:47.440491+0000', 'last_peered': '2024-09-15T21:25:47.440491+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:25:47.440491+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34668544, 'num_objects': 8464, 'num_object_clones': 0, 'num_object_copies': 16928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12688, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4332}, {'shards': '0,2', 'objects': 2020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:52.587 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:25:52.587 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'12790", 'reported_seq': 12914, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:48.823264+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:25:48.823264+0000', 'last_peered': '2024-09-15T21:25:48.823264+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:25:48.823264+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34058264, 'num_objects': 8316, 'num_object_clones': 0, 'num_object_copies': 16632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8316, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10890, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2032}, {'shards': '2', 'objects': 4049}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:52.587 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:52.587 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12447", 'reported_seq': 12581, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:47.509311+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:25:47.509311+0000', 'last_peered': '2024-09-15T21:25:47.509311+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:25:47.509311+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33538531, 'num_objects': 8189, 'num_object_clones': 0, 'num_object_copies': 16378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8189, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9123, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4113}, {'shards': '0,2', 'objects': 1949}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:52.587 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:52.587 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12494", 'reported_seq': 12631, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:47.496012+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:25:47.496012+0000', 'last_peered': '2024-09-15T21:25:47.496012+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:25:47.496012+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33489862, 'num_objects': 8178, 'num_object_clones': 0, 'num_object_copies': 16356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8805, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4017}, {'shards': '0,2', 'objects': 2005}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:52.587 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:25:52.588 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12406", 'reported_seq': 12533, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:48.720685+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:25:48.720685+0000', 'last_peered': '2024-09-15T21:25:48.720685+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:25:48.720685+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33206755, 'num_objects': 8108, 'num_object_clones': 0, 'num_object_copies': 16216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8108, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10338, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1939}, {'shards': '2', 'objects': 4021}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:52.588 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:52.588 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12276", 'reported_seq': 12407, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:47.353317+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:25:47.353317+0000', 'last_peered': '2024-09-15T21:25:47.353317+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:25:47.353317+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33611776, 'num_objects': 8206, 'num_object_clones': 0, 'num_object_copies': 16412, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8206, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8949, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4186}, {'shards': '0,2', 'objects': 1989}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:52.588 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:52.588 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12463", 'reported_seq': 12581, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:48.879555+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:25:48.879555+0000', 'last_peered': '2024-09-15T21:25:48.879555+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:25:48.879555+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33505763, 'num_objects': 8181, 'num_object_clones': 0, 'num_object_copies': 16362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8181, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10322, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1951}, {'shards': '2', 'objects': 4091}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:52.588 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:52.588 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12449", 'reported_seq': 12583, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:47.432144+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:25:47.432144+0000', 'last_peered': '2024-09-15T21:25:47.432144+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:25:47.432144+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33738752, 'num_objects': 8237, 'num_object_clones': 0, 'num_object_copies': 16474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8237, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9393, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4143}, {'shards': '0,2', 'objects': 1989}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:52.588 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:25:52.588 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12447", 'reported_seq': 12580, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:47.492820+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:25:47.492820+0000', 'last_peered': '2024-09-15T21:25:47.492820+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:25:47.492820+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33353728, 'num_objects': 8143, 'num_object_clones': 0, 'num_object_copies': 16286, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8143, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9356, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4004}, {'shards': '0,2', 'objects': 1991}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:52.593 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:52.593 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12316", 'reported_seq': 16969, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:25:47.514917+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:25:47.514917+0000', 'last_peered': '2024-09-15T21:25:47.514917+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:25:47.514917+0000', 'last_undegraded': '2024-09-15T21:25:47.514917+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33062912, 'num_objects': 8072, 'num_object_clones': 0, 'num_object_copies': 16144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1825, 'num_objects_unfound': 0, 'num_objects_dirty': 8072, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8800, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2259, 'num_bytes_recovered': 9244672, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6247}, {'shards': '0,2', 'objects': 1825}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:52.593 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:25:52.594 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'12578", 'reported_seq': 12717, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:47.522263+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:25:47.522263+0000', 'last_peered': '2024-09-15T21:25:47.522263+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:25:47.522263+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33948614, 'num_objects': 8290, 'num_object_clones': 0, 'num_object_copies': 16580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8290, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9282, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4143}, {'shards': '0,2', 'objects': 2008}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:52.594 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:53.300 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:53.300 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/259446 objects degraded (15.890%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":405,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":129723,"data_bytes":531297033,"bytes_used":2191179776,"bytes_avail":287719112704,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":259446,"degraded_ratio":0.1588962635770064,"misplaced_objects":1713,"misplaced_total":259446,"misplaced_ratio":0.0066025300062440735,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":104413,"recovering_keys_per_sec":0,"num_objects_recovered":308,"num_bytes_recovered":1253376,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":498},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:25:53.328 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:53.329 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:53.871 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:53.872 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:53.896 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-15T21:25:52.779789+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":531297033,"num_objects":129723,"num_object_clones":0,"num_object_copies":259446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1713,"num_objects_unfound":0,"num_objects_dirty":129723,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":160736,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6537,"num_bytes_recovered":26755091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160744,"ondisk_log_size":160744,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2139824,"kb_used_data":1156284,"kb_used_omap":12,"kb_used_meta":983475,"kb_avail":280975696,"statfs":{"total":289910292480,"available":287719112704,"internally_reserved":0,"allocated":1184034816,"data_stored":1183171276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1007078403},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,4,11,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24502272,"num_objects":-5982,"num_object_clones":0,"num_object_copies":-11964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-306,"num_objects_unfound":0,"num_objects_dirty":-5982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5982,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":1253376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003907"},"pg_stats":[{"pgid":"2.f","version":"45'12833","reported_seq":12909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.527230+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:52.527230+0000","last_peered":"2024-09-15T21:25:52.527230+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:52.527230+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34074624,"num_objects":8319,"num_object_clones":0,"num_object_copies":16638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12833,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4187},{"shards":"0,2","objects":1875}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12428","reported_seq":12533,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:48.893834+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:48.893834+0000","last_peered":"2024-09-15T21:25:48.893834+0000","last_clean":"2024-09-15T21:25:48.893834+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:48.893834+0000","last_undegraded":"2024-09-15T21:25:48.893834+0000","last_fullsized":"2024-09-15T21:25:48.893834+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33776582,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10709,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12790","reported_seq":12914,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:48.823264+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:48.823264+0000","last_peered":"2024-09-15T21:25:48.823264+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:48.823264+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34058264,"num_objects":8316,"num_object_clones":0,"num_object_copies":16632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10890,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2032},{"shards":"2","objects":4049}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12311","reported_seq":12382,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:48.853130+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:48.853130+0000","last_peered":"2024-09-15T21:25:48.853130+0000","last_clean":"2024-09-15T21:25:48.853130+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:48.853130+0000","last_undegraded":"2024-09-15T21:25:48.853130+0000","last_fullsized":"2024-09-15T21:25:48.853130+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32780288,"num_objects":8003,"num_object_clones":0,"num_object_copies":16006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12311,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12457","reported_seq":12590,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:52.511905+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:52.511905+0000","last_peered":"2024-09-15T21:25:52.511905+0000","last_clean":"2024-09-15T21:25:52.511905+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:52.511905+0000","last_undegraded":"2024-09-15T21:25:52.511905+0000","last_fullsized":"2024-09-15T21:25:52.511905+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32965091,"num_objects":8049,"num_object_clones":0,"num_object_copies":16098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8912,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12590","reported_seq":12727,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.535296+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:52.535296+0000","last_peered":"2024-09-15T21:25:52.535296+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:52.535296+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32952803,"num_objects":8046,"num_object_clones":0,"num_object_copies":16092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8046,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9266,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3970},{"shards":"0,2","objects":1806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12644","reported_seq":12779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.513019+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:52.513019+0000","last_peered":"2024-09-15T21:25:52.513019+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:52.513019+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32875462,"num_objects":8028,"num_object_clones":0,"num_object_copies":16056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8955,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3867},{"shards":"0,2","objects":1855}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12748","reported_seq":12876,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:52.554014+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:52.554014+0000","last_peered":"2024-09-15T21:25:52.554014+0000","last_clean":"2024-09-15T21:25:52.554014+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:52.554014+0000","last_undegraded":"2024-09-15T21:25:52.554014+0000","last_fullsized":"2024-09-15T21:25:52.554014+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33259520,"num_objects":8120,"num_object_clones":0,"num_object_copies":16240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9520,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12406","reported_seq":12533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:48.720685+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:48.720685+0000","last_peered":"2024-09-15T21:25:48.720685+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:48.720685+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33206755,"num_objects":8108,"num_object_clones":0,"num_object_copies":16216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8108,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10338,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1939},{"shards":"2","objects":4021}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12416","reported_seq":12547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.536733+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:52.536733+0000","last_peered":"2024-09-15T21:25:52.536733+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:52.536733+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33038336,"num_objects":8066,"num_object_clones":0,"num_object_copies":16132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9089,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4046},{"shards":"0,2","objects":1849}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12463","reported_seq":12581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:48.879555+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:48.879555+0000","last_peered":"2024-09-15T21:25:48.879555+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:48.879555+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33505763,"num_objects":8181,"num_object_clones":0,"num_object_copies":16362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8181,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10322,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1951},{"shards":"2","objects":4091}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12479","reported_seq":20938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:48.858377+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:48.858377+0000","last_peered":"2024-09-15T21:25:48.858377+0000","last_clean":"2024-09-15T21:25:48.858377+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:48.858377+0000","last_undegraded":"2024-09-15T21:25:48.858377+0000","last_fullsized":"2024-09-15T21:25:48.858377+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33099776,"num_objects":8081,"num_object_clones":0,"num_object_copies":16162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10172,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12593","reported_seq":12727,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.529696+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:52.529696+0000","last_peered":"2024-09-15T21:25:52.529696+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:52.529696+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33148928,"num_objects":8093,"num_object_clones":0,"num_object_copies":16186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9537,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3999},{"shards":"0,2","objects":1845}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12604","reported_seq":12737,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.490493+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:52.490493+0000","last_peered":"2024-09-15T21:25:52.490493+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:52.490493+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32710656,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9513,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":1834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12437","reported_seq":17316,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:52.602119+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:52.602119+0000","last_peered":"2024-09-15T21:25:52.602119+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:52.602119+0000","last_undegraded":"2024-09-15T21:25:52.602119+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32567296,"num_objects":7951,"num_object_clones":0,"num_object_copies":15902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1713,"num_objects_unfound":0,"num_objects_dirty":7951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8921,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2371,"num_bytes_recovered":9703424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6238},{"shards":"0,2","objects":1713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12742","reported_seq":12881,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.499425+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:52.499425+0000","last_peered":"2024-09-15T21:25:52.499425+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:52.499425+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33276870,"num_objects":8126,"num_object_clones":0,"num_object_copies":16252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8126,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9446,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3979},{"shards":"0,2","objects":1844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":531297014,"num_objects":129721,"num_object_clones":0,"num_object_copies":259442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1713,"num_objects_unfound":0,"num_objects_dirty":129721,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":160734,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6536,"num_bytes_recovered":26755072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1181151232,"data_stored":1181002181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160741,"ondisk_log_size":160741,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695896,"kb_used_data":393476,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93675944,"statfs":{"total":96636764160,"available":95924166656,"internally_reserved":0,"allocated":402919424,"data_stored":402643777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":838044,"kb_used_data":455180,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93533796,"statfs":{"total":96636764160,"available":95778607104,"internally_reserved":0,"allocated":466104320,"data_stored":465828668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":605884,"kb_used_data":307628,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93765956,"statfs":{"total":96636764160,"available":96016338944,"internally_reserved":0,"allocated":315011072,"data_stored":314698831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305393327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,3,1,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":464551936,"data_stored":464515806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314540032,"data_stored":314467313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":402059264,"data_stored":402019062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:56.899 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:57.459 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:57.462 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:57.480 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-15T21:25:56.780932+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527725321,"num_objects":128851,"num_object_clones":0,"num_object_copies":257702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1713,"num_objects_unfound":0,"num_objects_dirty":128851,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":161608,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6537,"num_bytes_recovered":26755091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160716,"ondisk_log_size":160716,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2147104,"kb_used_data":1151080,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":280968416,"statfs":{"total":289910292480,"available":287711657984,"internally_reserved":0,"allocated":1178705920,"data_stored":1177842380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1019857923},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,10,15,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18747392,"num_objects":-4577,"num_object_clones":0,"num_object_copies":-9154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-206,"num_objects_unfound":0,"num_objects_dirty":-4577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4577,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003996"},"pg_stats":[{"pgid":"2.f","version":"45'12833","reported_seq":12909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.527230+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:52.527230+0000","last_peered":"2024-09-15T21:25:52.527230+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:52.527230+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34074624,"num_objects":8319,"num_object_clones":0,"num_object_copies":16638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12833,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4187},{"shards":"0,2","objects":1875}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12578","reported_seq":12683,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:53.879198+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:53.879198+0000","last_peered":"2024-09-15T21:25:53.879198+0000","last_clean":"2024-09-15T21:25:53.879198+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:53.879198+0000","last_undegraded":"2024-09-15T21:25:53.879198+0000","last_fullsized":"2024-09-15T21:25:53.879198+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33162182,"num_objects":8098,"num_object_clones":0,"num_object_copies":16196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8098,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10859,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12926","reported_seq":13053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:53.885764+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:53.885764+0000","last_peered":"2024-09-15T21:25:53.885764+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:53.885764+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33501208,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11026,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1896},{"shards":"2","objects":3913}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12450","reported_seq":12521,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:53.878355+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:53.878355+0000","last_peered":"2024-09-15T21:25:53.878355+0000","last_clean":"2024-09-15T21:25:53.878355+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:53.878355+0000","last_undegraded":"2024-09-15T21:25:53.878355+0000","last_fullsized":"2024-09-15T21:25:53.878355+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32210944,"num_objects":7864,"num_object_clones":0,"num_object_copies":15728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12450,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12457","reported_seq":12590,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:52.511905+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:52.511905+0000","last_peered":"2024-09-15T21:25:52.511905+0000","last_clean":"2024-09-15T21:25:52.511905+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:52.511905+0000","last_undegraded":"2024-09-15T21:25:52.511905+0000","last_fullsized":"2024-09-15T21:25:52.511905+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32965091,"num_objects":8049,"num_object_clones":0,"num_object_copies":16098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8912,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12590","reported_seq":12727,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.535296+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:52.535296+0000","last_peered":"2024-09-15T21:25:52.535296+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:52.535296+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32952803,"num_objects":8046,"num_object_clones":0,"num_object_copies":16092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8046,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9266,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3970},{"shards":"0,2","objects":1806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12644","reported_seq":12779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.513019+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:52.513019+0000","last_peered":"2024-09-15T21:25:52.513019+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:52.513019+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32875462,"num_objects":8028,"num_object_clones":0,"num_object_copies":16056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8955,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3867},{"shards":"0,2","objects":1855}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12748","reported_seq":12876,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:52.554014+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:52.554014+0000","last_peered":"2024-09-15T21:25:52.554014+0000","last_clean":"2024-09-15T21:25:52.554014+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:52.554014+0000","last_undegraded":"2024-09-15T21:25:52.554014+0000","last_fullsized":"2024-09-15T21:25:52.554014+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33259520,"num_objects":8120,"num_object_clones":0,"num_object_copies":16240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9520,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12551","reported_seq":12678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:53.909471+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:53.909471+0000","last_peered":"2024-09-15T21:25:53.909471+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:53.909471+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32612835,"num_objects":7963,"num_object_clones":0,"num_object_copies":15926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10483,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1794},{"shards":"2","objects":3876}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12416","reported_seq":12547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.536733+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:52.536733+0000","last_peered":"2024-09-15T21:25:52.536733+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:52.536733+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33038336,"num_objects":8066,"num_object_clones":0,"num_object_copies":16132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9089,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4046},{"shards":"0,2","objects":1849}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12610","reported_seq":12729,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:53.858205+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:53.858205+0000","last_peered":"2024-09-15T21:25:53.858205+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:53.858205+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32903651,"num_objects":8034,"num_object_clones":0,"num_object_copies":16068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8034,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10469,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1804},{"shards":"2","objects":3944}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12634","reported_seq":21093,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:53.825993+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:53.825993+0000","last_peered":"2024-09-15T21:25:53.825993+0000","last_clean":"2024-09-15T21:25:53.825993+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:53.825993+0000","last_undegraded":"2024-09-15T21:25:53.825993+0000","last_fullsized":"2024-09-15T21:25:53.825993+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32464896,"num_objects":7926,"num_object_clones":0,"num_object_copies":15852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10327,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12593","reported_seq":12727,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.529696+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:52.529696+0000","last_peered":"2024-09-15T21:25:52.529696+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:52.529696+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33148928,"num_objects":8093,"num_object_clones":0,"num_object_copies":16186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9537,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3999},{"shards":"0,2","objects":1845}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12604","reported_seq":12737,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.490493+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:52.490493+0000","last_peered":"2024-09-15T21:25:52.490493+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:52.490493+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32710656,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9513,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":1834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12437","reported_seq":17316,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:52.602119+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:52.602119+0000","last_peered":"2024-09-15T21:25:52.602119+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:52.602119+0000","last_undegraded":"2024-09-15T21:25:52.602119+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32567296,"num_objects":7951,"num_object_clones":0,"num_object_copies":15902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1713,"num_objects_unfound":0,"num_objects_dirty":7951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8921,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2371,"num_bytes_recovered":9703424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6238},{"shards":"0,2","objects":1713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12742","reported_seq":12881,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.499425+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:52.499425+0000","last_peered":"2024-09-15T21:25:52.499425+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:52.499425+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33276870,"num_objects":8126,"num_object_clones":0,"num_object_copies":16252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8126,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9446,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3979},{"shards":"0,2","objects":1844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":527725302,"num_objects":128849,"num_object_clones":0,"num_object_copies":257698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1713,"num_objects_unfound":0,"num_objects_dirty":128849,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":161606,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6536,"num_bytes_recovered":26755072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1175724032,"data_stored":1175574981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695512,"kb_used_data":388928,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93676328,"statfs":{"total":96636764160,"available":95924559872,"internally_reserved":0,"allocated":398262272,"data_stored":397986625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,6,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":838044,"kb_used_data":455180,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93533796,"statfs":{"total":96636764160,"available":95778607104,"internally_reserved":0,"allocated":466104320,"data_stored":465828668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613548,"kb_used_data":306972,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93758292,"statfs":{"total":96636764160,"available":96008491008,"internally_reserved":0,"allocated":314339328,"data_stored":314027087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,7,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":464551936,"data_stored":464515806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313843712,"data_stored":313770993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":397328384,"data_stored":397288182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:57.481 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:58.042 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:58.042 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:58.064 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-15T21:25:56.780932+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":527725321,"num_objects":128851,"num_object_clones":0,"num_object_copies":257702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1713,"num_objects_unfound":0,"num_objects_dirty":128851,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":161608,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6537,"num_bytes_recovered":26755091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160716,"ondisk_log_size":160716,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2147104,"kb_used_data":1151080,"kb_used_omap":12,"kb_used_meta":995955,"kb_avail":280968416,"statfs":{"total":289910292480,"available":287711657984,"internally_reserved":0,"allocated":1178705920,"data_stored":1177842380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1019857923},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,10,15,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18747392,"num_objects":-4577,"num_object_clones":0,"num_object_copies":-9154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-206,"num_objects_unfound":0,"num_objects_dirty":-4577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4577,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":208,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003996"},"pg_stats":[{"pgid":"2.f","version":"45'12833","reported_seq":12909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.527230+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:52.527230+0000","last_peered":"2024-09-15T21:25:52.527230+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:52.527230+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34074624,"num_objects":8319,"num_object_clones":0,"num_object_copies":16638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8319,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12833,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4187},{"shards":"0,2","objects":1875}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12578","reported_seq":12683,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:53.879198+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:53.879198+0000","last_peered":"2024-09-15T21:25:53.879198+0000","last_clean":"2024-09-15T21:25:53.879198+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:53.879198+0000","last_undegraded":"2024-09-15T21:25:53.879198+0000","last_fullsized":"2024-09-15T21:25:53.879198+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33162182,"num_objects":8098,"num_object_clones":0,"num_object_copies":16196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8098,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10859,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12926","reported_seq":13053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:53.885764+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:53.885764+0000","last_peered":"2024-09-15T21:25:53.885764+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:53.885764+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33501208,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11026,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1896},{"shards":"2","objects":3913}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12450","reported_seq":12521,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:53.878355+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:53.878355+0000","last_peered":"2024-09-15T21:25:53.878355+0000","last_clean":"2024-09-15T21:25:53.878355+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:53.878355+0000","last_undegraded":"2024-09-15T21:25:53.878355+0000","last_fullsized":"2024-09-15T21:25:53.878355+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32210944,"num_objects":7864,"num_object_clones":0,"num_object_copies":15728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12450,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12457","reported_seq":12590,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:52.511905+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:52.511905+0000","last_peered":"2024-09-15T21:25:52.511905+0000","last_clean":"2024-09-15T21:25:52.511905+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:52.511905+0000","last_undegraded":"2024-09-15T21:25:52.511905+0000","last_fullsized":"2024-09-15T21:25:52.511905+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32965091,"num_objects":8049,"num_object_clones":0,"num_object_copies":16098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8912,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12590","reported_seq":12727,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.535296+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:52.535296+0000","last_peered":"2024-09-15T21:25:52.535296+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:52.535296+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32952803,"num_objects":8046,"num_object_clones":0,"num_object_copies":16092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8046,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9266,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3970},{"shards":"0,2","objects":1806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12644","reported_seq":12779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.513019+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:52.513019+0000","last_peered":"2024-09-15T21:25:52.513019+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:52.513019+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32875462,"num_objects":8028,"num_object_clones":0,"num_object_copies":16056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8028,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8955,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3867},{"shards":"0,2","objects":1855}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12748","reported_seq":12876,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:52.554014+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:52.554014+0000","last_peered":"2024-09-15T21:25:52.554014+0000","last_clean":"2024-09-15T21:25:52.554014+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:52.554014+0000","last_undegraded":"2024-09-15T21:25:52.554014+0000","last_fullsized":"2024-09-15T21:25:52.554014+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33259520,"num_objects":8120,"num_object_clones":0,"num_object_copies":16240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9520,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12551","reported_seq":12678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:53.909471+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:53.909471+0000","last_peered":"2024-09-15T21:25:53.909471+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:53.909471+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32612835,"num_objects":7963,"num_object_clones":0,"num_object_copies":15926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10483,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1794},{"shards":"2","objects":3876}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12416","reported_seq":12547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.536733+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:52.536733+0000","last_peered":"2024-09-15T21:25:52.536733+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:52.536733+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33038336,"num_objects":8066,"num_object_clones":0,"num_object_copies":16132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9089,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4046},{"shards":"0,2","objects":1849}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12610","reported_seq":12729,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:53.858205+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:53.858205+0000","last_peered":"2024-09-15T21:25:53.858205+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:53.858205+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32903651,"num_objects":8034,"num_object_clones":0,"num_object_copies":16068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8034,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10469,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1804},{"shards":"2","objects":3944}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12634","reported_seq":21093,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:53.825993+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:53.825993+0000","last_peered":"2024-09-15T21:25:53.825993+0000","last_clean":"2024-09-15T21:25:53.825993+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:53.825993+0000","last_undegraded":"2024-09-15T21:25:53.825993+0000","last_fullsized":"2024-09-15T21:25:53.825993+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32464896,"num_objects":7926,"num_object_clones":0,"num_object_copies":15852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10327,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12593","reported_seq":12727,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.529696+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:52.529696+0000","last_peered":"2024-09-15T21:25:52.529696+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:52.529696+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33148928,"num_objects":8093,"num_object_clones":0,"num_object_copies":16186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9537,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3999},{"shards":"0,2","objects":1845}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12604","reported_seq":12737,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.490493+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:52.490493+0000","last_peered":"2024-09-15T21:25:52.490493+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:52.490493+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32710656,"num_objects":7986,"num_object_clones":0,"num_object_copies":15972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9513,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3847},{"shards":"0,2","objects":1834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12437","reported_seq":17316,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:52.602119+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:52.602119+0000","last_peered":"2024-09-15T21:25:52.602119+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:52.602119+0000","last_undegraded":"2024-09-15T21:25:52.602119+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32567296,"num_objects":7951,"num_object_clones":0,"num_object_copies":15902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1713,"num_objects_unfound":0,"num_objects_dirty":7951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8921,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2371,"num_bytes_recovered":9703424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6238},{"shards":"0,2","objects":1713}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12742","reported_seq":12881,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:52.499425+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:52.499425+0000","last_peered":"2024-09-15T21:25:52.499425+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:52.499425+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33276870,"num_objects":8126,"num_object_clones":0,"num_object_copies":16252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8126,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9446,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3979},{"shards":"0,2","objects":1844}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":527725302,"num_objects":128849,"num_object_clones":0,"num_object_copies":257698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1713,"num_objects_unfound":0,"num_objects_dirty":128849,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":161606,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6536,"num_bytes_recovered":26755072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1175724032,"data_stored":1175574981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695512,"kb_used_data":388928,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93676328,"statfs":{"total":96636764160,"available":95924559872,"internally_reserved":0,"allocated":398262272,"data_stored":397986625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,6,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":838044,"kb_used_data":455180,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93533796,"statfs":{"total":96636764160,"available":95778607104,"internally_reserved":0,"allocated":466104320,"data_stored":465828668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613548,"kb_used_data":306972,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93758292,"statfs":{"total":96636764160,"available":96008491008,"internally_reserved":0,"allocated":314339328,"data_stored":314027087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313913007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,7,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":464551936,"data_stored":464515806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313843712,"data_stored":313770993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":397328384,"data_stored":397288182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:25:58.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:25:58.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'12833", 'reported_seq': 12909, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:52.527230+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:25:52.527230+0000', 'last_peered': '2024-09-15T21:25:52.527230+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:25:52.527230+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34074624, 'num_objects': 8319, 'num_object_clones': 0, 'num_object_copies': 16638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8319, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12833, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4187}, {'shards': '0,2', 'objects': 1875}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:58.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:25:58.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'12926", 'reported_seq': 13053, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:53.885764+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:25:53.885764+0000', 'last_peered': '2024-09-15T21:25:53.885764+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:25:53.885764+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33501208, 'num_objects': 8180, 'num_object_clones': 0, 'num_object_copies': 16360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8180, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11026, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1896}, {'shards': '2', 'objects': 3913}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:58.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:25:58.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12590", 'reported_seq': 12727, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:52.535296+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:25:52.535296+0000', 'last_peered': '2024-09-15T21:25:52.535296+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:25:52.535296+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32952803, 'num_objects': 8046, 'num_object_clones': 0, 'num_object_copies': 16092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8046, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9266, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3970}, {'shards': '0,2', 'objects': 1806}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:58.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:25:58.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12644", 'reported_seq': 12779, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:52.513019+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:25:52.513019+0000', 'last_peered': '2024-09-15T21:25:52.513019+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:25:52.513019+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32875462, 'num_objects': 8028, 'num_object_clones': 0, 'num_object_copies': 16056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8028, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8955, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3867}, {'shards': '0,2', 'objects': 1855}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:58.066 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:25:58.066 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12551", 'reported_seq': 12678, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:53.909471+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:25:53.909471+0000', 'last_peered': '2024-09-15T21:25:53.909471+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:25:53.909471+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32612835, 'num_objects': 7963, 'num_object_clones': 0, 'num_object_copies': 15926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7963, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10483, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1794}, {'shards': '2', 'objects': 3876}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:58.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:25:58.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12416", 'reported_seq': 12547, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:52.536733+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:25:52.536733+0000', 'last_peered': '2024-09-15T21:25:52.536733+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:25:52.536733+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33038336, 'num_objects': 8066, 'num_object_clones': 0, 'num_object_copies': 16132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8066, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9089, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4046}, {'shards': '0,2', 'objects': 1849}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:58.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:25:58.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12610", 'reported_seq': 12729, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:53.858205+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:25:53.858205+0000', 'last_peered': '2024-09-15T21:25:53.858205+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:25:53.858205+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32903651, 'num_objects': 8034, 'num_object_clones': 0, 'num_object_copies': 16068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8034, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10469, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1804}, {'shards': '2', 'objects': 3944}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:25:58.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:25:58.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12593", 'reported_seq': 12727, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:52.529696+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:25:52.529696+0000', 'last_peered': '2024-09-15T21:25:52.529696+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:25:52.529696+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33148928, 'num_objects': 8093, 'num_object_clones': 0, 'num_object_copies': 16186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8093, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9537, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3999}, {'shards': '0,2', 'objects': 1845}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:58.067 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:25:58.067 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12604", 'reported_seq': 12737, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:52.490493+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:25:52.490493+0000', 'last_peered': '2024-09-15T21:25:52.490493+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:25:52.490493+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32710656, 'num_objects': 7986, 'num_object_clones': 0, 'num_object_copies': 15972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7986, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9513, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3847}, {'shards': '0,2', 'objects': 1834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:58.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:25:58.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12437", 'reported_seq': 17316, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:25:52.602119+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:25:52.602119+0000', 'last_peered': '2024-09-15T21:25:52.602119+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:25:52.602119+0000', 'last_undegraded': '2024-09-15T21:25:52.602119+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32567296, 'num_objects': 7951, 'num_object_clones': 0, 'num_object_copies': 15902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1713, 'num_objects_unfound': 0, 'num_objects_dirty': 7951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8921, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2371, 'num_bytes_recovered': 9703424, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6238}, {'shards': '0,2', 'objects': 1713}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:58.068 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:25:58.068 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'12742", 'reported_seq': 12881, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:52.499425+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:25:52.499425+0000', 'last_peered': '2024-09-15T21:25:52.499425+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:25:52.499425+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33276870, 'num_objects': 8126, 'num_object_clones': 0, 'num_object_copies': 16252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8126, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9446, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3979}, {'shards': '0,2', 'objects': 1844}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:25:58.068 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:25:58.752 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:58.752 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/257702 objects degraded (15.997%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":410,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":128851,"data_bytes":527725321,"bytes_used":2198634496,"bytes_avail":287711657984,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":257702,"degraded_ratio":0.15997159509821421,"misplaced_objects":1713,"misplaced_total":257702,"misplaced_ratio":0.0066472126720009935,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":70291,"recovering_keys_per_sec":0,"num_objects_recovered":208,"num_bytes_recovered":843776,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":381},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:25:58.780 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:25:58.780 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:25:59.327 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:25:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:25:59.350 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-15T21:25:58.781500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":521831177,"num_objects":127412,"num_object_clones":0,"num_object_copies":254824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1608,"num_objects_unfound":0,"num_objects_dirty":127412,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":163047,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6642,"num_bytes_recovered":27185171,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2135420,"kb_used_data":1143544,"kb_used_omap":12,"kb_used_meta":991795,"kb_avail":280980100,"statfs":{"total":289910292480,"available":287723622400,"internally_reserved":0,"allocated":1170989056,"data_stored":1170125516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1015598083},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,2,2,17,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24641536,"num_objects":-6016,"num_object_clones":0,"num_object_copies":-12032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-311,"num_objects_unfound":0,"num_objects_dirty":-6016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6016,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1273856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004095"},"pg_stats":[{"pgid":"2.f","version":"45'12969","reported_seq":13045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:57.575624+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:25:57.575624+0000","last_peered":"2024-09-15T21:25:57.575624+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:25:57.575624+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33517568,"num_objects":8183,"num_object_clones":0,"num_object_copies":16366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12969,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4051},{"shards":"0,2","objects":1739}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12578","reported_seq":12683,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:53.879198+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:53.879198+0000","last_peered":"2024-09-15T21:25:53.879198+0000","last_clean":"2024-09-15T21:25:53.879198+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:53.879198+0000","last_undegraded":"2024-09-15T21:25:53.879198+0000","last_fullsized":"2024-09-15T21:25:53.879198+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33162182,"num_objects":8098,"num_object_clones":0,"num_object_copies":16196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8098,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10859,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12926","reported_seq":13053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:53.885764+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:53.885764+0000","last_peered":"2024-09-15T21:25:53.885764+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:53.885764+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33501208,"num_objects":8180,"num_object_clones":0,"num_object_copies":16360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11026,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1896},{"shards":"2","objects":3913}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12450","reported_seq":12521,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:53.878355+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:53.878355+0000","last_peered":"2024-09-15T21:25:53.878355+0000","last_clean":"2024-09-15T21:25:53.878355+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:53.878355+0000","last_undegraded":"2024-09-15T21:25:53.878355+0000","last_fullsized":"2024-09-15T21:25:53.878355+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32210944,"num_objects":7864,"num_object_clones":0,"num_object_copies":15728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12450,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12620","reported_seq":12753,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:57.630108+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:25:57.630108+0000","last_peered":"2024-09-15T21:25:57.630108+0000","last_clean":"2024-09-15T21:25:57.630108+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:25:57.630108+0000","last_undegraded":"2024-09-15T21:25:57.630108+0000","last_fullsized":"2024-09-15T21:25:57.630108+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32297443,"num_objects":7886,"num_object_clones":0,"num_object_copies":15772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9075,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12716","reported_seq":12853,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:57.464448+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:25:57.464448+0000","last_peered":"2024-09-15T21:25:57.464448+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:25:57.464448+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32436707,"num_objects":7920,"num_object_clones":0,"num_object_copies":15840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7920,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9392,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3844},{"shards":"0,2","objects":1680}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12781","reported_seq":12918,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:57.575874+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:25:57.575874+0000","last_peered":"2024-09-15T21:25:57.575874+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:25:57.575874+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32314310,"num_objects":7891,"num_object_clones":0,"num_object_copies":15782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9092,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3730},{"shards":"0,2","objects":1718}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12905","reported_seq":13034,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:57.590312+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:25:57.590312+0000","last_peered":"2024-09-15T21:25:57.590312+0000","last_clean":"2024-09-15T21:25:57.590312+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:25:57.590312+0000","last_undegraded":"2024-09-15T21:25:57.590312+0000","last_fullsized":"2024-09-15T21:25:57.590312+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32616448,"num_objects":7963,"num_object_clones":0,"num_object_copies":15926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9677,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12551","reported_seq":12678,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:53.909471+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:53.909471+0000","last_peered":"2024-09-15T21:25:53.909471+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:53.909471+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32612835,"num_objects":7963,"num_object_clones":0,"num_object_copies":15926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10483,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1794},{"shards":"2","objects":3876}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12566","reported_seq":12694,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:57.614610+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:25:57.614610+0000","last_peered":"2024-09-15T21:25:57.614610+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:25:57.614610+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32423936,"num_objects":7916,"num_object_clones":0,"num_object_copies":15832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9239,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3896},{"shards":"0,2","objects":1699}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12610","reported_seq":12729,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:53.858205+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:53.858205+0000","last_peered":"2024-09-15T21:25:53.858205+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:53.858205+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32903651,"num_objects":8034,"num_object_clones":0,"num_object_copies":16068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8034,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10469,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1804},{"shards":"2","objects":3944}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12634","reported_seq":21093,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:53.825993+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:53.825993+0000","last_peered":"2024-09-15T21:25:53.825993+0000","last_clean":"2024-09-15T21:25:53.825993+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:53.825993+0000","last_undegraded":"2024-09-15T21:25:53.825993+0000","last_fullsized":"2024-09-15T21:25:53.825993+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32464896,"num_objects":7926,"num_object_clones":0,"num_object_copies":15852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10327,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12735","reported_seq":12868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:57.604244+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:25:57.604244+0000","last_peered":"2024-09-15T21:25:57.604244+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:25:57.604244+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32567296,"num_objects":7951,"num_object_clones":0,"num_object_copies":15902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9679,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3857},{"shards":"0,2","objects":1703}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12764","reported_seq":12897,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:57.500399+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:25:57.500399+0000","last_peered":"2024-09-15T21:25:57.500399+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:25:57.500399+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32055296,"num_objects":7826,"num_object_clones":0,"num_object_copies":15652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9673,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3687},{"shards":"0,2","objects":1674}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12581","reported_seq":17670,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:25:57.593613+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:25:57.593613+0000","last_peered":"2024-09-15T21:25:57.593613+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:25:57.593613+0000","last_undegraded":"2024-09-15T21:25:57.593613+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31977472,"num_objects":7807,"num_object_clones":0,"num_object_copies":15614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1608,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9065,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2476,"num_bytes_recovered":10133504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6199},{"shards":"0,2","objects":1608}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12866","reported_seq":13005,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:57.484986+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:25:57.484986+0000","last_peered":"2024-09-15T21:25:57.484986+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:25:57.484986+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32768966,"num_objects":8002,"num_object_clones":0,"num_object_copies":16004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8002,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3855},{"shards":"0,2","objects":1720}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":521831158,"num_objects":127410,"num_object_clones":0,"num_object_copies":254820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1608,"num_objects_unfound":0,"num_objects_dirty":127410,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":163045,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6641,"num_bytes_recovered":27185152,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1167921152,"data_stored":1167772101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160752,"ondisk_log_size":160752,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695512,"kb_used_data":388928,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93676328,"statfs":{"total":96636764160,"available":95924559872,"internally_reserved":0,"allocated":398262272,"data_stored":397986625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,6,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":822924,"kb_used_data":448368,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93548916,"statfs":{"total":96636764160,"available":95794089984,"internally_reserved":0,"allocated":459128832,"data_stored":458853180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,1,6,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616984,"kb_used_data":306248,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93754856,"statfs":{"total":96636764160,"available":96004972544,"internally_reserved":0,"allocated":313597952,"data_stored":313285711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":457482240,"data_stored":457446110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313110528,"data_stored":313037809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":397328384,"data_stored":397288182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:02.352 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:02.937 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:02.937 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:02.965 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-15T21:26:02.782404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":512434953,"num_objects":125118,"num_object_clones":0,"num_object_copies":250236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1514,"num_objects_unfound":0,"num_objects_dirty":125118,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":165341,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6736,"num_bytes_recovered":27570195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2132232,"kb_used_data":1132044,"kb_used_omap":12,"kb_used_meta":1000115,"kb_avail":280983288,"statfs":{"total":289910292480,"available":287726886912,"internally_reserved":0,"allocated":1159213056,"data_stored":1158349516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1024117763},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,7,5,15,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24743936,"num_objects":-6041,"num_object_clones":0,"num_object_copies":-12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-311,"num_objects_unfound":0,"num_objects_dirty":-6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6041,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":1273856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003326"},"pg_stats":[{"pgid":"2.f","version":"45'13114","reported_seq":13189,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.689127+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:02.689127+0000","last_peered":"2024-09-15T21:26:02.689127+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:02.689127+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13114,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3906},{"shards":"0,2","objects":1594}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12716","reported_seq":12822,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:58.920581+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:58.920581+0000","last_peered":"2024-09-15T21:25:58.920581+0000","last_clean":"2024-09-15T21:25:58.920581+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:58.920581+0000","last_undegraded":"2024-09-15T21:25:58.920581+0000","last_fullsized":"2024-09-15T21:25:58.920581+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32596934,"num_objects":7960,"num_object_clones":0,"num_object_copies":15920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7960,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10997,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13059","reported_seq":13186,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:58.855693+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:58.855693+0000","last_peered":"2024-09-15T21:25:58.855693+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:58.855693+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32956440,"num_objects":8047,"num_object_clones":0,"num_object_copies":16094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8047,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11159,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1763},{"shards":"2","objects":3780}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12591","reported_seq":12662,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:58.857793+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:58.857793+0000","last_peered":"2024-09-15T21:25:58.857793+0000","last_clean":"2024-09-15T21:25:58.857793+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:58.857793+0000","last_undegraded":"2024-09-15T21:25:58.857793+0000","last_fullsized":"2024-09-15T21:25:58.857793+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31633408,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12591,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12758","reported_seq":12891,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:02.635022+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:02.635022+0000","last_peered":"2024-09-15T21:26:02.635022+0000","last_clean":"2024-09-15T21:26:02.635022+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:02.635022+0000","last_undegraded":"2024-09-15T21:26:02.635022+0000","last_fullsized":"2024-09-15T21:26:02.635022+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31732195,"num_objects":7748,"num_object_clones":0,"num_object_copies":15496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9213,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12890","reported_seq":13025,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.630131+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:02.630131+0000","last_peered":"2024-09-15T21:26:02.630131+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:02.630131+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31724003,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9566,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3670},{"shards":"0,2","objects":1506}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12926","reported_seq":13062,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.626914+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:02.626914+0000","last_peered":"2024-09-15T21:26:02.626914+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:02.626914+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31720390,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9237,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3585},{"shards":"0,2","objects":1573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13055","reported_seq":13184,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:02.589573+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:02.589573+0000","last_peered":"2024-09-15T21:26:02.589573+0000","last_clean":"2024-09-15T21:26:02.589573+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:02.589573+0000","last_undegraded":"2024-09-15T21:26:02.589573+0000","last_fullsized":"2024-09-15T21:26:02.589573+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32002048,"num_objects":7813,"num_object_clones":0,"num_object_copies":15626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9827,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12696","reported_seq":12823,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:58.892992+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:58.892992+0000","last_peered":"2024-09-15T21:25:58.892992+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:58.892992+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32018915,"num_objects":7818,"num_object_clones":0,"num_object_copies":15636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7818,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10628,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1649},{"shards":"2","objects":3731}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12705","reported_seq":12836,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.628483+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:02.628483+0000","last_peered":"2024-09-15T21:26:02.628483+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:02.628483+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31854592,"num_objects":7777,"num_object_clones":0,"num_object_copies":15554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9378,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3757},{"shards":"0,2","objects":1560}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12748","reported_seq":12867,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:58.928998+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:58.928998+0000","last_peered":"2024-09-15T21:25:58.928998+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:58.928998+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32338403,"num_objects":7896,"num_object_clones":0,"num_object_copies":15792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7896,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10607,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1666},{"shards":"2","objects":3806}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12786","reported_seq":21245,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:58.854989+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:58.854989+0000","last_peered":"2024-09-15T21:25:58.854989+0000","last_clean":"2024-09-15T21:25:58.854989+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:58.854989+0000","last_undegraded":"2024-09-15T21:25:58.854989+0000","last_fullsized":"2024-09-15T21:25:58.854989+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31842304,"num_objects":7774,"num_object_clones":0,"num_object_copies":15548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10479,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12860","reported_seq":12993,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.631863+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:02.631863+0000","last_peered":"2024-09-15T21:26:02.631863+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:02.631863+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32055296,"num_objects":7826,"num_object_clones":0,"num_object_copies":15652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9804,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3732},{"shards":"0,2","objects":1578}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12895","reported_seq":13028,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.497723+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:02.497723+0000","last_peered":"2024-09-15T21:26:02.497723+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:02.497723+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31518720,"num_objects":7695,"num_object_clones":0,"num_object_copies":15390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9804,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3556},{"shards":"0,2","objects":1543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12732","reported_seq":18009,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:26:02.694678+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:26:02.694678+0000","last_peered":"2024-09-15T21:26:02.694678+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:02.694678+0000","last_undegraded":"2024-09-15T21:26:02.694678+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31358976,"num_objects":7656,"num_object_clones":0,"num_object_copies":15312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1514,"num_objects_unfound":0,"num_objects_dirty":7656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2570,"num_bytes_recovered":10518528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6142},{"shards":"0,2","objects":1514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13015","reported_seq":13154,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.614687+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:02.614687+0000","last_peered":"2024-09-15T21:26:02.614687+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:02.614687+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32158662,"num_objects":7853,"num_object_clones":0,"num_object_copies":15706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7853,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9719,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":1571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":512434934,"num_objects":125116,"num_object_clones":0,"num_object_copies":250232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1514,"num_objects_unfound":0,"num_objects_dirty":125116,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":165339,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6735,"num_bytes_recovered":27570176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1156108288,"data_stored":1155959237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160846,"ondisk_log_size":160846,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695020,"kb_used_data":384284,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93676820,"statfs":{"total":96636764160,"available":95925063680,"internally_reserved":0,"allocated":393506816,"data_stored":393231169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820228,"kb_used_data":441512,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93551612,"statfs":{"total":96636764160,"available":95796850688,"internally_reserved":0,"allocated":452108288,"data_stored":451832636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616984,"kb_used_data":306248,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93754856,"statfs":{"total":96636764160,"available":96004972544,"internally_reserved":0,"allocated":313597952,"data_stored":313285711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":450404352,"data_stored":450368222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313110528,"data_stored":313037809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":392593408,"data_stored":392553206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:02.967 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:03.507 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:03.509 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:03.542 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":245,"stamp":"2024-09-15T21:26:02.782404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":512434953,"num_objects":125118,"num_object_clones":0,"num_object_copies":250236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1514,"num_objects_unfound":0,"num_objects_dirty":125118,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":165341,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6736,"num_bytes_recovered":27570195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160849,"ondisk_log_size":160849,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2132232,"kb_used_data":1132044,"kb_used_omap":12,"kb_used_meta":1000115,"kb_avail":280983288,"statfs":{"total":289910292480,"available":287726886912,"internally_reserved":0,"allocated":1159213056,"data_stored":1158349516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1024117763},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,7,5,15,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24743936,"num_objects":-6041,"num_object_clones":0,"num_object_copies":-12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-311,"num_objects_unfound":0,"num_objects_dirty":-6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6041,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":1273856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003326"},"pg_stats":[{"pgid":"2.f","version":"45'13114","reported_seq":13189,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.689127+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:02.689127+0000","last_peered":"2024-09-15T21:26:02.689127+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:02.689127+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13114,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3906},{"shards":"0,2","objects":1594}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12716","reported_seq":12822,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:58.920581+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:25:58.920581+0000","last_peered":"2024-09-15T21:25:58.920581+0000","last_clean":"2024-09-15T21:25:58.920581+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:25:58.920581+0000","last_undegraded":"2024-09-15T21:25:58.920581+0000","last_fullsized":"2024-09-15T21:25:58.920581+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32596934,"num_objects":7960,"num_object_clones":0,"num_object_copies":15920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7960,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10997,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13059","reported_seq":13186,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:58.855693+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:25:58.855693+0000","last_peered":"2024-09-15T21:25:58.855693+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:25:58.855693+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32956440,"num_objects":8047,"num_object_clones":0,"num_object_copies":16094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8047,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11159,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1763},{"shards":"2","objects":3780}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12591","reported_seq":12662,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:58.857793+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:25:58.857793+0000","last_peered":"2024-09-15T21:25:58.857793+0000","last_clean":"2024-09-15T21:25:58.857793+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:25:58.857793+0000","last_undegraded":"2024-09-15T21:25:58.857793+0000","last_fullsized":"2024-09-15T21:25:58.857793+0000","mapping_epoch":35,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31633408,"num_objects":7723,"num_object_clones":0,"num_object_copies":15446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12591,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12758","reported_seq":12891,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:02.635022+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:02.635022+0000","last_peered":"2024-09-15T21:26:02.635022+0000","last_clean":"2024-09-15T21:26:02.635022+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:02.635022+0000","last_undegraded":"2024-09-15T21:26:02.635022+0000","last_fullsized":"2024-09-15T21:26:02.635022+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31732195,"num_objects":7748,"num_object_clones":0,"num_object_copies":15496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9213,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12890","reported_seq":13025,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.630131+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:02.630131+0000","last_peered":"2024-09-15T21:26:02.630131+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:02.630131+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31724003,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9566,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3670},{"shards":"0,2","objects":1506}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12926","reported_seq":13062,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.626914+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:02.626914+0000","last_peered":"2024-09-15T21:26:02.626914+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:02.626914+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31720390,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9237,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3585},{"shards":"0,2","objects":1573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13055","reported_seq":13184,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:02.589573+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:02.589573+0000","last_peered":"2024-09-15T21:26:02.589573+0000","last_clean":"2024-09-15T21:26:02.589573+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:02.589573+0000","last_undegraded":"2024-09-15T21:26:02.589573+0000","last_fullsized":"2024-09-15T21:26:02.589573+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32002048,"num_objects":7813,"num_object_clones":0,"num_object_copies":15626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9827,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12696","reported_seq":12823,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:58.892992+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:25:58.892992+0000","last_peered":"2024-09-15T21:25:58.892992+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:25:58.892992+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32018915,"num_objects":7818,"num_object_clones":0,"num_object_copies":15636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7818,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10628,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1649},{"shards":"2","objects":3731}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12705","reported_seq":12836,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.628483+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:02.628483+0000","last_peered":"2024-09-15T21:26:02.628483+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:02.628483+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31854592,"num_objects":7777,"num_object_clones":0,"num_object_copies":15554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9378,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3757},{"shards":"0,2","objects":1560}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12748","reported_seq":12867,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:25:58.928998+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:25:58.928998+0000","last_peered":"2024-09-15T21:25:58.928998+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:25:58.928998+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32338403,"num_objects":7896,"num_object_clones":0,"num_object_copies":15792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7896,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10607,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1666},{"shards":"2","objects":3806}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12786","reported_seq":21245,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:25:58.854989+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:25:58.854989+0000","last_peered":"2024-09-15T21:25:58.854989+0000","last_clean":"2024-09-15T21:25:58.854989+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:25:58.854989+0000","last_undegraded":"2024-09-15T21:25:58.854989+0000","last_fullsized":"2024-09-15T21:25:58.854989+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31842304,"num_objects":7774,"num_object_clones":0,"num_object_copies":15548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10479,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12860","reported_seq":12993,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.631863+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:02.631863+0000","last_peered":"2024-09-15T21:26:02.631863+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:02.631863+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32055296,"num_objects":7826,"num_object_clones":0,"num_object_copies":15652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9804,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3732},{"shards":"0,2","objects":1578}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12895","reported_seq":13028,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.497723+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:02.497723+0000","last_peered":"2024-09-15T21:26:02.497723+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:02.497723+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31518720,"num_objects":7695,"num_object_clones":0,"num_object_copies":15390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9804,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3556},{"shards":"0,2","objects":1543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12732","reported_seq":18009,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:26:02.694678+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:26:02.694678+0000","last_peered":"2024-09-15T21:26:02.694678+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:02.694678+0000","last_undegraded":"2024-09-15T21:26:02.694678+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31358976,"num_objects":7656,"num_object_clones":0,"num_object_copies":15312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1514,"num_objects_unfound":0,"num_objects_dirty":7656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2570,"num_bytes_recovered":10518528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6142},{"shards":"0,2","objects":1514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13015","reported_seq":13154,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.614687+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:02.614687+0000","last_peered":"2024-09-15T21:26:02.614687+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:02.614687+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32158662,"num_objects":7853,"num_object_clones":0,"num_object_copies":15706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7853,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9719,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":1571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":512434934,"num_objects":125116,"num_object_clones":0,"num_object_copies":250232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1514,"num_objects_unfound":0,"num_objects_dirty":125116,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":165339,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6735,"num_bytes_recovered":27570176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1156108288,"data_stored":1155959237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160846,"ondisk_log_size":160846,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":695020,"kb_used_data":384284,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93676820,"statfs":{"total":96636764160,"available":95925063680,"internally_reserved":0,"allocated":393506816,"data_stored":393231169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820228,"kb_used_data":441512,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93551612,"statfs":{"total":96636764160,"available":95796850688,"internally_reserved":0,"allocated":452108288,"data_stored":451832636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616984,"kb_used_data":306248,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93754856,"statfs":{"total":96636764160,"available":96004972544,"internally_reserved":0,"allocated":313597952,"data_stored":313285711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318172847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":450404352,"data_stored":450368222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313110528,"data_stored":313037809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":392593408,"data_stored":392553206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:03.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:03.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13114", 'reported_seq': 13189, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:02.689127+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:26:02.689127+0000', 'last_peered': '2024-09-15T21:26:02.689127+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:26:02.689127+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32923648, 'num_objects': 8038, 'num_object_clones': 0, 'num_object_copies': 16076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8038, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13114, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3906}, {'shards': '0,2', 'objects': 1594}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:03.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:26:03.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'13059", 'reported_seq': 13186, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:58.855693+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:25:58.855693+0000', 'last_peered': '2024-09-15T21:25:58.855693+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:25:58.855693+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32956440, 'num_objects': 8047, 'num_object_clones': 0, 'num_object_copies': 16094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8047, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11159, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1763}, {'shards': '2', 'objects': 3780}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:03.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:03.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'12890", 'reported_seq': 13025, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:02.630131+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:26:02.630131+0000', 'last_peered': '2024-09-15T21:26:02.630131+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:26:02.630131+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31724003, 'num_objects': 7746, 'num_object_clones': 0, 'num_object_copies': 15492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7746, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9566, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3670}, {'shards': '0,2', 'objects': 1506}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:03.544 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:03.544 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'12926", 'reported_seq': 13062, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:02.626914+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:26:02.626914+0000', 'last_peered': '2024-09-15T21:26:02.626914+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:26:02.626914+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31720390, 'num_objects': 7746, 'num_object_clones': 0, 'num_object_copies': 15492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7746, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9237, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3585}, {'shards': '0,2', 'objects': 1573}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:03.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:26:03.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12696", 'reported_seq': 12823, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:58.892992+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:25:58.892992+0000', 'last_peered': '2024-09-15T21:25:58.892992+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:25:58.892992+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32018915, 'num_objects': 7818, 'num_object_clones': 0, 'num_object_copies': 15636, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7818, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10628, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1649}, {'shards': '2', 'objects': 3731}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:03.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:03.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12705", 'reported_seq': 12836, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:02.628483+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:26:02.628483+0000', 'last_peered': '2024-09-15T21:26:02.628483+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:26:02.628483+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31854592, 'num_objects': 7777, 'num_object_clones': 0, 'num_object_copies': 15554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7777, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9378, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3757}, {'shards': '0,2', 'objects': 1560}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:03.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:03.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12748", 'reported_seq': 12867, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:25:58.928998+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:25:58.928998+0000', 'last_peered': '2024-09-15T21:25:58.928998+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:25:58.928998+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32338403, 'num_objects': 7896, 'num_object_clones': 0, 'num_object_copies': 15792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7896, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10607, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1666}, {'shards': '2', 'objects': 3806}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:03.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:03.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12860", 'reported_seq': 12993, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:02.631863+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:26:02.631863+0000', 'last_peered': '2024-09-15T21:26:02.631863+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:26:02.631863+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32055296, 'num_objects': 7826, 'num_object_clones': 0, 'num_object_copies': 15652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9804, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3732}, {'shards': '0,2', 'objects': 1578}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:03.545 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:26:03.545 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'12895", 'reported_seq': 13028, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:02.497723+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:26:02.497723+0000', 'last_peered': '2024-09-15T21:26:02.497723+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:26:02.497723+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31518720, 'num_objects': 7695, 'num_object_clones': 0, 'num_object_copies': 15390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9804, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3556}, {'shards': '0,2', 'objects': 1543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:03.546 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:03.546 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12732", 'reported_seq': 18009, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-15T21:26:02.694678+0000', 'last_change': '2024-09-15T21:23:34.754815+0000', 'last_active': '2024-09-15T21:26:02.694678+0000', 'last_peered': '2024-09-15T21:26:02.694678+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:26:02.694678+0000', 'last_undegraded': '2024-09-15T21:26:02.694678+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31358976, 'num_objects': 7656, 'num_object_clones': 0, 'num_object_copies': 15312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1514, 'num_objects_unfound': 0, 'num_objects_dirty': 7656, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9216, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2570, 'num_bytes_recovered': 10518528, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6142}, {'shards': '0,2', 'objects': 1514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:03.546 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:26:03.546 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13015", 'reported_seq': 13154, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:02.614687+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:26:02.614687+0000', 'last_peered': '2024-09-15T21:26:02.614687+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:26:02.614687+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32158662, 'num_objects': 7853, 'num_object_clones': 0, 'num_object_copies': 15706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7853, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9719, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3706}, {'shards': '0,2', 'objects': 1571}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:03.546 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:04.284 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:04.285 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/250236 objects degraded (16.474%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":416,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":125118,"data_bytes":512434953,"bytes_used":2183405568,"bytes_avail":287726886912,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":250236,"degraded_ratio":0.164744481209738,"misplaced_objects":1514,"misplaced_total":250236,"misplaced_ratio":0.0060502885276299163,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":106125,"recovering_keys_per_sec":0,"num_objects_recovered":311,"num_bytes_recovered":1273856,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":503},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:04.308 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:04.308 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:04.888 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:04.888 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:04.909 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-15T21:26:04.782876+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":508891913,"num_objects":124253,"num_object_clones":0,"num_object_copies":248506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1514,"num_objects_unfound":0,"num_objects_dirty":124253,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":166206,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6736,"num_bytes_recovered":27570195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2120040,"kb_used_data":1126764,"kb_used_omap":12,"kb_used_meta":993203,"kb_avail":280995480,"statfs":{"total":289910292480,"available":287739371520,"internally_reserved":0,"allocated":1153806336,"data_stored":1152942796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1017039875},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,7,9,9,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22405120,"num_objects":-5470,"num_object_clones":0,"num_object_copies":-10940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-199,"num_objects_unfound":0,"num_objects_dirty":-5470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5470,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":815104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003086"},"pg_stats":[{"pgid":"2.f","version":"45'13114","reported_seq":13189,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.689127+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:02.689127+0000","last_peered":"2024-09-15T21:26:02.689127+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:02.689127+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13114,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3906},{"shards":"0,2","objects":1594}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12864","reported_seq":12970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:03.896774+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:03.896774+0000","last_peered":"2024-09-15T21:26:03.896774+0000","last_clean":"2024-09-15T21:26:03.896774+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:03.896774+0000","last_undegraded":"2024-09-15T21:26:03.896774+0000","last_fullsized":"2024-09-15T21:26:03.896774+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31990726,"num_objects":7812,"num_object_clones":0,"num_object_copies":15624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7812,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11145,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13246","reported_seq":13372,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:03.891055+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:03.891055+0000","last_peered":"2024-09-15T21:26:03.891055+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:03.891055+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32190488,"num_objects":7860,"num_object_clones":0,"num_object_copies":15720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11346,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1576},{"shards":"2","objects":3593}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12714","reported_seq":12784,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:03.877033+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:03.877033+0000","last_peered":"2024-09-15T21:26:03.877033+0000","last_clean":"2024-09-15T21:26:03.877033+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:03.877033+0000","last_undegraded":"2024-09-15T21:26:03.877033+0000","last_fullsized":"2024-09-15T21:26:03.877033+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12714,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12758","reported_seq":12891,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:02.635022+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:02.635022+0000","last_peered":"2024-09-15T21:26:02.635022+0000","last_clean":"2024-09-15T21:26:02.635022+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:02.635022+0000","last_undegraded":"2024-09-15T21:26:02.635022+0000","last_fullsized":"2024-09-15T21:26:02.635022+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31732195,"num_objects":7748,"num_object_clones":0,"num_object_copies":15496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9213,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12890","reported_seq":13025,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.630131+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:02.630131+0000","last_peered":"2024-09-15T21:26:02.630131+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:02.630131+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31724003,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9566,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3670},{"shards":"0,2","objects":1506}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12926","reported_seq":13062,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.626914+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:02.626914+0000","last_peered":"2024-09-15T21:26:02.626914+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:02.626914+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31720390,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9237,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3585},{"shards":"0,2","objects":1573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13055","reported_seq":13184,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:02.589573+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:02.589573+0000","last_peered":"2024-09-15T21:26:02.589573+0000","last_clean":"2024-09-15T21:26:02.589573+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:02.589573+0000","last_undegraded":"2024-09-15T21:26:02.589573+0000","last_fullsized":"2024-09-15T21:26:02.589573+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32002048,"num_objects":7813,"num_object_clones":0,"num_object_copies":15626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9827,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12842","reported_seq":12969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:03.855439+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:03.855439+0000","last_peered":"2024-09-15T21:26:03.855439+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:03.855439+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31420899,"num_objects":7672,"num_object_clones":0,"num_object_copies":15344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10774,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1503},{"shards":"2","objects":3585}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12705","reported_seq":12836,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.628483+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:02.628483+0000","last_peered":"2024-09-15T21:26:02.628483+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:02.628483+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31854592,"num_objects":7777,"num_object_clones":0,"num_object_copies":15554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9378,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3757},{"shards":"0,2","objects":1560}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12879","reported_seq":12998,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:03.917246+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:03.917246+0000","last_peered":"2024-09-15T21:26:03.917246+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:03.917246+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801827,"num_objects":7765,"num_object_clones":0,"num_object_copies":15530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7765,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10738,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1535},{"shards":"2","objects":3675}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12916","reported_seq":21374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:03.932990+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:03.932990+0000","last_peered":"2024-09-15T21:26:03.932990+0000","last_clean":"2024-09-15T21:26:03.932990+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:03.932990+0000","last_undegraded":"2024-09-15T21:26:03.932990+0000","last_fullsized":"2024-09-15T21:26:03.932990+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31309824,"num_objects":7644,"num_object_clones":0,"num_object_copies":15288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10609,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12860","reported_seq":12993,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.631863+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:02.631863+0000","last_peered":"2024-09-15T21:26:02.631863+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:02.631863+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32055296,"num_objects":7826,"num_object_clones":0,"num_object_copies":15652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9804,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3732},{"shards":"0,2","objects":1578}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12895","reported_seq":13028,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.497723+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:02.497723+0000","last_peered":"2024-09-15T21:26:02.497723+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:02.497723+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31518720,"num_objects":7695,"num_object_clones":0,"num_object_copies":15390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9804,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3556},{"shards":"0,2","objects":1543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12732","reported_seq":18009,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:26:02.694678+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:26:02.694678+0000","last_peered":"2024-09-15T21:26:02.694678+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:02.694678+0000","last_undegraded":"2024-09-15T21:26:02.694678+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31358976,"num_objects":7656,"num_object_clones":0,"num_object_copies":15312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1514,"num_objects_unfound":0,"num_objects_dirty":7656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2570,"num_bytes_recovered":10518528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6142},{"shards":"0,2","objects":1514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13015","reported_seq":13154,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.614687+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:02.614687+0000","last_peered":"2024-09-15T21:26:02.614687+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:02.614687+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32158662,"num_objects":7853,"num_object_clones":0,"num_object_copies":15706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7853,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9719,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":1571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":508891894,"num_objects":124251,"num_object_clones":0,"num_object_copies":248502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1514,"num_objects_unfound":0,"num_objects_dirty":124251,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":166204,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6735,"num_bytes_recovered":27570176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1150631936,"data_stored":1150482885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678040,"kb_used_data":379784,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93693800,"statfs":{"total":96636764160,"available":95942451200,"internally_reserved":0,"allocated":388898816,"data_stored":388623169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820228,"kb_used_data":441512,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93551612,"statfs":{"total":96636764160,"available":95796850688,"internally_reserved":0,"allocated":452108288,"data_stored":451832636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621772,"kb_used_data":305468,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93750068,"statfs":{"total":96636764160,"available":96000069632,"internally_reserved":0,"allocated":312799232,"data_stored":312486991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323874479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,5,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":450404352,"data_stored":450368222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312311808,"data_stored":312239089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387915776,"data_stored":387875574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:07.912 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:08.460 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:08.462 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:08.479 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-09-15T21:26:06.783353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":508891913,"num_objects":124253,"num_object_clones":0,"num_object_copies":248506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1514,"num_objects_unfound":0,"num_objects_dirty":124253,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":166206,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6736,"num_bytes_recovered":27570195,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2120040,"kb_used_data":1126764,"kb_used_omap":12,"kb_used_meta":993203,"kb_avail":280995480,"statfs":{"total":289910292480,"available":287739371520,"internally_reserved":0,"allocated":1153806336,"data_stored":1152942796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1017039875},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,7,9,9,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18833408,"num_objects":-4598,"num_object_clones":0,"num_object_copies":-9196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-199,"num_objects_unfound":0,"num_objects_dirty":-4598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4598,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":815104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003098"},"pg_stats":[{"pgid":"2.f","version":"45'13114","reported_seq":13189,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.689127+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:02.689127+0000","last_peered":"2024-09-15T21:26:02.689127+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:02.689127+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32923648,"num_objects":8038,"num_object_clones":0,"num_object_copies":16076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13114,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3906},{"shards":"0,2","objects":1594}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12864","reported_seq":12970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:03.896774+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:03.896774+0000","last_peered":"2024-09-15T21:26:03.896774+0000","last_clean":"2024-09-15T21:26:03.896774+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:03.896774+0000","last_undegraded":"2024-09-15T21:26:03.896774+0000","last_fullsized":"2024-09-15T21:26:03.896774+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31990726,"num_objects":7812,"num_object_clones":0,"num_object_copies":15624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7812,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11145,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13246","reported_seq":13372,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:03.891055+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:03.891055+0000","last_peered":"2024-09-15T21:26:03.891055+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:03.891055+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32190488,"num_objects":7860,"num_object_clones":0,"num_object_copies":15720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11346,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1576},{"shards":"2","objects":3593}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12714","reported_seq":12784,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:03.877033+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:03.877033+0000","last_peered":"2024-09-15T21:26:03.877033+0000","last_clean":"2024-09-15T21:26:03.877033+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:03.877033+0000","last_undegraded":"2024-09-15T21:26:03.877033+0000","last_fullsized":"2024-09-15T21:26:03.877033+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12714,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12758","reported_seq":12891,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:02.635022+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:02.635022+0000","last_peered":"2024-09-15T21:26:02.635022+0000","last_clean":"2024-09-15T21:26:02.635022+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:02.635022+0000","last_undegraded":"2024-09-15T21:26:02.635022+0000","last_fullsized":"2024-09-15T21:26:02.635022+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31732195,"num_objects":7748,"num_object_clones":0,"num_object_copies":15496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9213,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12890","reported_seq":13025,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.630131+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:02.630131+0000","last_peered":"2024-09-15T21:26:02.630131+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:02.630131+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31724003,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9566,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3670},{"shards":"0,2","objects":1506}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12926","reported_seq":13062,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.626914+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:02.626914+0000","last_peered":"2024-09-15T21:26:02.626914+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:02.626914+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31720390,"num_objects":7746,"num_object_clones":0,"num_object_copies":15492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9237,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3585},{"shards":"0,2","objects":1573}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13055","reported_seq":13184,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:02.589573+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:02.589573+0000","last_peered":"2024-09-15T21:26:02.589573+0000","last_clean":"2024-09-15T21:26:02.589573+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:02.589573+0000","last_undegraded":"2024-09-15T21:26:02.589573+0000","last_fullsized":"2024-09-15T21:26:02.589573+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32002048,"num_objects":7813,"num_object_clones":0,"num_object_copies":15626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9827,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12842","reported_seq":12969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:03.855439+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:03.855439+0000","last_peered":"2024-09-15T21:26:03.855439+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:03.855439+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31420899,"num_objects":7672,"num_object_clones":0,"num_object_copies":15344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10774,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1503},{"shards":"2","objects":3585}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12705","reported_seq":12836,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.628483+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:02.628483+0000","last_peered":"2024-09-15T21:26:02.628483+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:02.628483+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31854592,"num_objects":7777,"num_object_clones":0,"num_object_copies":15554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9378,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3757},{"shards":"0,2","objects":1560}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12879","reported_seq":12998,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:03.917246+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:03.917246+0000","last_peered":"2024-09-15T21:26:03.917246+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:03.917246+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801827,"num_objects":7765,"num_object_clones":0,"num_object_copies":15530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7765,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10738,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1535},{"shards":"2","objects":3675}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12916","reported_seq":21374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:03.932990+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:03.932990+0000","last_peered":"2024-09-15T21:26:03.932990+0000","last_clean":"2024-09-15T21:26:03.932990+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:03.932990+0000","last_undegraded":"2024-09-15T21:26:03.932990+0000","last_fullsized":"2024-09-15T21:26:03.932990+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31309824,"num_objects":7644,"num_object_clones":0,"num_object_copies":15288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10609,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12860","reported_seq":12993,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.631863+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:02.631863+0000","last_peered":"2024-09-15T21:26:02.631863+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:02.631863+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32055296,"num_objects":7826,"num_object_clones":0,"num_object_copies":15652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9804,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3732},{"shards":"0,2","objects":1578}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12895","reported_seq":13028,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.497723+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:02.497723+0000","last_peered":"2024-09-15T21:26:02.497723+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:02.497723+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31518720,"num_objects":7695,"num_object_clones":0,"num_object_copies":15390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9804,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3556},{"shards":"0,2","objects":1543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12732","reported_seq":18009,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-15T21:26:02.694678+0000","last_change":"2024-09-15T21:23:34.754815+0000","last_active":"2024-09-15T21:26:02.694678+0000","last_peered":"2024-09-15T21:26:02.694678+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:02.694678+0000","last_undegraded":"2024-09-15T21:26:02.694678+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31358976,"num_objects":7656,"num_object_clones":0,"num_object_copies":15312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1514,"num_objects_unfound":0,"num_objects_dirty":7656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9216,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2570,"num_bytes_recovered":10518528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6142},{"shards":"0,2","objects":1514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13015","reported_seq":13154,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:02.614687+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:02.614687+0000","last_peered":"2024-09-15T21:26:02.614687+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:02.614687+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32158662,"num_objects":7853,"num_object_clones":0,"num_object_copies":15706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7853,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9719,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3706},{"shards":"0,2","objects":1571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":508891894,"num_objects":124251,"num_object_clones":0,"num_object_copies":248502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":1514,"num_objects_unfound":0,"num_objects_dirty":124251,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":166204,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6735,"num_bytes_recovered":27570176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1150631936,"data_stored":1150482885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160711,"ondisk_log_size":160711,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678040,"kb_used_data":379784,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93693800,"statfs":{"total":96636764160,"available":95942451200,"internally_reserved":0,"allocated":388898816,"data_stored":388623169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820228,"kb_used_data":441512,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93551612,"statfs":{"total":96636764160,"available":95796850688,"internally_reserved":0,"allocated":452108288,"data_stored":451832636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621772,"kb_used_data":305468,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93750068,"statfs":{"total":96636764160,"available":96000069632,"internally_reserved":0,"allocated":312799232,"data_stored":312486991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323874479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,5,3,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":450404352,"data_stored":450368222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312311808,"data_stored":312239089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387915776,"data_stored":387875574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:08.481 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:09.022 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:09.025 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:09.047 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-15T21:26:08.783786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":503202569,"num_objects":122864,"num_object_clones":0,"num_object_copies":245728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41232,"num_objects_misplaced":1411,"num_objects_unfound":0,"num_objects_dirty":122864,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":167595,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6832,"num_bytes_recovered":27963411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160903,"ondisk_log_size":160903,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2125108,"kb_used_data":1119352,"kb_used_omap":12,"kb_used_meta":1005683,"kb_avail":280990412,"statfs":{"total":289910292480,"available":287734181888,"internally_reserved":0,"allocated":1146216448,"data_stored":1145352908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1029819395},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,0,9,4,8,20,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24522752,"num_objects":-5987,"num_object_clones":0,"num_object_copies":-11974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":-302,"num_objects_unfound":0,"num_objects_dirty":-5987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5987,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":1208320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002853"},"pg_stats":[{"pgid":"2.f","version":"45'13258","reported_seq":13334,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.605021+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:07.605021+0000","last_peered":"2024-09-15T21:26:07.605021+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:07.605021+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32333824,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13258,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3762},{"shards":"0,2","objects":1450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12864","reported_seq":12970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:03.896774+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:03.896774+0000","last_peered":"2024-09-15T21:26:03.896774+0000","last_clean":"2024-09-15T21:26:03.896774+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:03.896774+0000","last_undegraded":"2024-09-15T21:26:03.896774+0000","last_fullsized":"2024-09-15T21:26:03.896774+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31990726,"num_objects":7812,"num_object_clones":0,"num_object_copies":15624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7812,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11145,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13246","reported_seq":13372,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:03.891055+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:03.891055+0000","last_peered":"2024-09-15T21:26:03.891055+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:03.891055+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32190488,"num_objects":7860,"num_object_clones":0,"num_object_copies":15720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11346,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1576},{"shards":"2","objects":3593}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12714","reported_seq":12784,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:03.877033+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:03.877033+0000","last_peered":"2024-09-15T21:26:03.877033+0000","last_clean":"2024-09-15T21:26:03.877033+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:03.877033+0000","last_undegraded":"2024-09-15T21:26:03.877033+0000","last_fullsized":"2024-09-15T21:26:03.877033+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12714,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12878","reported_seq":13011,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:07.670152+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:07.670152+0000","last_peered":"2024-09-15T21:26:07.670152+0000","last_clean":"2024-09-15T21:26:07.670152+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:07.670152+0000","last_undegraded":"2024-09-15T21:26:07.670152+0000","last_fullsized":"2024-09-15T21:26:07.670152+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31240675,"num_objects":7628,"num_object_clones":0,"num_object_copies":15256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9333,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13017","reported_seq":13154,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.686998+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:07.686998+0000","last_peered":"2024-09-15T21:26:07.686998+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:07.686998+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31203811,"num_objects":7619,"num_object_clones":0,"num_object_copies":15238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7619,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9693,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3543},{"shards":"0,2","objects":1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13073","reported_seq":13210,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.673315+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:07.673315+0000","last_peered":"2024-09-15T21:26:07.673315+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:07.673315+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31118278,"num_objects":7599,"num_object_clones":0,"num_object_copies":15198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9384,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3438},{"shards":"0,2","objects":1426}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13198","reported_seq":13327,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:07.636137+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:07.636137+0000","last_peered":"2024-09-15T21:26:07.636137+0000","last_clean":"2024-09-15T21:26:07.636137+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:07.636137+0000","last_undegraded":"2024-09-15T21:26:07.636137+0000","last_fullsized":"2024-09-15T21:26:07.636137+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31416320,"num_objects":7670,"num_object_clones":0,"num_object_copies":15340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9970,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12842","reported_seq":12969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:03.855439+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:03.855439+0000","last_peered":"2024-09-15T21:26:03.855439+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:03.855439+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31420899,"num_objects":7672,"num_object_clones":0,"num_object_copies":15344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10774,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1503},{"shards":"2","objects":3585}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12852","reported_seq":12983,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.547715+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:07.547715+0000","last_peered":"2024-09-15T21:26:07.547715+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:07.547715+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31252480,"num_objects":7630,"num_object_clones":0,"num_object_copies":15260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9525,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3610},{"shards":"0,2","objects":1413}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12879","reported_seq":12998,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:03.917246+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:03.917246+0000","last_peered":"2024-09-15T21:26:03.917246+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:03.917246+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801827,"num_objects":7765,"num_object_clones":0,"num_object_copies":15530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7765,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10738,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1535},{"shards":"2","objects":3675}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12916","reported_seq":21374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:03.932990+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:03.932990+0000","last_peered":"2024-09-15T21:26:03.932990+0000","last_clean":"2024-09-15T21:26:03.932990+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:03.932990+0000","last_undegraded":"2024-09-15T21:26:03.932990+0000","last_fullsized":"2024-09-15T21:26:03.932990+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31309824,"num_objects":7644,"num_object_clones":0,"num_object_copies":15288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10609,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12988","reported_seq":13122,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.669275+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:07.669275+0000","last_peered":"2024-09-15T21:26:07.669275+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:07.669275+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31531008,"num_objects":7698,"num_object_clones":0,"num_object_copies":15396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9932,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3604},{"shards":"0,2","objects":1450}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13044","reported_seq":13177,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.628062+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:07.628062+0000","last_peered":"2024-09-15T21:26:07.628062+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:07.628062+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30908416,"num_objects":7546,"num_object_clones":0,"num_object_copies":15092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9953,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3407},{"shards":"0,2","objects":1394}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12866","reported_seq":18334,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.714548+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:07.714548+0000","last_peered":"2024-09-15T21:26:07.714548+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:07.714548+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30810112,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":1411,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9350,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":10911744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6104},{"shards":"0,2","objects":1411}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13165","reported_seq":13304,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.650267+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:07.650267+0000","last_peered":"2024-09-15T21:26:07.650267+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:07.650267+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31544262,"num_objects":7703,"num_object_clones":0,"num_object_copies":15406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7703,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9869,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3556},{"shards":"0,2","objects":1421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":503202550,"num_objects":122862,"num_object_clones":0,"num_object_copies":245724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41232,"num_objects_misplaced":1411,"num_objects_unfound":0,"num_objects_dirty":122862,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":167593,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6831,"num_bytes_recovered":27963392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1142984704,"data_stored":1142835653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160900,"ondisk_log_size":160900,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678040,"kb_used_data":379784,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93693800,"statfs":{"total":96636764160,"available":95942451200,"internally_reserved":0,"allocated":388898816,"data_stored":388623169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":817744,"kb_used_data":434868,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93554096,"statfs":{"total":96636764160,"available":95799394304,"internally_reserved":0,"allocated":445304832,"data_stored":445029180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,0,5,0,3,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629324,"kb_used_data":304700,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93742516,"statfs":{"total":96636764160,"available":95992336384,"internally_reserved":0,"allocated":312012800,"data_stored":311700559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,3,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443555840,"data_stored":443519710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311513088,"data_stored":311440369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387915776,"data_stored":387875574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:09.048 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:09.048 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13258", 'reported_seq': 13334, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:07.605021+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:26:07.605021+0000', 'last_peered': '2024-09-15T21:26:07.605021+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:26:07.605021+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32333824, 'num_objects': 7894, 'num_object_clones': 0, 'num_object_copies': 15788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7894, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13258, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3762}, {'shards': '0,2', 'objects': 1450}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:09.049 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:26:09.049 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'13246", 'reported_seq': 13372, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:03.891055+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:26:03.891055+0000', 'last_peered': '2024-09-15T21:26:03.891055+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:26:03.891055+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32190488, 'num_objects': 7860, 'num_object_clones': 0, 'num_object_copies': 15720, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7860, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11346, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1576}, {'shards': '2', 'objects': 3593}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:09.049 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:09.049 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13017", 'reported_seq': 13154, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:07.686998+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:26:07.686998+0000', 'last_peered': '2024-09-15T21:26:07.686998+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:26:07.686998+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31203811, 'num_objects': 7619, 'num_object_clones': 0, 'num_object_copies': 15238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7619, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9693, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3543}, {'shards': '0,2', 'objects': 1379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:09.049 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:09.049 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13073", 'reported_seq': 13210, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:07.673315+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:26:07.673315+0000', 'last_peered': '2024-09-15T21:26:07.673315+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:26:07.673315+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31118278, 'num_objects': 7599, 'num_object_clones': 0, 'num_object_copies': 15198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7599, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9384, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3438}, {'shards': '0,2', 'objects': 1426}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:09.049 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:26:09.049 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12842", 'reported_seq': 12969, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:03.855439+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:26:03.855439+0000', 'last_peered': '2024-09-15T21:26:03.855439+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:26:03.855439+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31420899, 'num_objects': 7672, 'num_object_clones': 0, 'num_object_copies': 15344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7672, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10774, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1503}, {'shards': '2', 'objects': 3585}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:09.049 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:09.049 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12852", 'reported_seq': 12983, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:07.547715+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:26:07.547715+0000', 'last_peered': '2024-09-15T21:26:07.547715+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:26:07.547715+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31252480, 'num_objects': 7630, 'num_object_clones': 0, 'num_object_copies': 15260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7630, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9525, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3610}, {'shards': '0,2', 'objects': 1413}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:09.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:09.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'12879", 'reported_seq': 12998, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:03.917246+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:26:03.917246+0000', 'last_peered': '2024-09-15T21:26:03.917246+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:26:03.917246+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31801827, 'num_objects': 7765, 'num_object_clones': 0, 'num_object_copies': 15530, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7765, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10738, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1535}, {'shards': '2', 'objects': 3675}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:09.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:09.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'12988", 'reported_seq': 13122, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:07.669275+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:26:07.669275+0000', 'last_peered': '2024-09-15T21:26:07.669275+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:26:07.669275+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31531008, 'num_objects': 7698, 'num_object_clones': 0, 'num_object_copies': 15396, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7698, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9932, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3604}, {'shards': '0,2', 'objects': 1450}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:09.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:26:09.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13044", 'reported_seq': 13177, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:07.628062+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:26:07.628062+0000', 'last_peered': '2024-09-15T21:26:07.628062+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:26:07.628062+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30908416, 'num_objects': 7546, 'num_object_clones': 0, 'num_object_copies': 15092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7546, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9953, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3407}, {'shards': '0,2', 'objects': 1394}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:09.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:09.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'12866", 'reported_seq': 18334, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:07.714548+0000', 'last_change': '2024-09-15T21:26:05.146366+0000', 'last_active': '2024-09-15T21:26:07.714548+0000', 'last_peered': '2024-09-15T21:26:07.714548+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:26:07.714548+0000', 'last_undegraded': '2024-09-15T21:26:05.134624+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30810112, 'num_objects': 7522, 'num_object_clones': 0, 'num_object_copies': 15044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 7, 'num_objects_misplaced': 1411, 'num_objects_unfound': 0, 'num_objects_dirty': 7522, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9350, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2666, 'num_bytes_recovered': 10911744, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6104}, {'shards': '0,2', 'objects': 1411}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:09.050 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:26:09.050 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13165", 'reported_seq': 13304, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:07.650267+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:26:07.650267+0000', 'last_peered': '2024-09-15T21:26:07.650267+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:26:07.650267+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31544262, 'num_objects': 7703, 'num_object_clones': 0, 'num_object_copies': 15406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7703, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9869, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3556}, {'shards': '0,2', 'objects': 1421}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:09.051 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:09.768 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41232/245728 objects degraded (16.780%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":421,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":122864,"data_bytes":503202569,"bytes_used":2176110592,"bytes_avail":287734181888,"bytes_total":289910292480,"degraded_objects":41232,"degraded_total":245728,"degraded_ratio":0.16779528584451101,"misplaced_objects":1411,"misplaced_total":245728,"misplaced_ratio":0.0057421213699700479,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":100669,"recovering_keys_per_sec":0,"num_objects_recovered":295,"num_bytes_recovered":1208320,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":498},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:09.788 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:09.788 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:10.337 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:10.339 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:10.366 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-15T21:26:08.783786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":503202569,"num_objects":122864,"num_object_clones":0,"num_object_copies":245728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41232,"num_objects_misplaced":1411,"num_objects_unfound":0,"num_objects_dirty":122864,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":167595,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6832,"num_bytes_recovered":27963411,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160903,"ondisk_log_size":160903,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2125108,"kb_used_data":1119352,"kb_used_omap":12,"kb_used_meta":1005683,"kb_avail":280990412,"statfs":{"total":289910292480,"available":287734181888,"internally_reserved":0,"allocated":1146216448,"data_stored":1145352908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1029819395},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,0,9,4,8,20,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24522752,"num_objects":-5987,"num_object_clones":0,"num_object_copies":-11974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":-302,"num_objects_unfound":0,"num_objects_dirty":-5987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5987,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":1208320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002853"},"pg_stats":[{"pgid":"2.f","version":"45'13258","reported_seq":13334,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.605021+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:07.605021+0000","last_peered":"2024-09-15T21:26:07.605021+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:07.605021+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32333824,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13258,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3762},{"shards":"0,2","objects":1450}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12864","reported_seq":12970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:03.896774+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:03.896774+0000","last_peered":"2024-09-15T21:26:03.896774+0000","last_clean":"2024-09-15T21:26:03.896774+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:03.896774+0000","last_undegraded":"2024-09-15T21:26:03.896774+0000","last_fullsized":"2024-09-15T21:26:03.896774+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31990726,"num_objects":7812,"num_object_clones":0,"num_object_copies":15624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7812,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11145,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13246","reported_seq":13372,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:03.891055+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:03.891055+0000","last_peered":"2024-09-15T21:26:03.891055+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:03.891055+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32190488,"num_objects":7860,"num_object_clones":0,"num_object_copies":15720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7860,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11346,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1576},{"shards":"2","objects":3593}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12714","reported_seq":12784,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:03.877033+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:03.877033+0000","last_peered":"2024-09-15T21:26:03.877033+0000","last_clean":"2024-09-15T21:26:03.877033+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:03.877033+0000","last_undegraded":"2024-09-15T21:26:03.877033+0000","last_fullsized":"2024-09-15T21:26:03.877033+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12714,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12878","reported_seq":13011,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:07.670152+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:07.670152+0000","last_peered":"2024-09-15T21:26:07.670152+0000","last_clean":"2024-09-15T21:26:07.670152+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:07.670152+0000","last_undegraded":"2024-09-15T21:26:07.670152+0000","last_fullsized":"2024-09-15T21:26:07.670152+0000","mapping_epoch":34,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31240675,"num_objects":7628,"num_object_clones":0,"num_object_copies":15256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9333,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13017","reported_seq":13154,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.686998+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:07.686998+0000","last_peered":"2024-09-15T21:26:07.686998+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:07.686998+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31203811,"num_objects":7619,"num_object_clones":0,"num_object_copies":15238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7619,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9693,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3543},{"shards":"0,2","objects":1379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13073","reported_seq":13210,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.673315+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:07.673315+0000","last_peered":"2024-09-15T21:26:07.673315+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:07.673315+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31118278,"num_objects":7599,"num_object_clones":0,"num_object_copies":15198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9384,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3438},{"shards":"0,2","objects":1426}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13198","reported_seq":13327,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:07.636137+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:07.636137+0000","last_peered":"2024-09-15T21:26:07.636137+0000","last_clean":"2024-09-15T21:26:07.636137+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:07.636137+0000","last_undegraded":"2024-09-15T21:26:07.636137+0000","last_fullsized":"2024-09-15T21:26:07.636137+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31416320,"num_objects":7670,"num_object_clones":0,"num_object_copies":15340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9970,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12842","reported_seq":12969,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:03.855439+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:03.855439+0000","last_peered":"2024-09-15T21:26:03.855439+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:03.855439+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31420899,"num_objects":7672,"num_object_clones":0,"num_object_copies":15344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7672,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10774,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1503},{"shards":"2","objects":3585}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12852","reported_seq":12983,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.547715+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:07.547715+0000","last_peered":"2024-09-15T21:26:07.547715+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:07.547715+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31252480,"num_objects":7630,"num_object_clones":0,"num_object_copies":15260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9525,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3610},{"shards":"0,2","objects":1413}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12879","reported_seq":12998,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:03.917246+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:03.917246+0000","last_peered":"2024-09-15T21:26:03.917246+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:03.917246+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31801827,"num_objects":7765,"num_object_clones":0,"num_object_copies":15530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7765,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10738,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1535},{"shards":"2","objects":3675}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12916","reported_seq":21374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:03.932990+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:03.932990+0000","last_peered":"2024-09-15T21:26:03.932990+0000","last_clean":"2024-09-15T21:26:03.932990+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:03.932990+0000","last_undegraded":"2024-09-15T21:26:03.932990+0000","last_fullsized":"2024-09-15T21:26:03.932990+0000","mapping_epoch":44,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31309824,"num_objects":7644,"num_object_clones":0,"num_object_copies":15288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10609,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12988","reported_seq":13122,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.669275+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:07.669275+0000","last_peered":"2024-09-15T21:26:07.669275+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:07.669275+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31531008,"num_objects":7698,"num_object_clones":0,"num_object_copies":15396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9932,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3604},{"shards":"0,2","objects":1450}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13044","reported_seq":13177,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.628062+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:07.628062+0000","last_peered":"2024-09-15T21:26:07.628062+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:07.628062+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30908416,"num_objects":7546,"num_object_clones":0,"num_object_copies":15092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9953,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3407},{"shards":"0,2","objects":1394}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12866","reported_seq":18334,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.714548+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:07.714548+0000","last_peered":"2024-09-15T21:26:07.714548+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:07.714548+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30810112,"num_objects":7522,"num_object_clones":0,"num_object_copies":15044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":1411,"num_objects_unfound":0,"num_objects_dirty":7522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9350,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":10911744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6104},{"shards":"0,2","objects":1411}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13165","reported_seq":13304,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:07.650267+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:07.650267+0000","last_peered":"2024-09-15T21:26:07.650267+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:07.650267+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31544262,"num_objects":7703,"num_object_clones":0,"num_object_copies":15406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7703,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9869,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3556},{"shards":"0,2","objects":1421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":503202550,"num_objects":122862,"num_object_clones":0,"num_object_copies":245724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41232,"num_objects_misplaced":1411,"num_objects_unfound":0,"num_objects_dirty":122862,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":167593,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6831,"num_bytes_recovered":27963392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1142984704,"data_stored":1142835653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160900,"ondisk_log_size":160900,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":678040,"kb_used_data":379784,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93693800,"statfs":{"total":96636764160,"available":95942451200,"internally_reserved":0,"allocated":388898816,"data_stored":388623169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":817744,"kb_used_data":434868,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93554096,"statfs":{"total":96636764160,"available":95799394304,"internally_reserved":0,"allocated":445304832,"data_stored":445029180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,0,5,0,3,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629324,"kb_used_data":304700,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93742516,"statfs":{"total":96636764160,"available":95992336384,"internally_reserved":0,"allocated":312012800,"data_stored":311700559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,3,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":443555840,"data_stored":443519710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311513088,"data_stored":311440369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387915776,"data_stored":387875574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:13.369 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:13.923 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:13.928 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:13.954 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-15T21:26:12.784783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":493740809,"num_objects":120554,"num_object_clones":0,"num_object_copies":241108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41273,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":120554,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":169905,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6936,"num_bytes_recovered":28389395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2126336,"kb_used_data":1106704,"kb_used_omap":12,"kb_used_meta":1019571,"kb_avail":280989184,"statfs":{"total":289910292480,"available":287732924416,"internally_reserved":0,"allocated":1133264896,"data_stored":1132401356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1044040707},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,5,4,7,10,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24621056,"num_objects":-6011,"num_object_clones":0,"num_object_copies":-12022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":-342,"num_objects_unfound":0,"num_objects_dirty":-6011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6011,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":1204224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002839"},"pg_stats":[{"pgid":"2.f","version":"45'13418","reported_seq":13494,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.717045+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:12.717045+0000","last_peered":"2024-09-15T21:26:12.717045+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:12.717045+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31678464,"num_objects":7734,"num_object_clones":0,"num_object_copies":15468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13418,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3602},{"shards":"0,2","objects":1290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13016","reported_seq":13122,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:08.896131+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:08.896131+0000","last_peered":"2024-09-15T21:26:08.896131+0000","last_clean":"2024-09-15T21:26:08.896131+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:08.896131+0000","last_undegraded":"2024-09-15T21:26:08.896131+0000","last_fullsized":"2024-09-15T21:26:08.896131+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31368134,"num_objects":7660,"num_object_clones":0,"num_object_copies":15320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7660,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11297,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13410","reported_seq":13535,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:08.965125+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:08.965125+0000","last_peered":"2024-09-15T21:26:08.965125+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:08.965125+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31518744,"num_objects":7696,"num_object_clones":0,"num_object_copies":15392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7696,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11510,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1412},{"shards":"2","objects":3429}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12842","reported_seq":12913,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:08.942563+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:08.942563+0000","last_peered":"2024-09-15T21:26:08.942563+0000","last_clean":"2024-09-15T21:26:08.942563+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:08.942563+0000","last_undegraded":"2024-09-15T21:26:08.942563+0000","last_fullsized":"2024-09-15T21:26:08.942563+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12842,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13031","reported_seq":13164,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:12.647972+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:12.647972+0000","last_peered":"2024-09-15T21:26:12.647972+0000","last_clean":"2024-09-15T21:26:12.647972+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:12.647972+0000","last_undegraded":"2024-09-15T21:26:12.647972+0000","last_fullsized":"2024-09-15T21:26:12.647972+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30613987,"num_objects":7475,"num_object_clones":0,"num_object_copies":14950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9486,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13145","reported_seq":13280,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.698262+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:12.698262+0000","last_peered":"2024-09-15T21:26:12.698262+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:12.698262+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30679523,"num_objects":7491,"num_object_clones":0,"num_object_copies":14982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7491,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9821,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3415},{"shards":"0,2","objects":1251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13246","reported_seq":13382,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.722405+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:12.722405+0000","last_peered":"2024-09-15T21:26:12.722405+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:12.722405+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30409670,"num_objects":7426,"num_object_clones":0,"num_object_copies":14852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9557,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3265},{"shards":"0,2","objects":1253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13326","reported_seq":13455,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:12.729723+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:12.729723+0000","last_peered":"2024-09-15T21:26:12.729723+0000","last_clean":"2024-09-15T21:26:12.729723+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:12.729723+0000","last_undegraded":"2024-09-15T21:26:12.729723+0000","last_fullsized":"2024-09-15T21:26:12.729723+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30892032,"num_objects":7542,"num_object_clones":0,"num_object_copies":15084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10098,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12958","reported_seq":13085,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:08.870177+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:08.870177+0000","last_peered":"2024-09-15T21:26:08.870177+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:08.870177+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30945763,"num_objects":7556,"num_object_clones":0,"num_object_copies":15112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10890,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1387},{"shards":"2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12977","reported_seq":13108,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.643103+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:12.643103+0000","last_peered":"2024-09-15T21:26:12.643103+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:12.643103+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30740480,"num_objects":7505,"num_object_clones":0,"num_object_copies":15010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9650,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3485},{"shards":"0,2","objects":1288}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13027","reported_seq":13146,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:08.961993+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:08.961993+0000","last_peered":"2024-09-15T21:26:08.961993+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:08.961993+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195619,"num_objects":7617,"num_object_clones":0,"num_object_copies":15234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7617,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10886,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1387},{"shards":"2","objects":3527}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13083","reported_seq":21542,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:08.953115+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:08.953115+0000","last_peered":"2024-09-15T21:26:08.953115+0000","last_clean":"2024-09-15T21:26:08.953115+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:08.953115+0000","last_undegraded":"2024-09-15T21:26:08.953115+0000","last_fullsized":"2024-09-15T21:26:08.953115+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30625792,"num_objects":7477,"num_object_clones":0,"num_object_copies":14954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10776,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13131","reported_seq":13263,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.706767+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:12.706767+0000","last_peered":"2024-09-15T21:26:12.706767+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:12.706767+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30945280,"num_objects":7555,"num_object_clones":0,"num_object_copies":15110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10075,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3461},{"shards":"0,2","objects":1307}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13198","reported_seq":13331,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.606910+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:12.606910+0000","last_peered":"2024-09-15T21:26:12.606910+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:12.606910+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30277632,"num_objects":7392,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10107,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3253},{"shards":"0,2","objects":1240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13011","reported_seq":18688,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.738871+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:12.738871+0000","last_peered":"2024-09-15T21:26:12.738871+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:12.738871+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30216192,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9495,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2770,"num_bytes_recovered":11337728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6063},{"shards":"0,2","objects":1266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13291","reported_seq":13430,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.666478+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:12.666478+0000","last_peered":"2024-09-15T21:26:12.666478+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:12.666478+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31028166,"num_objects":7577,"num_object_clones":0,"num_object_copies":15154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7577,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9995,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3430},{"shards":"0,2","objects":1295}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":493740790,"num_objects":120552,"num_object_clones":0,"num_object_copies":241104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41273,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":120552,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":169903,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6935,"num_bytes_recovered":28389376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1131372544,"data_stored":1131223493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677572,"kb_used_data":375156,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93694268,"statfs":{"total":96636764160,"available":95942930432,"internally_reserved":0,"allocated":384159744,"data_stored":383884097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":819440,"kb_used_data":426848,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93552400,"statfs":{"total":96636764160,"available":95797657600,"internally_reserved":0,"allocated":437092352,"data_stored":436816700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,0,3,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629324,"kb_used_data":304700,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93742516,"statfs":{"total":96636764160,"available":95992336384,"internally_reserved":0,"allocated":312012800,"data_stored":311700559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,3,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":436629504,"data_stored":436593374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311513088,"data_stored":311440369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383229952,"data_stored":383189750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:13.955 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:14.502 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:14.502 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:14.523 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-15T21:26:12.784783+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":493740809,"num_objects":120554,"num_object_clones":0,"num_object_copies":241108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41273,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":120554,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":169905,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6936,"num_bytes_recovered":28389395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160713,"ondisk_log_size":160713,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2126336,"kb_used_data":1106704,"kb_used_omap":12,"kb_used_meta":1019571,"kb_avail":280989184,"statfs":{"total":289910292480,"available":287732924416,"internally_reserved":0,"allocated":1133264896,"data_stored":1132401356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1044040707},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,5,4,7,10,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24621056,"num_objects":-6011,"num_object_clones":0,"num_object_copies":-12022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":-342,"num_objects_unfound":0,"num_objects_dirty":-6011,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6011,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":294,"num_bytes_recovered":1204224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002839"},"pg_stats":[{"pgid":"2.f","version":"45'13418","reported_seq":13494,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.717045+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:12.717045+0000","last_peered":"2024-09-15T21:26:12.717045+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:12.717045+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31678464,"num_objects":7734,"num_object_clones":0,"num_object_copies":15468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13418,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3602},{"shards":"0,2","objects":1290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13016","reported_seq":13122,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:08.896131+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:08.896131+0000","last_peered":"2024-09-15T21:26:08.896131+0000","last_clean":"2024-09-15T21:26:08.896131+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:08.896131+0000","last_undegraded":"2024-09-15T21:26:08.896131+0000","last_fullsized":"2024-09-15T21:26:08.896131+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31368134,"num_objects":7660,"num_object_clones":0,"num_object_copies":15320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7660,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11297,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13410","reported_seq":13535,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:08.965125+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:08.965125+0000","last_peered":"2024-09-15T21:26:08.965125+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:08.965125+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31518744,"num_objects":7696,"num_object_clones":0,"num_object_copies":15392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7696,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11510,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1412},{"shards":"2","objects":3429}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12842","reported_seq":12913,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:08.942563+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:08.942563+0000","last_peered":"2024-09-15T21:26:08.942563+0000","last_clean":"2024-09-15T21:26:08.942563+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:08.942563+0000","last_undegraded":"2024-09-15T21:26:08.942563+0000","last_fullsized":"2024-09-15T21:26:08.942563+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30605312,"num_objects":7472,"num_object_clones":0,"num_object_copies":14944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12842,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13031","reported_seq":13164,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:12.647972+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:12.647972+0000","last_peered":"2024-09-15T21:26:12.647972+0000","last_clean":"2024-09-15T21:26:12.647972+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:12.647972+0000","last_undegraded":"2024-09-15T21:26:12.647972+0000","last_fullsized":"2024-09-15T21:26:12.647972+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30613987,"num_objects":7475,"num_object_clones":0,"num_object_copies":14950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9486,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13145","reported_seq":13280,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.698262+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:12.698262+0000","last_peered":"2024-09-15T21:26:12.698262+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:12.698262+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30679523,"num_objects":7491,"num_object_clones":0,"num_object_copies":14982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7491,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9821,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3415},{"shards":"0,2","objects":1251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13246","reported_seq":13382,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.722405+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:12.722405+0000","last_peered":"2024-09-15T21:26:12.722405+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:12.722405+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30409670,"num_objects":7426,"num_object_clones":0,"num_object_copies":14852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9557,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3265},{"shards":"0,2","objects":1253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13326","reported_seq":13455,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:12.729723+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:12.729723+0000","last_peered":"2024-09-15T21:26:12.729723+0000","last_clean":"2024-09-15T21:26:12.729723+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:12.729723+0000","last_undegraded":"2024-09-15T21:26:12.729723+0000","last_fullsized":"2024-09-15T21:26:12.729723+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30892032,"num_objects":7542,"num_object_clones":0,"num_object_copies":15084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10098,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12958","reported_seq":13085,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:08.870177+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:08.870177+0000","last_peered":"2024-09-15T21:26:08.870177+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:08.870177+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30945763,"num_objects":7556,"num_object_clones":0,"num_object_copies":15112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10890,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1387},{"shards":"2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12977","reported_seq":13108,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.643103+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:12.643103+0000","last_peered":"2024-09-15T21:26:12.643103+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:12.643103+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30740480,"num_objects":7505,"num_object_clones":0,"num_object_copies":15010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9650,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3485},{"shards":"0,2","objects":1288}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13027","reported_seq":13146,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:08.961993+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:08.961993+0000","last_peered":"2024-09-15T21:26:08.961993+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:08.961993+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31195619,"num_objects":7617,"num_object_clones":0,"num_object_copies":15234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7617,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10886,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1387},{"shards":"2","objects":3527}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13083","reported_seq":21542,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:08.953115+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:08.953115+0000","last_peered":"2024-09-15T21:26:08.953115+0000","last_clean":"2024-09-15T21:26:08.953115+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:08.953115+0000","last_undegraded":"2024-09-15T21:26:08.953115+0000","last_fullsized":"2024-09-15T21:26:08.953115+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30625792,"num_objects":7477,"num_object_clones":0,"num_object_copies":14954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10776,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13131","reported_seq":13263,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.706767+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:12.706767+0000","last_peered":"2024-09-15T21:26:12.706767+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:12.706767+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30945280,"num_objects":7555,"num_object_clones":0,"num_object_copies":15110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10075,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3461},{"shards":"0,2","objects":1307}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13198","reported_seq":13331,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.606910+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:12.606910+0000","last_peered":"2024-09-15T21:26:12.606910+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:12.606910+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30277632,"num_objects":7392,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10107,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3253},{"shards":"0,2","objects":1240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13011","reported_seq":18688,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.738871+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:12.738871+0000","last_peered":"2024-09-15T21:26:12.738871+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:12.738871+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30216192,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9495,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2770,"num_bytes_recovered":11337728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6063},{"shards":"0,2","objects":1266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13291","reported_seq":13430,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.666478+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:12.666478+0000","last_peered":"2024-09-15T21:26:12.666478+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:12.666478+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31028166,"num_objects":7577,"num_object_clones":0,"num_object_copies":15154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7577,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9995,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3430},{"shards":"0,2","objects":1295}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":493740790,"num_objects":120552,"num_object_clones":0,"num_object_copies":241104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41273,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":120552,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":169903,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6935,"num_bytes_recovered":28389376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1131372544,"data_stored":1131223493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160710,"ondisk_log_size":160710,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677572,"kb_used_data":375156,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93694268,"statfs":{"total":96636764160,"available":95942930432,"internally_reserved":0,"allocated":384159744,"data_stored":383884097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":819440,"kb_used_data":426848,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93552400,"statfs":{"total":96636764160,"available":95797657600,"internally_reserved":0,"allocated":437092352,"data_stored":436816700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,0,3,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629324,"kb_used_data":304700,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93742516,"statfs":{"total":96636764160,"available":95992336384,"internally_reserved":0,"allocated":312012800,"data_stored":311700559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,3,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":436629504,"data_stored":436593374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311513088,"data_stored":311440369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383229952,"data_stored":383189750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:14.524 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:14.525 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13418", 'reported_seq': 13494, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:12.717045+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:26:12.717045+0000', 'last_peered': '2024-09-15T21:26:12.717045+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:26:12.717045+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31678464, 'num_objects': 7734, 'num_object_clones': 0, 'num_object_copies': 15468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7734, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13418, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3602}, {'shards': '0,2', 'objects': 1290}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:14.525 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:26:14.525 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'13410", 'reported_seq': 13535, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:08.965125+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:26:08.965125+0000', 'last_peered': '2024-09-15T21:26:08.965125+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:26:08.965125+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31518744, 'num_objects': 7696, 'num_object_clones': 0, 'num_object_copies': 15392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7696, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11510, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1412}, {'shards': '2', 'objects': 3429}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:14.525 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:14.525 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13145", 'reported_seq': 13280, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:12.698262+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:26:12.698262+0000', 'last_peered': '2024-09-15T21:26:12.698262+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:26:12.698262+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30679523, 'num_objects': 7491, 'num_object_clones': 0, 'num_object_copies': 14982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7491, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9821, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3415}, {'shards': '0,2', 'objects': 1251}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:14.525 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:14.525 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13246", 'reported_seq': 13382, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:12.722405+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:26:12.722405+0000', 'last_peered': '2024-09-15T21:26:12.722405+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:26:12.722405+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30409670, 'num_objects': 7426, 'num_object_clones': 0, 'num_object_copies': 14852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9557, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3265}, {'shards': '0,2', 'objects': 1253}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:14.525 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:26:14.525 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'12958", 'reported_seq': 13085, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:08.870177+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:26:08.870177+0000', 'last_peered': '2024-09-15T21:26:08.870177+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:26:08.870177+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30945763, 'num_objects': 7556, 'num_object_clones': 0, 'num_object_copies': 15112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7556, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10890, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1387}, {'shards': '2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:14.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:14.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'12977", 'reported_seq': 13108, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:12.643103+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:26:12.643103+0000', 'last_peered': '2024-09-15T21:26:12.643103+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:26:12.643103+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30740480, 'num_objects': 7505, 'num_object_clones': 0, 'num_object_copies': 15010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7505, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9650, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3485}, {'shards': '0,2', 'objects': 1288}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:14.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:14.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13027", 'reported_seq': 13146, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:08.961993+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:26:08.961993+0000', 'last_peered': '2024-09-15T21:26:08.961993+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:26:08.961993+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31195619, 'num_objects': 7617, 'num_object_clones': 0, 'num_object_copies': 15234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7617, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10886, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1387}, {'shards': '2', 'objects': 3527}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:14.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:14.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13131", 'reported_seq': 13263, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:12.706767+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:26:12.706767+0000', 'last_peered': '2024-09-15T21:26:12.706767+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:26:12.706767+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30945280, 'num_objects': 7555, 'num_object_clones': 0, 'num_object_copies': 15110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7555, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10075, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3461}, {'shards': '0,2', 'objects': 1307}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:14.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:26:14.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13198", 'reported_seq': 13331, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:12.606910+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:26:12.606910+0000', 'last_peered': '2024-09-15T21:26:12.606910+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:26:12.606910+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30277632, 'num_objects': 7392, 'num_object_clones': 0, 'num_object_copies': 14784, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7392, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10107, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3253}, {'shards': '0,2', 'objects': 1240}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:14.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:14.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13011", 'reported_seq': 18688, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:12.738871+0000', 'last_change': '2024-09-15T21:26:05.146366+0000', 'last_active': '2024-09-15T21:26:12.738871+0000', 'last_peered': '2024-09-15T21:26:12.738871+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:26:12.738871+0000', 'last_undegraded': '2024-09-15T21:26:05.134624+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30216192, 'num_objects': 7377, 'num_object_clones': 0, 'num_object_copies': 14754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 48, 'num_objects_misplaced': 1266, 'num_objects_unfound': 0, 'num_objects_dirty': 7377, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9495, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2770, 'num_bytes_recovered': 11337728, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6063}, {'shards': '0,2', 'objects': 1266}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:14.526 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:26:14.526 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13291", 'reported_seq': 13430, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:12.666478+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:26:12.666478+0000', 'last_peered': '2024-09-15T21:26:12.666478+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:26:12.666478+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31028166, 'num_objects': 7577, 'num_object_clones': 0, 'num_object_copies': 15154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7577, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9995, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3430}, {'shards': '0,2', 'objects': 1295}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:14.527 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:15.205 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:15.206 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41273/241108 objects degraded (17.118%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":426,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":120554,"data_bytes":493740809,"bytes_used":2177368064,"bytes_avail":287732924416,"bytes_total":289910292480,"degraded_objects":41273,"degraded_total":241108,"degraded_ratio":0.17118054979511257,"misplaced_objects":1266,"misplaced_total":241108,"misplaced_ratio":0.0052507589959686117,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":100328,"recovering_keys_per_sec":0,"num_objects_recovered":294,"num_bytes_recovered":1204224,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":500},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:15.230 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:15.230 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:15.793 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:15.817 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-09-15T21:26:14.785261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":490165001,"num_objects":119681,"num_object_clones":0,"num_object_copies":239362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41273,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":119681,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":170778,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6936,"num_bytes_recovered":28389395,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160786,"ondisk_log_size":160786,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2129392,"kb_used_data":1101412,"kb_used_omap":12,"kb_used_meta":1027891,"kb_avail":280986128,"statfs":{"total":289910292480,"available":287729795072,"internally_reserved":0,"allocated":1127845888,"data_stored":1126982348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1052560387},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,6,1,4,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22269952,"num_objects":-5437,"num_object_clones":0,"num_object_copies":-10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":-248,"num_objects_unfound":0,"num_objects_dirty":-5437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5437,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":819200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002856"},"pg_stats":[{"pgid":"2.f","version":"45'13418","reported_seq":13494,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.717045+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:12.717045+0000","last_peered":"2024-09-15T21:26:12.717045+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:12.717045+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31678464,"num_objects":7734,"num_object_clones":0,"num_object_copies":15468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13418,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3602},{"shards":"0,2","objects":1290}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13160","reported_seq":13266,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:13.903121+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:13.903121+0000","last_peered":"2024-09-15T21:26:13.903121+0000","last_clean":"2024-09-15T21:26:13.903121+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:13.903121+0000","last_undegraded":"2024-09-15T21:26:13.903121+0000","last_fullsized":"2024-09-15T21:26:13.903121+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30778310,"num_objects":7516,"num_object_clones":0,"num_object_copies":15032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7516,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11441,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13546","reported_seq":13673,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:13.888473+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:13.888473+0000","last_peered":"2024-09-15T21:26:13.888473+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:13.888473+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30961688,"num_objects":7560,"num_object_clones":0,"num_object_copies":15120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11646,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1276},{"shards":"2","objects":3293}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12984","reported_seq":13055,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:13.932258+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:13.932258+0000","last_peered":"2024-09-15T21:26:13.932258+0000","last_clean":"2024-09-15T21:26:13.932258+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:13.932258+0000","last_undegraded":"2024-09-15T21:26:13.932258+0000","last_fullsized":"2024-09-15T21:26:13.932258+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30023680,"num_objects":7330,"num_object_clones":0,"num_object_copies":14660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12984,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13031","reported_seq":13164,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:12.647972+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:12.647972+0000","last_peered":"2024-09-15T21:26:12.647972+0000","last_clean":"2024-09-15T21:26:12.647972+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:12.647972+0000","last_undegraded":"2024-09-15T21:26:12.647972+0000","last_fullsized":"2024-09-15T21:26:12.647972+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30613987,"num_objects":7475,"num_object_clones":0,"num_object_copies":14950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9486,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13145","reported_seq":13280,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.698262+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:12.698262+0000","last_peered":"2024-09-15T21:26:12.698262+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:12.698262+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30679523,"num_objects":7491,"num_object_clones":0,"num_object_copies":14982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7491,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9821,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3415},{"shards":"0,2","objects":1251}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13246","reported_seq":13382,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.722405+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:12.722405+0000","last_peered":"2024-09-15T21:26:12.722405+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:12.722405+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30409670,"num_objects":7426,"num_object_clones":0,"num_object_copies":14852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9557,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3265},{"shards":"0,2","objects":1253}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13326","reported_seq":13455,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:12.729723+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:12.729723+0000","last_peered":"2024-09-15T21:26:12.729723+0000","last_clean":"2024-09-15T21:26:12.729723+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:12.729723+0000","last_undegraded":"2024-09-15T21:26:12.729723+0000","last_fullsized":"2024-09-15T21:26:12.729723+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30892032,"num_objects":7542,"num_object_clones":0,"num_object_copies":15084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10098,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13111","reported_seq":13238,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:13.951287+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:13.951287+0000","last_peered":"2024-09-15T21:26:13.951287+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:13.951287+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30319075,"num_objects":7403,"num_object_clones":0,"num_object_copies":14806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11043,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1234},{"shards":"2","objects":3316}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12977","reported_seq":13108,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.643103+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:12.643103+0000","last_peered":"2024-09-15T21:26:12.643103+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:12.643103+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30740480,"num_objects":7505,"num_object_clones":0,"num_object_copies":15010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9650,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3485},{"shards":"0,2","objects":1288}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877993+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:23:35.877993+0000","last_peered":"2024-09-15T21:23:35.877993+0000","last_clean":"2024-09-15T21:23:35.877993+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:23:35.877993+0000","last_undegraded":"2024-09-15T21:23:35.877993+0000","last_fullsized":"2024-09-15T21:23:35.877993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13185","reported_seq":13302,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:13.918829+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:13.918829+0000","last_peered":"2024-09-15T21:26:13.918829+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:13.918829+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30548451,"num_objects":7459,"num_object_clones":0,"num_object_copies":14918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11044,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1229},{"shards":"2","objects":3369}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13223","reported_seq":21682,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:13.812147+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:13.812147+0000","last_peered":"2024-09-15T21:26:13.812147+0000","last_clean":"2024-09-15T21:26:13.812147+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:13.812147+0000","last_undegraded":"2024-09-15T21:26:13.812147+0000","last_fullsized":"2024-09-15T21:26:13.812147+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30052352,"num_objects":7337,"num_object_clones":0,"num_object_copies":14674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10916,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13131","reported_seq":13263,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.706767+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:12.706767+0000","last_peered":"2024-09-15T21:26:12.706767+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:12.706767+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30945280,"num_objects":7555,"num_object_clones":0,"num_object_copies":15110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10075,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3461},{"shards":"0,2","objects":1307}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13198","reported_seq":13331,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.606910+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:12.606910+0000","last_peered":"2024-09-15T21:26:12.606910+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:12.606910+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30277632,"num_objects":7392,"num_object_clones":0,"num_object_copies":14784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7392,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10107,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3253},{"shards":"0,2","objects":1240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13011","reported_seq":18688,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.738871+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:12.738871+0000","last_peered":"2024-09-15T21:26:12.738871+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:12.738871+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30216192,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":48,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9495,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2770,"num_bytes_recovered":11337728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6063},{"shards":"0,2","objects":1266}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13291","reported_seq":13430,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:12.666478+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:12.666478+0000","last_peered":"2024-09-15T21:26:12.666478+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:12.666478+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31028166,"num_objects":7577,"num_object_clones":0,"num_object_copies":15154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7577,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9995,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3430},{"shards":"0,2","objects":1295}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":490164982,"num_objects":119679,"num_object_clones":0,"num_object_copies":239358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41273,"num_objects_misplaced":1266,"num_objects_unfound":0,"num_objects_dirty":119679,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":170776,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6935,"num_bytes_recovered":28389376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1125986304,"data_stored":1125837253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677064,"kb_used_data":370460,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93694776,"statfs":{"total":96636764160,"available":95943450624,"internally_reserved":0,"allocated":379351040,"data_stored":379075393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":819440,"kb_used_data":426848,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93552400,"statfs":{"total":96636764160,"available":95797657600,"internally_reserved":0,"allocated":437092352,"data_stored":436816700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,0,3,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632888,"kb_used_data":304104,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93738952,"statfs":{"total":96636764160,"available":95988686848,"internally_reserved":0,"allocated":311402496,"data_stored":311090255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336653999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":436629504,"data_stored":436593374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310915072,"data_stored":310842353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378441728,"data_stored":378401526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:18.820 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:19.422 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:19.423 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:19.454 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-15T21:26:18.787263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":484389641,"num_objects":118271,"num_object_clones":0,"num_object_copies":236542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41311,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":118271,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":172188,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7043,"num_bytes_recovered":28827667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2117800,"kb_used_data":1093980,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":280997720,"statfs":{"total":289910292480,"available":287741665280,"internally_reserved":0,"allocated":1120235520,"data_stored":1119371980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1048300547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,6,8,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24502272,"num_objects":-5982,"num_object_clones":0,"num_object_copies":-11964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":-393,"num_objects_unfound":0,"num_objects_dirty":-5982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5982,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":1257472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003909"},"pg_stats":[{"pgid":"2.f","version":"45'13554","reported_seq":13630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.780562+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:17.780562+0000","last_peered":"2024-09-15T21:26:17.780562+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:17.780562+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31121408,"num_objects":7598,"num_object_clones":0,"num_object_copies":15196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13554,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3466},{"shards":"0,2","objects":1154}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13160","reported_seq":13266,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:13.903121+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:13.903121+0000","last_peered":"2024-09-15T21:26:13.903121+0000","last_clean":"2024-09-15T21:26:13.903121+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:13.903121+0000","last_undegraded":"2024-09-15T21:26:13.903121+0000","last_fullsized":"2024-09-15T21:26:13.903121+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30778310,"num_objects":7516,"num_object_clones":0,"num_object_copies":15032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7516,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11441,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13546","reported_seq":13673,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:13.888473+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:13.888473+0000","last_peered":"2024-09-15T21:26:13.888473+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:13.888473+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30961688,"num_objects":7560,"num_object_clones":0,"num_object_copies":15120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11646,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1276},{"shards":"2","objects":3293}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12984","reported_seq":13055,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:13.932258+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:13.932258+0000","last_peered":"2024-09-15T21:26:13.932258+0000","last_clean":"2024-09-15T21:26:13.932258+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:13.932258+0000","last_undegraded":"2024-09-15T21:26:13.932258+0000","last_fullsized":"2024-09-15T21:26:13.932258+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30023680,"num_objects":7330,"num_object_clones":0,"num_object_copies":14660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12984,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13181","reported_seq":13314,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:17.696589+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:17.696589+0000","last_peered":"2024-09-15T21:26:17.696589+0000","last_clean":"2024-09-15T21:26:17.696589+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:17.696589+0000","last_undegraded":"2024-09-15T21:26:17.696589+0000","last_fullsized":"2024-09-15T21:26:17.696589+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29999587,"num_objects":7325,"num_object_clones":0,"num_object_copies":14650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9636,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13270","reported_seq":13407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.597167+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:17.597167+0000","last_peered":"2024-09-15T21:26:17.597167+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:17.597167+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30167523,"num_objects":7366,"num_object_clones":0,"num_object_copies":14732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7366,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9946,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3290},{"shards":"0,2","objects":1126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13383","reported_seq":13520,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.759596+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:17.759596+0000","last_peered":"2024-09-15T21:26:17.759596+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:17.759596+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29848518,"num_objects":7289,"num_object_clones":0,"num_object_copies":14578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9694,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3128},{"shards":"0,2","objects":1116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13463","reported_seq":13592,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:17.758459+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:17.758459+0000","last_peered":"2024-09-15T21:26:17.758459+0000","last_clean":"2024-09-15T21:26:17.758459+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:17.758459+0000","last_undegraded":"2024-09-15T21:26:17.758459+0000","last_fullsized":"2024-09-15T21:26:17.758459+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30330880,"num_objects":7405,"num_object_clones":0,"num_object_copies":14810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10235,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13111","reported_seq":13238,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:13.951287+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:13.951287+0000","last_peered":"2024-09-15T21:26:13.951287+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:13.951287+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30319075,"num_objects":7403,"num_object_clones":0,"num_object_copies":14806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11043,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1234},{"shards":"2","objects":3316}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13119","reported_seq":13247,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.772229+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:17.772229+0000","last_peered":"2024-09-15T21:26:17.772229+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:17.772229+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30158848,"num_objects":7363,"num_object_clones":0,"num_object_copies":14726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9792,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3343},{"shards":"0,2","objects":1146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13185","reported_seq":13302,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:13.918829+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:13.918829+0000","last_peered":"2024-09-15T21:26:13.918829+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:13.918829+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30548451,"num_objects":7459,"num_object_clones":0,"num_object_copies":14918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11044,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1229},{"shards":"2","objects":3369}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13223","reported_seq":21682,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:13.812147+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:13.812147+0000","last_peered":"2024-09-15T21:26:13.812147+0000","last_clean":"2024-09-15T21:26:13.812147+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:13.812147+0000","last_undegraded":"2024-09-15T21:26:13.812147+0000","last_fullsized":"2024-09-15T21:26:13.812147+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30052352,"num_objects":7337,"num_object_clones":0,"num_object_copies":14674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10916,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13289","reported_seq":13423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.781108+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:17.781108+0000","last_peered":"2024-09-15T21:26:17.781108+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:17.781108+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30298112,"num_objects":7397,"num_object_clones":0,"num_object_copies":14794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10233,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3303},{"shards":"0,2","objects":1149}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13346","reported_seq":13479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.678364+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:17.678364+0000","last_peered":"2024-09-15T21:26:17.678364+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:17.678364+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29671424,"num_objects":7244,"num_object_clones":0,"num_object_copies":14488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10255,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3105},{"shards":"0,2","objects":1092}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13152","reported_seq":19043,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.762945+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:17.762945+0000","last_peered":"2024-09-15T21:26:17.762945+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:17.762945+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29638656,"num_objects":7236,"num_object_clones":0,"num_object_copies":14472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":7236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9636,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2877,"num_bytes_recovered":11776000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6029},{"shards":"0,2","objects":1121}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13427","reported_seq":13566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.745678+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:17.745678+0000","last_peered":"2024-09-15T21:26:17.745678+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:17.745678+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30471110,"num_objects":7441,"num_object_clones":0,"num_object_copies":14882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10131,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3294},{"shards":"0,2","objects":1159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":484389622,"num_objects":118269,"num_object_clones":0,"num_object_copies":236538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41311,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":118269,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":172186,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7042,"num_bytes_recovered":28827648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1118416896,"data_stored":1118267845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677064,"kb_used_data":370460,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93694776,"statfs":{"total":96636764160,"available":95943450624,"internally_reserved":0,"allocated":379351040,"data_stored":379075393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":816784,"kb_used_data":420032,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93555056,"statfs":{"total":96636764160,"available":95800377344,"internally_reserved":0,"allocated":430112768,"data_stored":429837116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623952,"kb_used_data":303488,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93747888,"statfs":{"total":96636764160,"available":95997837312,"internally_reserved":0,"allocated":310771712,"data_stored":310459471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328134319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":429592576,"data_stored":429556446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310382592,"data_stored":310309873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378441728,"data_stored":378401526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:19.456 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:20.008 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:20.010 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:20.037 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":253,"stamp":"2024-09-15T21:26:18.787263+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":484389641,"num_objects":118271,"num_object_clones":0,"num_object_copies":236542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41311,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":118271,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":172188,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7043,"num_bytes_recovered":28827667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160896,"ondisk_log_size":160896,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2117800,"kb_used_data":1093980,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":280997720,"statfs":{"total":289910292480,"available":287741665280,"internally_reserved":0,"allocated":1120235520,"data_stored":1119371980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1048300547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,4,6,8,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24502272,"num_objects":-5982,"num_object_clones":0,"num_object_copies":-11964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":-393,"num_objects_unfound":0,"num_objects_dirty":-5982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5982,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":1257472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003909"},"pg_stats":[{"pgid":"2.f","version":"45'13554","reported_seq":13630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.780562+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:17.780562+0000","last_peered":"2024-09-15T21:26:17.780562+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:17.780562+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31121408,"num_objects":7598,"num_object_clones":0,"num_object_copies":15196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13554,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3466},{"shards":"0,2","objects":1154}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13160","reported_seq":13266,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:13.903121+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:13.903121+0000","last_peered":"2024-09-15T21:26:13.903121+0000","last_clean":"2024-09-15T21:26:13.903121+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:13.903121+0000","last_undegraded":"2024-09-15T21:26:13.903121+0000","last_fullsized":"2024-09-15T21:26:13.903121+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30778310,"num_objects":7516,"num_object_clones":0,"num_object_copies":15032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7516,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11441,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13546","reported_seq":13673,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:13.888473+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:13.888473+0000","last_peered":"2024-09-15T21:26:13.888473+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:13.888473+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30961688,"num_objects":7560,"num_object_clones":0,"num_object_copies":15120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11646,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1276},{"shards":"2","objects":3293}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12984","reported_seq":13055,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:13.932258+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:13.932258+0000","last_peered":"2024-09-15T21:26:13.932258+0000","last_clean":"2024-09-15T21:26:13.932258+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:13.932258+0000","last_undegraded":"2024-09-15T21:26:13.932258+0000","last_fullsized":"2024-09-15T21:26:13.932258+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30023680,"num_objects":7330,"num_object_clones":0,"num_object_copies":14660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7330,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12984,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13181","reported_seq":13314,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:17.696589+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:17.696589+0000","last_peered":"2024-09-15T21:26:17.696589+0000","last_clean":"2024-09-15T21:26:17.696589+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:17.696589+0000","last_undegraded":"2024-09-15T21:26:17.696589+0000","last_fullsized":"2024-09-15T21:26:17.696589+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29999587,"num_objects":7325,"num_object_clones":0,"num_object_copies":14650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9636,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13270","reported_seq":13407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.597167+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:17.597167+0000","last_peered":"2024-09-15T21:26:17.597167+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:17.597167+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30167523,"num_objects":7366,"num_object_clones":0,"num_object_copies":14732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7366,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9946,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3290},{"shards":"0,2","objects":1126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13383","reported_seq":13520,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.759596+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:17.759596+0000","last_peered":"2024-09-15T21:26:17.759596+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:17.759596+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29848518,"num_objects":7289,"num_object_clones":0,"num_object_copies":14578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9694,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3128},{"shards":"0,2","objects":1116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13463","reported_seq":13592,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:17.758459+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:17.758459+0000","last_peered":"2024-09-15T21:26:17.758459+0000","last_clean":"2024-09-15T21:26:17.758459+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:17.758459+0000","last_undegraded":"2024-09-15T21:26:17.758459+0000","last_fullsized":"2024-09-15T21:26:17.758459+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30330880,"num_objects":7405,"num_object_clones":0,"num_object_copies":14810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10235,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13111","reported_seq":13238,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:13.951287+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:13.951287+0000","last_peered":"2024-09-15T21:26:13.951287+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:13.951287+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30319075,"num_objects":7403,"num_object_clones":0,"num_object_copies":14806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7403,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11043,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1234},{"shards":"2","objects":3316}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13119","reported_seq":13247,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.772229+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:17.772229+0000","last_peered":"2024-09-15T21:26:17.772229+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:17.772229+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30158848,"num_objects":7363,"num_object_clones":0,"num_object_copies":14726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9792,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3343},{"shards":"0,2","objects":1146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13185","reported_seq":13302,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:13.918829+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:13.918829+0000","last_peered":"2024-09-15T21:26:13.918829+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:13.918829+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30548451,"num_objects":7459,"num_object_clones":0,"num_object_copies":14918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7459,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11044,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1229},{"shards":"2","objects":3369}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13223","reported_seq":21682,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:13.812147+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:13.812147+0000","last_peered":"2024-09-15T21:26:13.812147+0000","last_clean":"2024-09-15T21:26:13.812147+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:13.812147+0000","last_undegraded":"2024-09-15T21:26:13.812147+0000","last_fullsized":"2024-09-15T21:26:13.812147+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30052352,"num_objects":7337,"num_object_clones":0,"num_object_copies":14674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10916,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13289","reported_seq":13423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.781108+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:17.781108+0000","last_peered":"2024-09-15T21:26:17.781108+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:17.781108+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30298112,"num_objects":7397,"num_object_clones":0,"num_object_copies":14794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10233,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3303},{"shards":"0,2","objects":1149}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13346","reported_seq":13479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.678364+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:17.678364+0000","last_peered":"2024-09-15T21:26:17.678364+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:17.678364+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29671424,"num_objects":7244,"num_object_clones":0,"num_object_copies":14488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10255,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3105},{"shards":"0,2","objects":1092}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13152","reported_seq":19043,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.762945+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:17.762945+0000","last_peered":"2024-09-15T21:26:17.762945+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:17.762945+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29638656,"num_objects":7236,"num_object_clones":0,"num_object_copies":14472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":7236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9636,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2877,"num_bytes_recovered":11776000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6029},{"shards":"0,2","objects":1121}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13427","reported_seq":13566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.745678+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:17.745678+0000","last_peered":"2024-09-15T21:26:17.745678+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:17.745678+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30471110,"num_objects":7441,"num_object_clones":0,"num_object_copies":14882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10131,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3294},{"shards":"0,2","objects":1159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":484389622,"num_objects":118269,"num_object_clones":0,"num_object_copies":236538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41311,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":118269,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":172186,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7042,"num_bytes_recovered":28827648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1118416896,"data_stored":1118267845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160893,"ondisk_log_size":160893,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":677064,"kb_used_data":370460,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93694776,"statfs":{"total":96636764160,"available":95943450624,"internally_reserved":0,"allocated":379351040,"data_stored":379075393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":816784,"kb_used_data":420032,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93555056,"statfs":{"total":96636764160,"available":95800377344,"internally_reserved":0,"allocated":430112768,"data_stored":429837116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623952,"kb_used_data":303488,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93747888,"statfs":{"total":96636764160,"available":95997837312,"internally_reserved":0,"allocated":310771712,"data_stored":310459471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328134319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":429592576,"data_stored":429556446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310382592,"data_stored":310309873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378441728,"data_stored":378401526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:20.038 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:20.039 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13554", 'reported_seq': 13630, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:17.780562+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:26:17.780562+0000', 'last_peered': '2024-09-15T21:26:17.780562+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:26:17.780562+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31121408, 'num_objects': 7598, 'num_object_clones': 0, 'num_object_copies': 15196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7598, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13554, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3466}, {'shards': '0,2', 'objects': 1154}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:20.039 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:26:20.039 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'13546", 'reported_seq': 13673, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:13.888473+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:26:13.888473+0000', 'last_peered': '2024-09-15T21:26:13.888473+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:26:13.888473+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30961688, 'num_objects': 7560, 'num_object_clones': 0, 'num_object_copies': 15120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7560, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11646, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1276}, {'shards': '2', 'objects': 3293}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:20.039 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:20.039 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13270", 'reported_seq': 13407, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:17.597167+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:26:17.597167+0000', 'last_peered': '2024-09-15T21:26:17.597167+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:26:17.597167+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30167523, 'num_objects': 7366, 'num_object_clones': 0, 'num_object_copies': 14732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7366, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9946, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3290}, {'shards': '0,2', 'objects': 1126}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:20.039 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:20.039 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13383", 'reported_seq': 13520, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:17.759596+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:26:17.759596+0000', 'last_peered': '2024-09-15T21:26:17.759596+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:26:17.759596+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29848518, 'num_objects': 7289, 'num_object_clones': 0, 'num_object_copies': 14578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7289, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9694, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3128}, {'shards': '0,2', 'objects': 1116}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:20.039 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:26:20.039 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'13111", 'reported_seq': 13238, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:13.951287+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:26:13.951287+0000', 'last_peered': '2024-09-15T21:26:13.951287+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:26:13.951287+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30319075, 'num_objects': 7403, 'num_object_clones': 0, 'num_object_copies': 14806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7403, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11043, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1234}, {'shards': '2', 'objects': 3316}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:20.040 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:20.040 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13119", 'reported_seq': 13247, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:17.772229+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:26:17.772229+0000', 'last_peered': '2024-09-15T21:26:17.772229+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:26:17.772229+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30158848, 'num_objects': 7363, 'num_object_clones': 0, 'num_object_copies': 14726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7363, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9792, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3343}, {'shards': '0,2', 'objects': 1146}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:20.040 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:20.040 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13185", 'reported_seq': 13302, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:13.918829+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:26:13.918829+0000', 'last_peered': '2024-09-15T21:26:13.918829+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:26:13.918829+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30548451, 'num_objects': 7459, 'num_object_clones': 0, 'num_object_copies': 14918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7459, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11044, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1229}, {'shards': '2', 'objects': 3369}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:20.040 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:20.040 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13289", 'reported_seq': 13423, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:17.781108+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:26:17.781108+0000', 'last_peered': '2024-09-15T21:26:17.781108+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:26:17.781108+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30298112, 'num_objects': 7397, 'num_object_clones': 0, 'num_object_copies': 14794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7397, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10233, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3303}, {'shards': '0,2', 'objects': 1149}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:20.040 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:26:20.040 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13346", 'reported_seq': 13479, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:17.678364+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:26:17.678364+0000', 'last_peered': '2024-09-15T21:26:17.678364+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:26:17.678364+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29671424, 'num_objects': 7244, 'num_object_clones': 0, 'num_object_copies': 14488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7244, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10255, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3105}, {'shards': '0,2', 'objects': 1092}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:20.040 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:20.040 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13152", 'reported_seq': 19043, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:17.762945+0000', 'last_change': '2024-09-15T21:26:05.146366+0000', 'last_active': '2024-09-15T21:26:17.762945+0000', 'last_peered': '2024-09-15T21:26:17.762945+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:26:17.762945+0000', 'last_undegraded': '2024-09-15T21:26:05.134624+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29638656, 'num_objects': 7236, 'num_object_clones': 0, 'num_object_copies': 14472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 86, 'num_objects_misplaced': 1121, 'num_objects_unfound': 0, 'num_objects_dirty': 7236, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9636, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2877, 'num_bytes_recovered': 11776000, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6029}, {'shards': '0,2', 'objects': 1121}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:20.040 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:26:20.041 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13427", 'reported_seq': 13566, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:17.745678+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:26:17.745678+0000', 'last_peered': '2024-09-15T21:26:17.745678+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:26:17.745678+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30471110, 'num_objects': 7441, 'num_object_clones': 0, 'num_object_copies': 14882, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7441, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10131, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3294}, {'shards': '0,2', 'objects': 1159}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:20.041 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:20.711 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:20.711 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41311/236542 objects degraded (17.465%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":432,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":118271,"data_bytes":484389641,"bytes_used":2168627200,"bytes_avail":287741665280,"bytes_total":289910292480,"degraded_objects":41311,"degraded_total":236542,"degraded_ratio":0.17464551749794965,"misplaced_objects":1121,"misplaced_total":236542,"misplaced_ratio":0.0047391160977754481,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":104755,"recovering_keys_per_sec":0,"num_objects_recovered":307,"num_bytes_recovered":1257472,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":498},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:20.730 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:20.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:21.282 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:21.284 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:21.307 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-15T21:26:20.787776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":480887561,"num_objects":117416,"num_object_clones":0,"num_object_copies":234832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41311,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":117416,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":173043,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7043,"num_bytes_recovered":28827667,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160851,"ondisk_log_size":160851,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2117376,"kb_used_data":1089416,"kb_used_omap":12,"kb_used_meta":1027891,"kb_avail":280998144,"statfs":{"total":289910292480,"available":287742099456,"internally_reserved":0,"allocated":1115561984,"data_stored":1114698444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1052560387},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,7,11,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22315008,"num_objects":-5448,"num_object_clones":0,"num_object_copies":-10896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":79,"num_objects_misplaced":-290,"num_objects_unfound":0,"num_objects_dirty":-5448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5448,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":864256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003990"},"pg_stats":[{"pgid":"2.f","version":"45'13554","reported_seq":13630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.780562+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:17.780562+0000","last_peered":"2024-09-15T21:26:17.780562+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:17.780562+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31121408,"num_objects":7598,"num_object_clones":0,"num_object_copies":15196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13554,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3466},{"shards":"0,2","objects":1154}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13318","reported_seq":13424,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:18.922527+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:18.922527+0000","last_peered":"2024-09-15T21:26:18.922527+0000","last_clean":"2024-09-15T21:26:18.922527+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:18.922527+0000","last_undegraded":"2024-09-15T21:26:18.922527+0000","last_fullsized":"2024-09-15T21:26:18.922527+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30131142,"num_objects":7358,"num_object_clones":0,"num_object_copies":14716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7358,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11599,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13691","reported_seq":13818,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:18.978664+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:18.978664+0000","last_peered":"2024-09-15T21:26:18.978664+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:18.978664+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30367768,"num_objects":7415,"num_object_clones":0,"num_object_copies":14830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7415,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11791,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1131},{"shards":"2","objects":3148}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13120","reported_seq":13191,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:18.970300+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:18.970300+0000","last_peered":"2024-09-15T21:26:18.970300+0000","last_clean":"2024-09-15T21:26:18.970300+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:18.970300+0000","last_undegraded":"2024-09-15T21:26:18.970300+0000","last_fullsized":"2024-09-15T21:26:18.970300+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29466624,"num_objects":7194,"num_object_clones":0,"num_object_copies":14388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13120,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13181","reported_seq":13314,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:17.696589+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:17.696589+0000","last_peered":"2024-09-15T21:26:17.696589+0000","last_clean":"2024-09-15T21:26:17.696589+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:17.696589+0000","last_undegraded":"2024-09-15T21:26:17.696589+0000","last_fullsized":"2024-09-15T21:26:17.696589+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29999587,"num_objects":7325,"num_object_clones":0,"num_object_copies":14650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9636,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13270","reported_seq":13407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.597167+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:17.597167+0000","last_peered":"2024-09-15T21:26:17.597167+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:17.597167+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30167523,"num_objects":7366,"num_object_clones":0,"num_object_copies":14732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7366,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9946,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3290},{"shards":"0,2","objects":1126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13383","reported_seq":13520,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.759596+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:17.759596+0000","last_peered":"2024-09-15T21:26:17.759596+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:17.759596+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29848518,"num_objects":7289,"num_object_clones":0,"num_object_copies":14578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9694,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3128},{"shards":"0,2","objects":1116}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13463","reported_seq":13592,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:17.758459+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:17.758459+0000","last_peered":"2024-09-15T21:26:17.758459+0000","last_clean":"2024-09-15T21:26:17.758459+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:17.758459+0000","last_undegraded":"2024-09-15T21:26:17.758459+0000","last_fullsized":"2024-09-15T21:26:17.758459+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30330880,"num_objects":7405,"num_object_clones":0,"num_object_copies":14810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10235,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13252","reported_seq":13379,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:18.775965+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:18.775965+0000","last_peered":"2024-09-15T21:26:18.775965+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:18.775965+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29741539,"num_objects":7262,"num_object_clones":0,"num_object_copies":14524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7262,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11184,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1093},{"shards":"2","objects":3175}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13119","reported_seq":13247,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.772229+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:17.772229+0000","last_peered":"2024-09-15T21:26:17.772229+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:17.772229+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30158848,"num_objects":7363,"num_object_clones":0,"num_object_copies":14726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9792,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3343},{"shards":"0,2","objects":1146}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13324","reported_seq":13443,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:18.874839+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:18.874839+0000","last_peered":"2024-09-15T21:26:18.874839+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:18.874839+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29979107,"num_objects":7320,"num_object_clones":0,"num_object_copies":14640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7320,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11183,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1090},{"shards":"2","objects":3230}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13359","reported_seq":21818,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:18.924718+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:18.924718+0000","last_peered":"2024-09-15T21:26:18.924718+0000","last_clean":"2024-09-15T21:26:18.924718+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:18.924718+0000","last_undegraded":"2024-09-15T21:26:18.924718+0000","last_fullsized":"2024-09-15T21:26:18.924718+0000","mapping_epoch":44,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29495296,"num_objects":7201,"num_object_clones":0,"num_object_copies":14402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11052,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13289","reported_seq":13423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.781108+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:17.781108+0000","last_peered":"2024-09-15T21:26:17.781108+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:17.781108+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30298112,"num_objects":7397,"num_object_clones":0,"num_object_copies":14794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7397,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10233,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3303},{"shards":"0,2","objects":1149}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13346","reported_seq":13479,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.678364+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:17.678364+0000","last_peered":"2024-09-15T21:26:17.678364+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:17.678364+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29671424,"num_objects":7244,"num_object_clones":0,"num_object_copies":14488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10255,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3105},{"shards":"0,2","objects":1092}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13152","reported_seq":19043,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.762945+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:17.762945+0000","last_peered":"2024-09-15T21:26:17.762945+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:17.762945+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29638656,"num_objects":7236,"num_object_clones":0,"num_object_copies":14472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":86,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":7236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9636,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2877,"num_bytes_recovered":11776000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6029},{"shards":"0,2","objects":1121}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13427","reported_seq":13566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:17.745678+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:17.745678+0000","last_peered":"2024-09-15T21:26:17.745678+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:17.745678+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30471110,"num_objects":7441,"num_object_clones":0,"num_object_copies":14882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10131,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3294},{"shards":"0,2","objects":1159}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":480887542,"num_objects":117414,"num_object_clones":0,"num_object_copies":234828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41311,"num_objects_misplaced":1121,"num_objects_unfound":0,"num_objects_dirty":117414,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":173041,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7042,"num_bytes_recovered":28827648,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1113722880,"data_stored":1113573829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160848,"ondisk_log_size":160848,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":676640,"kb_used_data":365896,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93695200,"statfs":{"total":96636764160,"available":95943884800,"internally_reserved":0,"allocated":374677504,"data_stored":374401857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":816784,"kb_used_data":420032,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93555056,"statfs":{"total":96636764160,"available":95800377344,"internally_reserved":0,"allocated":430112768,"data_stored":429837116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,4,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623952,"kb_used_data":303488,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93747888,"statfs":{"total":96636764160,"available":95997837312,"internally_reserved":0,"allocated":310771712,"data_stored":310459471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328134319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":429592576,"data_stored":429556446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310382592,"data_stored":310309873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":373747712,"data_stored":373707510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:24.310 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:24.847 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:24.850 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:24.871 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-15T21:26:24.788652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":471597833,"num_objects":115148,"num_object_clones":0,"num_object_copies":230296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41371,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":115148,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":175311,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7143,"num_bytes_recovered":29237267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2105376,"kb_used_data":1077420,"kb_used_omap":12,"kb_used_meta":1027891,"kb_avail":281010144,"statfs":{"total":289910292480,"available":287754387456,"internally_reserved":0,"allocated":1103278080,"data_stored":1102414540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1052560387},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,5,8,3,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22142976,"num_objects":-5406,"num_object_clones":0,"num_object_copies":-10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":98,"num_objects_misplaced":-305,"num_objects_unfound":0,"num_objects_dirty":-5406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5406,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":847872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003869"},"pg_stats":[{"pgid":"2.f","version":"45'13706","reported_seq":13782,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.726575+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:22.726575+0000","last_peered":"2024-09-15T21:26:22.726575+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:22.726575+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30498816,"num_objects":7446,"num_object_clones":0,"num_object_copies":14892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13706,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3314},{"shards":"0,2","objects":1002}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13461","reported_seq":13567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:23.827016+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:23.827016+0000","last_peered":"2024-09-15T21:26:23.827016+0000","last_clean":"2024-09-15T21:26:23.827016+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:23.827016+0000","last_undegraded":"2024-09-15T21:26:23.827016+0000","last_fullsized":"2024-09-15T21:26:23.827016+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29545414,"num_objects":7215,"num_object_clones":0,"num_object_copies":14430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7215,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11742,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13834","reported_seq":13961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:23.928641+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:23.928641+0000","last_peered":"2024-09-15T21:26:23.928641+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:23.928641+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29782040,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11934,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":988},{"shards":"2","objects":3005}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13240","reported_seq":13311,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:23.916291+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:23.916291+0000","last_peered":"2024-09-15T21:26:23.916291+0000","last_clean":"2024-09-15T21:26:23.916291+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:23.916291+0000","last_undegraded":"2024-09-15T21:26:23.916291+0000","last_fullsized":"2024-09-15T21:26:23.916291+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28975104,"num_objects":7074,"num_object_clones":0,"num_object_copies":14148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13240,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13330","reported_seq":13463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:22.780229+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:22.780229+0000","last_peered":"2024-09-15T21:26:22.780229+0000","last_clean":"2024-09-15T21:26:22.780229+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:22.780229+0000","last_undegraded":"2024-09-15T21:26:22.780229+0000","last_fullsized":"2024-09-15T21:26:22.780229+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29389283,"num_objects":7176,"num_object_clones":0,"num_object_copies":14352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9785,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13409","reported_seq":13546,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.675207+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:22.675207+0000","last_peered":"2024-09-15T21:26:22.675207+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:22.675207+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29598179,"num_objects":7227,"num_object_clones":0,"num_object_copies":14454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7227,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10085,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3151},{"shards":"0,2","objects":987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13493","reported_seq":13630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.773989+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:22.773989+0000","last_peered":"2024-09-15T21:26:22.773989+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:22.773989+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29397958,"num_objects":7179,"num_object_clones":0,"num_object_copies":14358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9804,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3018},{"shards":"0,2","objects":1006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13604","reported_seq":13733,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:22.763339+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:22.763339+0000","last_peered":"2024-09-15T21:26:22.763339+0000","last_clean":"2024-09-15T21:26:22.763339+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:22.763339+0000","last_undegraded":"2024-09-15T21:26:22.763339+0000","last_fullsized":"2024-09-15T21:26:22.763339+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29753344,"num_objects":7264,"num_object_clones":0,"num_object_copies":14528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10376,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13392","reported_seq":13518,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:23.845121+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:23.845121+0000","last_peered":"2024-09-15T21:26:23.845121+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:23.845121+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29168099,"num_objects":7122,"num_object_clones":0,"num_object_copies":14244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11324,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":953},{"shards":"2","objects":3035}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13260","reported_seq":13392,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.660146+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:22.660146+0000","last_peered":"2024-09-15T21:26:22.660146+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:22.660146+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29581312,"num_objects":7222,"num_object_clones":0,"num_object_copies":14444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9933,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3202},{"shards":"0,2","objects":1005}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13473","reported_seq":13592,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:23.986684+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:23.986684+0000","last_peered":"2024-09-15T21:26:23.986684+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:23.986684+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29368803,"num_objects":7171,"num_object_clones":0,"num_object_copies":14342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11332,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":941},{"shards":"2","objects":3081}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13497","reported_seq":21956,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:23.937792+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:23.937792+0000","last_peered":"2024-09-15T21:26:23.937792+0000","last_clean":"2024-09-15T21:26:23.937792+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:23.937792+0000","last_undegraded":"2024-09-15T21:26:23.937792+0000","last_fullsized":"2024-09-15T21:26:23.937792+0000","mapping_epoch":44,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28930048,"num_objects":7063,"num_object_clones":0,"num_object_copies":14126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11190,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13415","reported_seq":13549,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.590719+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:22.590719+0000","last_peered":"2024-09-15T21:26:22.590719+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:22.590719+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29782016,"num_objects":7271,"num_object_clones":0,"num_object_copies":14542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10359,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3177},{"shards":"0,2","objects":1023}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13494","reported_seq":13627,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.724521+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:22.724521+0000","last_peered":"2024-09-15T21:26:22.724521+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:22.724521+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29065216,"num_objects":7096,"num_object_clones":0,"num_object_copies":14192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10403,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2957},{"shards":"0,2","objects":944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13317","reported_seq":19408,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.788159+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:22.788159+0000","last_peered":"2024-09-15T21:26:22.788159+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:22.788159+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28962816,"num_objects":7071,"num_object_clones":0,"num_object_copies":14142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":146,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":7071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2977,"num_bytes_recovered":12185600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5964},{"shards":"0,2","objects":961}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13591","reported_seq":13729,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.791544+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:22.791544+0000","last_peered":"2024-09-15T21:26:22.791544+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:22.791544+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29799366,"num_objects":7277,"num_object_clones":0,"num_object_copies":14554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7277,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10295,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3130},{"shards":"0,2","objects":995}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":471597814,"num_objects":115146,"num_object_clones":0,"num_object_copies":230292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41371,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":115146,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":175309,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7142,"num_bytes_recovered":29237248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1101475840,"data_stored":1101326789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160816,"ondisk_log_size":160816,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659740,"kb_used_data":361484,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93712100,"statfs":{"total":96636764160,"available":95961190400,"internally_reserved":0,"allocated":370159616,"data_stored":369883969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":818072,"kb_used_data":412996,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93553768,"statfs":{"total":96636764160,"available":95799058432,"internally_reserved":0,"allocated":422907904,"data_stored":422632252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":414772907},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627564,"kb_used_data":302940,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93744276,"statfs":{"total":96636764160,"available":95994138624,"internally_reserved":0,"allocated":310210560,"data_stored":309898319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,5,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":422514688,"data_stored":422478558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309821440,"data_stored":309748721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369139712,"data_stored":369099510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:24.873 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:25.455 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:25.456 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:25.489 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-15T21:26:24.788652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":471597833,"num_objects":115148,"num_object_clones":0,"num_object_copies":230296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41371,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":115148,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":175311,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7143,"num_bytes_recovered":29237267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2105376,"kb_used_data":1077420,"kb_used_omap":12,"kb_used_meta":1027891,"kb_avail":281010144,"statfs":{"total":289910292480,"available":287754387456,"internally_reserved":0,"allocated":1103278080,"data_stored":1102414540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1052560387},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,5,8,3,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22142976,"num_objects":-5406,"num_object_clones":0,"num_object_copies":-10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":98,"num_objects_misplaced":-305,"num_objects_unfound":0,"num_objects_dirty":-5406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5406,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":847872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003869"},"pg_stats":[{"pgid":"2.f","version":"45'13706","reported_seq":13782,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.726575+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:22.726575+0000","last_peered":"2024-09-15T21:26:22.726575+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:22.726575+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30498816,"num_objects":7446,"num_object_clones":0,"num_object_copies":14892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13706,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3314},{"shards":"0,2","objects":1002}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13461","reported_seq":13567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:23.827016+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:23.827016+0000","last_peered":"2024-09-15T21:26:23.827016+0000","last_clean":"2024-09-15T21:26:23.827016+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:23.827016+0000","last_undegraded":"2024-09-15T21:26:23.827016+0000","last_fullsized":"2024-09-15T21:26:23.827016+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29545414,"num_objects":7215,"num_object_clones":0,"num_object_copies":14430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7215,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11742,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13834","reported_seq":13961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:23.928641+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:23.928641+0000","last_peered":"2024-09-15T21:26:23.928641+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:23.928641+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29782040,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11934,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":988},{"shards":"2","objects":3005}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13240","reported_seq":13311,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:23.916291+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:23.916291+0000","last_peered":"2024-09-15T21:26:23.916291+0000","last_clean":"2024-09-15T21:26:23.916291+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:23.916291+0000","last_undegraded":"2024-09-15T21:26:23.916291+0000","last_fullsized":"2024-09-15T21:26:23.916291+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28975104,"num_objects":7074,"num_object_clones":0,"num_object_copies":14148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13240,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13330","reported_seq":13463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:22.780229+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:22.780229+0000","last_peered":"2024-09-15T21:26:22.780229+0000","last_clean":"2024-09-15T21:26:22.780229+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:22.780229+0000","last_undegraded":"2024-09-15T21:26:22.780229+0000","last_fullsized":"2024-09-15T21:26:22.780229+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29389283,"num_objects":7176,"num_object_clones":0,"num_object_copies":14352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9785,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13409","reported_seq":13546,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.675207+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:22.675207+0000","last_peered":"2024-09-15T21:26:22.675207+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:22.675207+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29598179,"num_objects":7227,"num_object_clones":0,"num_object_copies":14454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7227,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10085,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3151},{"shards":"0,2","objects":987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13493","reported_seq":13630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.773989+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:22.773989+0000","last_peered":"2024-09-15T21:26:22.773989+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:22.773989+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29397958,"num_objects":7179,"num_object_clones":0,"num_object_copies":14358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9804,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3018},{"shards":"0,2","objects":1006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13604","reported_seq":13733,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:22.763339+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:22.763339+0000","last_peered":"2024-09-15T21:26:22.763339+0000","last_clean":"2024-09-15T21:26:22.763339+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:22.763339+0000","last_undegraded":"2024-09-15T21:26:22.763339+0000","last_fullsized":"2024-09-15T21:26:22.763339+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29753344,"num_objects":7264,"num_object_clones":0,"num_object_copies":14528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10376,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13392","reported_seq":13518,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:23.845121+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:23.845121+0000","last_peered":"2024-09-15T21:26:23.845121+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:23.845121+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29168099,"num_objects":7122,"num_object_clones":0,"num_object_copies":14244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11324,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":953},{"shards":"2","objects":3035}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13260","reported_seq":13392,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.660146+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:22.660146+0000","last_peered":"2024-09-15T21:26:22.660146+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:22.660146+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29581312,"num_objects":7222,"num_object_clones":0,"num_object_copies":14444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9933,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3202},{"shards":"0,2","objects":1005}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13473","reported_seq":13592,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:23.986684+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:23.986684+0000","last_peered":"2024-09-15T21:26:23.986684+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:23.986684+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29368803,"num_objects":7171,"num_object_clones":0,"num_object_copies":14342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11332,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":941},{"shards":"2","objects":3081}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13497","reported_seq":21956,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:23.937792+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:23.937792+0000","last_peered":"2024-09-15T21:26:23.937792+0000","last_clean":"2024-09-15T21:26:23.937792+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:23.937792+0000","last_undegraded":"2024-09-15T21:26:23.937792+0000","last_fullsized":"2024-09-15T21:26:23.937792+0000","mapping_epoch":44,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28930048,"num_objects":7063,"num_object_clones":0,"num_object_copies":14126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11190,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13415","reported_seq":13549,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.590719+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:22.590719+0000","last_peered":"2024-09-15T21:26:22.590719+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:22.590719+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29782016,"num_objects":7271,"num_object_clones":0,"num_object_copies":14542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10359,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3177},{"shards":"0,2","objects":1023}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13494","reported_seq":13627,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.724521+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:22.724521+0000","last_peered":"2024-09-15T21:26:22.724521+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:22.724521+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29065216,"num_objects":7096,"num_object_clones":0,"num_object_copies":14192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10403,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2957},{"shards":"0,2","objects":944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13317","reported_seq":19408,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.788159+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:22.788159+0000","last_peered":"2024-09-15T21:26:22.788159+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:22.788159+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28962816,"num_objects":7071,"num_object_clones":0,"num_object_copies":14142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":146,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":7071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2977,"num_bytes_recovered":12185600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5964},{"shards":"0,2","objects":961}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13591","reported_seq":13729,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.791544+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:22.791544+0000","last_peered":"2024-09-15T21:26:22.791544+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:22.791544+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29799366,"num_objects":7277,"num_object_clones":0,"num_object_copies":14554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7277,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10295,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3130},{"shards":"0,2","objects":995}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":471597814,"num_objects":115146,"num_object_clones":0,"num_object_copies":230292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41371,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":115146,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":175309,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7142,"num_bytes_recovered":29237248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1101475840,"data_stored":1101326789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160816,"ondisk_log_size":160816,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659740,"kb_used_data":361484,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93712100,"statfs":{"total":96636764160,"available":95961190400,"internally_reserved":0,"allocated":370159616,"data_stored":369883969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":818072,"kb_used_data":412996,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93553768,"statfs":{"total":96636764160,"available":95799058432,"internally_reserved":0,"allocated":422907904,"data_stored":422632252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":414772907},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627564,"kb_used_data":302940,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93744276,"statfs":{"total":96636764160,"available":95994138624,"internally_reserved":0,"allocated":310210560,"data_stored":309898319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,5,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":422514688,"data_stored":422478558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309821440,"data_stored":309748721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369139712,"data_stored":369099510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:25.491 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:25.491 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13706", 'reported_seq': 13782, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:22.726575+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:26:22.726575+0000', 'last_peered': '2024-09-15T21:26:22.726575+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:26:22.726575+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30498816, 'num_objects': 7446, 'num_object_clones': 0, 'num_object_copies': 14892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7446, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13706, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3314}, {'shards': '0,2', 'objects': 1002}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:25.491 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:26:25.491 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'13834", 'reported_seq': 13961, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:23.928641+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:26:23.928641+0000', 'last_peered': '2024-09-15T21:26:23.928641+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:26:23.928641+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "36'3800", 'ondisk_log_start': "36'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29782040, 'num_objects': 7272, 'num_object_clones': 0, 'num_object_copies': 14544, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7272, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11934, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 988}, {'shards': '2', 'objects': 3005}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:25.491 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:25.491 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13409", 'reported_seq': 13546, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:22.675207+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:26:22.675207+0000', 'last_peered': '2024-09-15T21:26:22.675207+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:26:22.675207+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29598179, 'num_objects': 7227, 'num_object_clones': 0, 'num_object_copies': 14454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7227, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10085, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3151}, {'shards': '0,2', 'objects': 987}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:25.491 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:25.491 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13493", 'reported_seq': 13630, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:22.773989+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:26:22.773989+0000', 'last_peered': '2024-09-15T21:26:22.773989+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:26:22.773989+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29397958, 'num_objects': 7179, 'num_object_clones': 0, 'num_object_copies': 14358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7179, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9804, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3018}, {'shards': '0,2', 'objects': 1006}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:25.491 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:26:25.492 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'13392", 'reported_seq': 13518, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:23.845121+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:26:23.845121+0000', 'last_peered': '2024-09-15T21:26:23.845121+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:26:23.845121+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29168099, 'num_objects': 7122, 'num_object_clones': 0, 'num_object_copies': 14244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7122, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11324, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 953}, {'shards': '2', 'objects': 3035}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:25.492 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:25.492 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13260", 'reported_seq': 13392, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:22.660146+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:26:22.660146+0000', 'last_peered': '2024-09-15T21:26:22.660146+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:26:22.660146+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29581312, 'num_objects': 7222, 'num_object_clones': 0, 'num_object_copies': 14444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7222, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9933, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3202}, {'shards': '0,2', 'objects': 1005}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:25.492 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:25.492 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13473", 'reported_seq': 13592, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:23.986684+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:26:23.986684+0000', 'last_peered': '2024-09-15T21:26:23.986684+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:26:23.986684+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29368803, 'num_objects': 7171, 'num_object_clones': 0, 'num_object_copies': 14342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7171, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11332, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 941}, {'shards': '2', 'objects': 3081}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:25.492 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:25.492 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13415", 'reported_seq': 13549, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:22.590719+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:26:22.590719+0000', 'last_peered': '2024-09-15T21:26:22.590719+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:26:22.590719+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29782016, 'num_objects': 7271, 'num_object_clones': 0, 'num_object_copies': 14542, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7271, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10359, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3177}, {'shards': '0,2', 'objects': 1023}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:25.492 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:26:25.492 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13494", 'reported_seq': 13627, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:22.724521+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:26:22.724521+0000', 'last_peered': '2024-09-15T21:26:22.724521+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:26:22.724521+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29065216, 'num_objects': 7096, 'num_object_clones': 0, 'num_object_copies': 14192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10403, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2957}, {'shards': '0,2', 'objects': 944}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:25.492 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:25.492 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13317", 'reported_seq': 19408, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:22.788159+0000', 'last_change': '2024-09-15T21:26:05.146366+0000', 'last_active': '2024-09-15T21:26:22.788159+0000', 'last_peered': '2024-09-15T21:26:22.788159+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:26:22.788159+0000', 'last_undegraded': '2024-09-15T21:26:05.134624+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28962816, 'num_objects': 7071, 'num_object_clones': 0, 'num_object_copies': 14142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 146, 'num_objects_misplaced': 961, 'num_objects_unfound': 0, 'num_objects_dirty': 7071, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9801, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2977, 'num_bytes_recovered': 12185600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5964}, {'shards': '0,2', 'objects': 961}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:25.493 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:26:25.493 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13591", 'reported_seq': 13729, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:22.791544+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:26:22.791544+0000', 'last_peered': '2024-09-15T21:26:22.791544+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:26:22.791544+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29799366, 'num_objects': 7277, 'num_object_clones': 0, 'num_object_copies': 14554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7277, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10295, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3130}, {'shards': '0,2', 'objects': 995}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:25.493 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:26.175 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:26.175 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41371/230296 objects degraded (17.964%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":437,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":115148,"data_bytes":471597833,"bytes_used":2155905024,"bytes_avail":287754387456,"bytes_total":289910292480,"degraded_objects":41371,"degraded_total":230296,"degraded_ratio":0.17964272067252579,"misplaced_objects":961,"misplaced_total":230296,"misplaced_ratio":0.0041728905408691423,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":70633,"recovering_keys_per_sec":0,"num_objects_recovered":207,"num_bytes_recovered":847872,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":450},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:26.195 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:26.195 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:26.771 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:26.773 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:26.795 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":256,"stamp":"2024-09-15T21:26:24.788652+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":471597833,"num_objects":115148,"num_object_clones":0,"num_object_copies":230296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41371,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":115148,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":175311,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7143,"num_bytes_recovered":29237267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2105376,"kb_used_data":1077420,"kb_used_omap":12,"kb_used_meta":1027891,"kb_avail":281010144,"statfs":{"total":289910292480,"available":287754387456,"internally_reserved":0,"allocated":1103278080,"data_stored":1102414540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1052560387},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,5,8,3,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22142976,"num_objects":-5406,"num_object_clones":0,"num_object_copies":-10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":98,"num_objects_misplaced":-305,"num_objects_unfound":0,"num_objects_dirty":-5406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5406,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":847872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003869"},"pg_stats":[{"pgid":"2.f","version":"45'13706","reported_seq":13782,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.726575+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:22.726575+0000","last_peered":"2024-09-15T21:26:22.726575+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:22.726575+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30498816,"num_objects":7446,"num_object_clones":0,"num_object_copies":14892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13706,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3314},{"shards":"0,2","objects":1002}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13461","reported_seq":13567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:23.827016+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:23.827016+0000","last_peered":"2024-09-15T21:26:23.827016+0000","last_clean":"2024-09-15T21:26:23.827016+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:23.827016+0000","last_undegraded":"2024-09-15T21:26:23.827016+0000","last_fullsized":"2024-09-15T21:26:23.827016+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29545414,"num_objects":7215,"num_object_clones":0,"num_object_copies":14430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7215,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11742,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13834","reported_seq":13961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:23.928641+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:23.928641+0000","last_peered":"2024-09-15T21:26:23.928641+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:23.928641+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29782040,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11934,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":988},{"shards":"2","objects":3005}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13240","reported_seq":13311,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:23.916291+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:23.916291+0000","last_peered":"2024-09-15T21:26:23.916291+0000","last_clean":"2024-09-15T21:26:23.916291+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:23.916291+0000","last_undegraded":"2024-09-15T21:26:23.916291+0000","last_fullsized":"2024-09-15T21:26:23.916291+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28975104,"num_objects":7074,"num_object_clones":0,"num_object_copies":14148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13240,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13330","reported_seq":13463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:22.780229+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:22.780229+0000","last_peered":"2024-09-15T21:26:22.780229+0000","last_clean":"2024-09-15T21:26:22.780229+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:22.780229+0000","last_undegraded":"2024-09-15T21:26:22.780229+0000","last_fullsized":"2024-09-15T21:26:22.780229+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29389283,"num_objects":7176,"num_object_clones":0,"num_object_copies":14352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9785,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13409","reported_seq":13546,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.675207+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:22.675207+0000","last_peered":"2024-09-15T21:26:22.675207+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:22.675207+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29598179,"num_objects":7227,"num_object_clones":0,"num_object_copies":14454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7227,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10085,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3151},{"shards":"0,2","objects":987}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13493","reported_seq":13630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.773989+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:22.773989+0000","last_peered":"2024-09-15T21:26:22.773989+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:22.773989+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29397958,"num_objects":7179,"num_object_clones":0,"num_object_copies":14358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9804,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3018},{"shards":"0,2","objects":1006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13604","reported_seq":13733,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:22.763339+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:22.763339+0000","last_peered":"2024-09-15T21:26:22.763339+0000","last_clean":"2024-09-15T21:26:22.763339+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:22.763339+0000","last_undegraded":"2024-09-15T21:26:22.763339+0000","last_fullsized":"2024-09-15T21:26:22.763339+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29753344,"num_objects":7264,"num_object_clones":0,"num_object_copies":14528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10376,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13392","reported_seq":13518,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:23.845121+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:23.845121+0000","last_peered":"2024-09-15T21:26:23.845121+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:23.845121+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29168099,"num_objects":7122,"num_object_clones":0,"num_object_copies":14244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11324,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":953},{"shards":"2","objects":3035}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13260","reported_seq":13392,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.660146+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:22.660146+0000","last_peered":"2024-09-15T21:26:22.660146+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:22.660146+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29581312,"num_objects":7222,"num_object_clones":0,"num_object_copies":14444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9933,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3202},{"shards":"0,2","objects":1005}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13473","reported_seq":13592,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:23.986684+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:23.986684+0000","last_peered":"2024-09-15T21:26:23.986684+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:23.986684+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29368803,"num_objects":7171,"num_object_clones":0,"num_object_copies":14342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11332,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":941},{"shards":"2","objects":3081}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13497","reported_seq":21956,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:23.937792+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:23.937792+0000","last_peered":"2024-09-15T21:26:23.937792+0000","last_clean":"2024-09-15T21:26:23.937792+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:23.937792+0000","last_undegraded":"2024-09-15T21:26:23.937792+0000","last_fullsized":"2024-09-15T21:26:23.937792+0000","mapping_epoch":44,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28930048,"num_objects":7063,"num_object_clones":0,"num_object_copies":14126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11190,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13415","reported_seq":13549,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.590719+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:22.590719+0000","last_peered":"2024-09-15T21:26:22.590719+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:22.590719+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29782016,"num_objects":7271,"num_object_clones":0,"num_object_copies":14542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10359,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3177},{"shards":"0,2","objects":1023}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13494","reported_seq":13627,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.724521+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:22.724521+0000","last_peered":"2024-09-15T21:26:22.724521+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:22.724521+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29065216,"num_objects":7096,"num_object_clones":0,"num_object_copies":14192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10403,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2957},{"shards":"0,2","objects":944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13317","reported_seq":19408,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.788159+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:22.788159+0000","last_peered":"2024-09-15T21:26:22.788159+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:22.788159+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28962816,"num_objects":7071,"num_object_clones":0,"num_object_copies":14142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":146,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":7071,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2977,"num_bytes_recovered":12185600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5964},{"shards":"0,2","objects":961}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13591","reported_seq":13729,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:22.791544+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:22.791544+0000","last_peered":"2024-09-15T21:26:22.791544+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:22.791544+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29799366,"num_objects":7277,"num_object_clones":0,"num_object_copies":14554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7277,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10295,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3130},{"shards":"0,2","objects":995}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":471597814,"num_objects":115146,"num_object_clones":0,"num_object_copies":230292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41371,"num_objects_misplaced":961,"num_objects_unfound":0,"num_objects_dirty":115146,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":175309,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7142,"num_bytes_recovered":29237248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1101475840,"data_stored":1101326789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160816,"ondisk_log_size":160816,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659740,"kb_used_data":361484,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93712100,"statfs":{"total":96636764160,"available":95961190400,"internally_reserved":0,"allocated":370159616,"data_stored":369883969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":818072,"kb_used_data":412996,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93553768,"statfs":{"total":96636764160,"available":95799058432,"internally_reserved":0,"allocated":422907904,"data_stored":422632252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":414772907},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,5,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627564,"kb_used_data":302940,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93744276,"statfs":{"total":96636764160,"available":95994138624,"internally_reserved":0,"allocated":310210560,"data_stored":309898319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,5,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":422514688,"data_stored":422478558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309821440,"data_stored":309748721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369139712,"data_stored":369099510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:29.797 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:30.394 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:30.396 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:30.422 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-09-15T21:26:28.790123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":465367817,"num_objects":113627,"num_object_clones":0,"num_object_copies":227254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41425,"num_objects_misplaced":801,"num_objects_unfound":0,"num_objects_dirty":113627,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":176832,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7249,"num_bytes_recovered":29671443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160940,"ondisk_log_size":160940,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2087308,"kb_used_data":1071832,"kb_used_omap":12,"kb_used_meta":1015411,"kb_avail":281028212,"statfs":{"total":289910292480,"available":287772889088,"internally_reserved":0,"allocated":1097555968,"data_stored":1096692428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1039780867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,2,3,7,7,14,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24797184,"num_objects":-6054,"num_object_clones":0,"num_object_copies":-12108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":152,"num_objects_misplaced":-465,"num_objects_unfound":0,"num_objects_dirty":-6054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6054,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1282048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004357"},"pg_stats":[{"pgid":"2.f","version":"45'13854","reported_seq":13930,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.768034+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:27.768034+0000","last_peered":"2024-09-15T21:26:27.768034+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:27.768034+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29892608,"num_objects":7298,"num_object_clones":0,"num_object_copies":14596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13854,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3166},{"shards":"0,2","objects":854}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13461","reported_seq":13567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:23.827016+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:23.827016+0000","last_peered":"2024-09-15T21:26:23.827016+0000","last_clean":"2024-09-15T21:26:23.827016+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:23.827016+0000","last_undegraded":"2024-09-15T21:26:23.827016+0000","last_fullsized":"2024-09-15T21:26:23.827016+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29545414,"num_objects":7215,"num_object_clones":0,"num_object_copies":14430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7215,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11742,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13834","reported_seq":13961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:23.928641+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:23.928641+0000","last_peered":"2024-09-15T21:26:23.928641+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:23.928641+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29782040,"num_objects":7272,"num_object_clones":0,"num_object_copies":14544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7272,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11934,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":988},{"shards":"2","objects":3005}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13240","reported_seq":13311,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:23.916291+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:23.916291+0000","last_peered":"2024-09-15T21:26:23.916291+0000","last_clean":"2024-09-15T21:26:23.916291+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:23.916291+0000","last_undegraded":"2024-09-15T21:26:23.916291+0000","last_fullsized":"2024-09-15T21:26:23.916291+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28975104,"num_objects":7074,"num_object_clones":0,"num_object_copies":14148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13240,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13491","reported_seq":13623,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:27.873046+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:27.873046+0000","last_peered":"2024-09-15T21:26:27.873046+0000","last_clean":"2024-09-15T21:26:27.873046+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:27.873046+0000","last_undegraded":"2024-09-15T21:26:27.873046+0000","last_fullsized":"2024-09-15T21:26:27.873046+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729827,"num_objects":7015,"num_object_clones":0,"num_object_copies":14030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9946,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13558","reported_seq":13695,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.827492+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:27.827492+0000","last_peered":"2024-09-15T21:26:27.827492+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:27.827492+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28987875,"num_objects":7078,"num_object_clones":0,"num_object_copies":14156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7078,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10234,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3002},{"shards":"0,2","objects":838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13647","reported_seq":13784,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.891737+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:27.891737+0000","last_peered":"2024-09-15T21:26:27.891737+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:27.891737+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28767174,"num_objects":7025,"num_object_clones":0,"num_object_copies":14050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9958,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2864},{"shards":"0,2","objects":852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13753","reported_seq":13882,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:27.869614+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:27.869614+0000","last_peered":"2024-09-15T21:26:27.869614+0000","last_clean":"2024-09-15T21:26:27.869614+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:27.869614+0000","last_undegraded":"2024-09-15T21:26:27.869614+0000","last_fullsized":"2024-09-15T21:26:27.869614+0000","mapping_epoch":34,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29143040,"num_objects":7115,"num_object_clones":0,"num_object_copies":14230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10525,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13392","reported_seq":13518,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:23.845121+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:23.845121+0000","last_peered":"2024-09-15T21:26:23.845121+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:23.845121+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29168099,"num_objects":7122,"num_object_clones":0,"num_object_copies":14244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11324,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":953},{"shards":"2","objects":3035}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13413","reported_seq":13545,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.896015+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:27.896015+0000","last_peered":"2024-09-15T21:26:27.896015+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:27.896015+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10086,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3049},{"shards":"0,2","objects":852}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13473","reported_seq":13592,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:23.986684+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:23.986684+0000","last_peered":"2024-09-15T21:26:23.986684+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:23.986684+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29368803,"num_objects":7171,"num_object_clones":0,"num_object_copies":14342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11332,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":941},{"shards":"2","objects":3081}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13497","reported_seq":21956,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:23.937792+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:23.937792+0000","last_peered":"2024-09-15T21:26:23.937792+0000","last_clean":"2024-09-15T21:26:23.937792+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:23.937792+0000","last_undegraded":"2024-09-15T21:26:23.937792+0000","last_fullsized":"2024-09-15T21:26:23.937792+0000","mapping_epoch":44,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28930048,"num_objects":7063,"num_object_clones":0,"num_object_copies":14126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11190,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13572","reported_seq":13705,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.865543+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:27.865543+0000","last_peered":"2024-09-15T21:26:27.865543+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:27.865543+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29138944,"num_objects":7114,"num_object_clones":0,"num_object_copies":14228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10516,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3020},{"shards":"0,2","objects":866}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13646","reported_seq":13779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.817794+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:27.817794+0000","last_peered":"2024-09-15T21:26:27.817794+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:27.817794+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":6944,"num_object_clones":0,"num_object_copies":13888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10555,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2805},{"shards":"0,2","objects":792}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13475","reported_seq":19778,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.885233+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:27.885233+0000","last_peered":"2024-09-15T21:26:27.885233+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:27.885233+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28315648,"num_objects":6913,"num_object_clones":0,"num_object_copies":13826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":200,"num_objects_misplaced":801,"num_objects_unfound":0,"num_objects_dirty":6913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9959,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3083,"num_bytes_recovered":12619776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5912},{"shards":"0,2","objects":801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13731","reported_seq":13870,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.857073+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:27.857073+0000","last_peered":"2024-09-15T21:26:27.857073+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:27.857073+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29225926,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10435,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2990},{"shards":"0,2","objects":855}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":465367798,"num_objects":113625,"num_object_clones":0,"num_object_copies":227250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41425,"num_objects_misplaced":801,"num_objects_unfound":0,"num_objects_dirty":113625,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":176830,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7248,"num_bytes_recovered":29671424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1094238208,"data_stored":1094089157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160937,"ondisk_log_size":160937,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659740,"kb_used_data":361484,"kb_used_omap":4,"kb_used_meta":298235,"kb_avail":93712100,"statfs":{"total":96636764160,"available":95961190400,"internally_reserved":0,"allocated":370159616,"data_stored":369883969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":305393321},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":800004,"kb_used_data":407408,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93571836,"statfs":{"total":96636764160,"available":95817560064,"internally_reserved":0,"allocated":417185792,"data_stored":416910140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,3,1,4,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627564,"kb_used_data":302940,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93744276,"statfs":{"total":96636764160,"available":95994138624,"internally_reserved":0,"allocated":310210560,"data_stored":309898319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332394159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,5,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":415277056,"data_stored":415240926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309821440,"data_stored":309748721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369139712,"data_stored":369099510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:30.424 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:30.988 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:30.990 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:31.019 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-15T21:26:30.790821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":461976329,"num_objects":112799,"num_object_clones":0,"num_object_copies":225598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41425,"num_objects_misplaced":801,"num_objects_unfound":0,"num_objects_dirty":112799,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":177660,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7249,"num_bytes_recovered":29671443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2094820,"kb_used_data":1066856,"kb_used_omap":12,"kb_used_meta":1027891,"kb_avail":281020700,"statfs":{"total":289910292480,"available":287765196800,"internally_reserved":0,"allocated":1092460544,"data_stored":1091597004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1052560387},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,9,3,7,5,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22413312,"num_objects":-5472,"num_object_clones":0,"num_object_copies":-10944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":114,"num_objects_misplaced":-320,"num_objects_unfound":0,"num_objects_dirty":-5472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5472,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003558"},"pg_stats":[{"pgid":"2.f","version":"45'13854","reported_seq":13930,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.768034+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:27.768034+0000","last_peered":"2024-09-15T21:26:27.768034+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:27.768034+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29892608,"num_objects":7298,"num_object_clones":0,"num_object_copies":14596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13854,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3166},{"shards":"0,2","objects":854}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13589","reported_seq":13695,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:28.896273+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:28.896273+0000","last_peered":"2024-09-15T21:26:28.896273+0000","last_clean":"2024-09-15T21:26:28.896273+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:28.896273+0000","last_undegraded":"2024-09-15T21:26:28.896273+0000","last_fullsized":"2024-09-15T21:26:28.896273+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29021126,"num_objects":7087,"num_object_clones":0,"num_object_copies":14174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7087,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11870,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13995","reported_seq":14122,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:28.964423+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:28.964423+0000","last_peered":"2024-09-15T21:26:28.964423+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:28.964423+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29122584,"num_objects":7111,"num_object_clones":0,"num_object_copies":14222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12095,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":827},{"shards":"2","objects":2844}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13392","reported_seq":13463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:28.974031+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:28.974031+0000","last_peered":"2024-09-15T21:26:28.974031+0000","last_clean":"2024-09-15T21:26:28.974031+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:28.974031+0000","last_undegraded":"2024-09-15T21:26:28.974031+0000","last_fullsized":"2024-09-15T21:26:28.974031+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28352512,"num_objects":6922,"num_object_clones":0,"num_object_copies":13844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13392,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13491","reported_seq":13623,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:27.873046+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:27.873046+0000","last_peered":"2024-09-15T21:26:27.873046+0000","last_clean":"2024-09-15T21:26:27.873046+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:27.873046+0000","last_undegraded":"2024-09-15T21:26:27.873046+0000","last_fullsized":"2024-09-15T21:26:27.873046+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729827,"num_objects":7015,"num_object_clones":0,"num_object_copies":14030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9946,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13558","reported_seq":13695,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.827492+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:27.827492+0000","last_peered":"2024-09-15T21:26:27.827492+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:27.827492+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28987875,"num_objects":7078,"num_object_clones":0,"num_object_copies":14156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7078,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10234,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3002},{"shards":"0,2","objects":838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13647","reported_seq":13784,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.891737+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:27.891737+0000","last_peered":"2024-09-15T21:26:27.891737+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:27.891737+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28767174,"num_objects":7025,"num_object_clones":0,"num_object_copies":14050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9958,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2864},{"shards":"0,2","objects":852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13753","reported_seq":13882,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:27.869614+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:27.869614+0000","last_peered":"2024-09-15T21:26:27.869614+0000","last_clean":"2024-09-15T21:26:27.869614+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:27.869614+0000","last_undegraded":"2024-09-15T21:26:27.869614+0000","last_fullsized":"2024-09-15T21:26:27.869614+0000","mapping_epoch":34,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29143040,"num_objects":7115,"num_object_clones":0,"num_object_copies":14230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10525,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13519","reported_seq":13646,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:28.932869+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:28.932869+0000","last_peered":"2024-09-15T21:26:28.932869+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:28.932869+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28647907,"num_objects":6995,"num_object_clones":0,"num_object_copies":13990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6995,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11451,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":826},{"shards":"2","objects":2908}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13413","reported_seq":13545,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.896015+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:27.896015+0000","last_peered":"2024-09-15T21:26:27.896015+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:27.896015+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10086,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3049},{"shards":"0,2","objects":852}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13612","reported_seq":13731,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:28.895183+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:28.895183+0000","last_peered":"2024-09-15T21:26:28.895183+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:28.895183+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28799459,"num_objects":7032,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7032,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11471,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":802},{"shards":"2","objects":2942}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13618","reported_seq":22077,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:28.846689+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:28.846689+0000","last_peered":"2024-09-15T21:26:28.846689+0000","last_clean":"2024-09-15T21:26:28.846689+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:28.846689+0000","last_undegraded":"2024-09-15T21:26:28.846689+0000","last_fullsized":"2024-09-15T21:26:28.846689+0000","mapping_epoch":44,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28434432,"num_objects":6942,"num_object_clones":0,"num_object_copies":13884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11311,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13572","reported_seq":13705,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.865543+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:27.865543+0000","last_peered":"2024-09-15T21:26:27.865543+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:27.865543+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29138944,"num_objects":7114,"num_object_clones":0,"num_object_copies":14228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10516,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3020},{"shards":"0,2","objects":866}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13646","reported_seq":13779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.817794+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:27.817794+0000","last_peered":"2024-09-15T21:26:27.817794+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:27.817794+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":6944,"num_object_clones":0,"num_object_copies":13888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10555,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2805},{"shards":"0,2","objects":792}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13475","reported_seq":19778,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.885233+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:27.885233+0000","last_peered":"2024-09-15T21:26:27.885233+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:27.885233+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28315648,"num_objects":6913,"num_object_clones":0,"num_object_copies":13826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":200,"num_objects_misplaced":801,"num_objects_unfound":0,"num_objects_dirty":6913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9959,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3083,"num_bytes_recovered":12619776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5912},{"shards":"0,2","objects":801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13731","reported_seq":13870,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.857073+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:27.857073+0000","last_peered":"2024-09-15T21:26:27.857073+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:27.857073+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29225926,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10435,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2990},{"shards":"0,2","objects":855}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":461976310,"num_objects":112797,"num_object_clones":0,"num_object_copies":225594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41425,"num_objects_misplaced":801,"num_objects_unfound":0,"num_objects_dirty":112797,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":177658,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7248,"num_bytes_recovered":29671424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1089126400,"data_stored":1088977349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659452,"kb_used_data":357028,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93712388,"statfs":{"total":96636764160,"available":95961485312,"internally_reserved":0,"allocated":365596672,"data_stored":365321025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":800004,"kb_used_data":407408,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93571836,"statfs":{"total":96636764160,"available":95817560064,"internally_reserved":0,"allocated":417185792,"data_stored":416910140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,3,1,4,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635364,"kb_used_data":302420,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93736476,"statfs":{"total":96636764160,"available":95986151424,"internally_reserved":0,"allocated":309678080,"data_stored":309365839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":340913839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":415277056,"data_stored":415240926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309284864,"data_stored":309212145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":364564480,"data_stored":364524278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:31.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:31.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'13854", 'reported_seq': 13930, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:27.768034+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:26:27.768034+0000', 'last_peered': '2024-09-15T21:26:27.768034+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:26:27.768034+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29892608, 'num_objects': 7298, 'num_object_clones': 0, 'num_object_copies': 14596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7298, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13854, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3166}, {'shards': '0,2', 'objects': 854}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:31.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:26:31.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'13995", 'reported_seq': 14122, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:28.964423+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:26:28.964423+0000', 'last_peered': '2024-09-15T21:26:28.964423+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:26:28.964423+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "39'3900", 'ondisk_log_start': "39'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29122584, 'num_objects': 7111, 'num_object_clones': 0, 'num_object_copies': 14222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7111, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12095, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 827}, {'shards': '2', 'objects': 2844}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:31.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:31.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13558", 'reported_seq': 13695, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:27.827492+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:26:27.827492+0000', 'last_peered': '2024-09-15T21:26:27.827492+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:26:27.827492+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28987875, 'num_objects': 7078, 'num_object_clones': 0, 'num_object_copies': 14156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7078, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10234, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3002}, {'shards': '0,2', 'objects': 838}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:31.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:31.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13647", 'reported_seq': 13784, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:27.891737+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:26:27.891737+0000', 'last_peered': '2024-09-15T21:26:27.891737+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:26:27.891737+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28767174, 'num_objects': 7025, 'num_object_clones': 0, 'num_object_copies': 14050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7025, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9958, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2864}, {'shards': '0,2', 'objects': 852}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:31.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:26:31.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'13519", 'reported_seq': 13646, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:28.932869+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:26:28.932869+0000', 'last_peered': '2024-09-15T21:26:28.932869+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:26:28.932869+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28647907, 'num_objects': 6995, 'num_object_clones': 0, 'num_object_copies': 13990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6995, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11451, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 826}, {'shards': '2', 'objects': 2908}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:31.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:31.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13413", 'reported_seq': 13545, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:27.896015+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:26:27.896015+0000', 'last_peered': '2024-09-15T21:26:27.896015+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:26:27.896015+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28954624, 'num_objects': 7069, 'num_object_clones': 0, 'num_object_copies': 14138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7069, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10086, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3049}, {'shards': '0,2', 'objects': 852}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:31.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:31.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13612", 'reported_seq': 13731, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:28.895183+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:26:28.895183+0000', 'last_peered': '2024-09-15T21:26:28.895183+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:26:28.895183+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28799459, 'num_objects': 7032, 'num_object_clones': 0, 'num_object_copies': 14064, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7032, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11471, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 802}, {'shards': '2', 'objects': 2942}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:31.021 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:31.021 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13572", 'reported_seq': 13705, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:27.865543+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:26:27.865543+0000', 'last_peered': '2024-09-15T21:26:27.865543+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:26:27.865543+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29138944, 'num_objects': 7114, 'num_object_clones': 0, 'num_object_copies': 14228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7114, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10516, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3020}, {'shards': '0,2', 'objects': 866}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:31.022 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:26:31.022 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13646", 'reported_seq': 13779, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:27.817794+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:26:27.817794+0000', 'last_peered': '2024-09-15T21:26:27.817794+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:26:27.817794+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28442624, 'num_objects': 6944, 'num_object_clones': 0, 'num_object_copies': 13888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6944, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10555, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2805}, {'shards': '0,2', 'objects': 792}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:31.022 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:31.022 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13475", 'reported_seq': 19778, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:27.885233+0000', 'last_change': '2024-09-15T21:26:05.146366+0000', 'last_active': '2024-09-15T21:26:27.885233+0000', 'last_peered': '2024-09-15T21:26:27.885233+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:26:27.885233+0000', 'last_undegraded': '2024-09-15T21:26:05.134624+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28315648, 'num_objects': 6913, 'num_object_clones': 0, 'num_object_copies': 13826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 200, 'num_objects_misplaced': 801, 'num_objects_unfound': 0, 'num_objects_dirty': 6913, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9959, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3083, 'num_bytes_recovered': 12619776, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5912}, {'shards': '0,2', 'objects': 801}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:31.022 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:26:31.022 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13731", 'reported_seq': 13870, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:27.857073+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:26:27.857073+0000', 'last_peered': '2024-09-15T21:26:27.857073+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:26:27.857073+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29225926, 'num_objects': 7137, 'num_object_clones': 0, 'num_object_copies': 14274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7137, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10435, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2990}, {'shards': '0,2', 'objects': 855}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:31.022 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:31.727 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:31.727 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41425/225598 objects degraded (18.362%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":443,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":112799,"data_bytes":461976329,"bytes_used":2145095680,"bytes_avail":287765196800,"bytes_total":289910292480,"degraded_objects":41425,"degraded_total":225598,"degraded_ratio":0.18362308176490927,"misplaced_objects":801,"misplaced_total":225598,"misplaced_ratio":0.0035505633915194285,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":70293,"recovering_keys_per_sec":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":455},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:31.758 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:31.758 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:32.318 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:32.321 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:32.343 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-15T21:26:30.790821+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":461976329,"num_objects":112799,"num_object_clones":0,"num_object_copies":225598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41425,"num_objects_misplaced":801,"num_objects_unfound":0,"num_objects_dirty":112799,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":177660,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7249,"num_bytes_recovered":29671443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160868,"ondisk_log_size":160868,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2094820,"kb_used_data":1066856,"kb_used_omap":12,"kb_used_meta":1027891,"kb_avail":281020700,"statfs":{"total":289910292480,"available":287765196800,"internally_reserved":0,"allocated":1092460544,"data_stored":1091597004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1052560387},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,9,3,7,5,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22413312,"num_objects":-5472,"num_object_clones":0,"num_object_copies":-10944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":114,"num_objects_misplaced":-320,"num_objects_unfound":0,"num_objects_dirty":-5472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5472,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003558"},"pg_stats":[{"pgid":"2.f","version":"45'13854","reported_seq":13930,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.768034+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:27.768034+0000","last_peered":"2024-09-15T21:26:27.768034+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:27.768034+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29892608,"num_objects":7298,"num_object_clones":0,"num_object_copies":14596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13854,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3166},{"shards":"0,2","objects":854}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13589","reported_seq":13695,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:28.896273+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:28.896273+0000","last_peered":"2024-09-15T21:26:28.896273+0000","last_clean":"2024-09-15T21:26:28.896273+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:28.896273+0000","last_undegraded":"2024-09-15T21:26:28.896273+0000","last_fullsized":"2024-09-15T21:26:28.896273+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29021126,"num_objects":7087,"num_object_clones":0,"num_object_copies":14174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7087,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":11870,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'13995","reported_seq":14122,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:28.964423+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:28.964423+0000","last_peered":"2024-09-15T21:26:28.964423+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:28.964423+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29122584,"num_objects":7111,"num_object_clones":0,"num_object_copies":14222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12095,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":827},{"shards":"2","objects":2844}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13392","reported_seq":13463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:28.974031+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:28.974031+0000","last_peered":"2024-09-15T21:26:28.974031+0000","last_clean":"2024-09-15T21:26:28.974031+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:28.974031+0000","last_undegraded":"2024-09-15T21:26:28.974031+0000","last_fullsized":"2024-09-15T21:26:28.974031+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28352512,"num_objects":6922,"num_object_clones":0,"num_object_copies":13844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13392,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13491","reported_seq":13623,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:27.873046+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:27.873046+0000","last_peered":"2024-09-15T21:26:27.873046+0000","last_clean":"2024-09-15T21:26:27.873046+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:27.873046+0000","last_undegraded":"2024-09-15T21:26:27.873046+0000","last_fullsized":"2024-09-15T21:26:27.873046+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729827,"num_objects":7015,"num_object_clones":0,"num_object_copies":14030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9946,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13558","reported_seq":13695,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.827492+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:27.827492+0000","last_peered":"2024-09-15T21:26:27.827492+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:27.827492+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28987875,"num_objects":7078,"num_object_clones":0,"num_object_copies":14156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7078,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10234,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3002},{"shards":"0,2","objects":838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13647","reported_seq":13784,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.891737+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:27.891737+0000","last_peered":"2024-09-15T21:26:27.891737+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:27.891737+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28767174,"num_objects":7025,"num_object_clones":0,"num_object_copies":14050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9958,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2864},{"shards":"0,2","objects":852}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13753","reported_seq":13882,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:27.869614+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:27.869614+0000","last_peered":"2024-09-15T21:26:27.869614+0000","last_clean":"2024-09-15T21:26:27.869614+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:27.869614+0000","last_undegraded":"2024-09-15T21:26:27.869614+0000","last_fullsized":"2024-09-15T21:26:27.869614+0000","mapping_epoch":34,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29143040,"num_objects":7115,"num_object_clones":0,"num_object_copies":14230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10525,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13519","reported_seq":13646,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:28.932869+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:28.932869+0000","last_peered":"2024-09-15T21:26:28.932869+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:28.932869+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28647907,"num_objects":6995,"num_object_clones":0,"num_object_copies":13990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6995,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11451,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":826},{"shards":"2","objects":2908}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.888093+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:23:35.888093+0000","last_peered":"2024-09-15T21:23:35.888093+0000","last_clean":"2024-09-15T21:23:35.888093+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:23:35.888093+0000","last_undegraded":"2024-09-15T21:23:35.888093+0000","last_fullsized":"2024-09-15T21:23:35.888093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13413","reported_seq":13545,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.896015+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:27.896015+0000","last_peered":"2024-09-15T21:26:27.896015+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:27.896015+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10086,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3049},{"shards":"0,2","objects":852}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13612","reported_seq":13731,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:28.895183+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:28.895183+0000","last_peered":"2024-09-15T21:26:28.895183+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:28.895183+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28799459,"num_objects":7032,"num_object_clones":0,"num_object_copies":14064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7032,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11471,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":802},{"shards":"2","objects":2942}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13618","reported_seq":22077,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:28.846689+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:28.846689+0000","last_peered":"2024-09-15T21:26:28.846689+0000","last_clean":"2024-09-15T21:26:28.846689+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:28.846689+0000","last_undegraded":"2024-09-15T21:26:28.846689+0000","last_fullsized":"2024-09-15T21:26:28.846689+0000","mapping_epoch":44,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28434432,"num_objects":6942,"num_object_clones":0,"num_object_copies":13884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11311,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13572","reported_seq":13705,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.865543+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:27.865543+0000","last_peered":"2024-09-15T21:26:27.865543+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:27.865543+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29138944,"num_objects":7114,"num_object_clones":0,"num_object_copies":14228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10516,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3020},{"shards":"0,2","objects":866}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13646","reported_seq":13779,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.817794+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:27.817794+0000","last_peered":"2024-09-15T21:26:27.817794+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:27.817794+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28442624,"num_objects":6944,"num_object_clones":0,"num_object_copies":13888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10555,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2805},{"shards":"0,2","objects":792}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.891318+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:23:35.891318+0000","last_peered":"2024-09-15T21:23:35.891318+0000","last_clean":"2024-09-15T21:23:35.891318+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:23:35.891318+0000","last_undegraded":"2024-09-15T21:23:35.891318+0000","last_fullsized":"2024-09-15T21:23:35.891318+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13475","reported_seq":19778,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.885233+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:27.885233+0000","last_peered":"2024-09-15T21:26:27.885233+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:27.885233+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28315648,"num_objects":6913,"num_object_clones":0,"num_object_copies":13826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":200,"num_objects_misplaced":801,"num_objects_unfound":0,"num_objects_dirty":6913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9959,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3083,"num_bytes_recovered":12619776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5912},{"shards":"0,2","objects":801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13731","reported_seq":13870,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:27.857073+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:27.857073+0000","last_peered":"2024-09-15T21:26:27.857073+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:27.857073+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29225926,"num_objects":7137,"num_object_clones":0,"num_object_copies":14274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7137,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10435,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2990},{"shards":"0,2","objects":855}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":461976310,"num_objects":112797,"num_object_clones":0,"num_object_copies":225594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41425,"num_objects_misplaced":801,"num_objects_unfound":0,"num_objects_dirty":112797,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":177658,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7248,"num_bytes_recovered":29671424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1089126400,"data_stored":1088977349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659452,"kb_used_data":357028,"kb_used_omap":4,"kb_used_meta":302395,"kb_avail":93712388,"statfs":{"total":96636764160,"available":95961485312,"internally_reserved":0,"allocated":365596672,"data_stored":365321025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":309653161},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":800004,"kb_used_data":407408,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93571836,"statfs":{"total":96636764160,"available":95817560064,"internally_reserved":0,"allocated":417185792,"data_stored":416910140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,3,1,4,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":635364,"kb_used_data":302420,"kb_used_omap":4,"kb_used_meta":332923,"kb_avail":93736476,"statfs":{"total":96636764160,"available":95986151424,"internally_reserved":0,"allocated":309678080,"data_stored":309365839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":340913839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":415277056,"data_stored":415240926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309284864,"data_stored":309212145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":364564480,"data_stored":364524278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:35.347 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:35.887 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:35.888 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:35.923 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-15T21:26:34.792582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":452600585,"num_objects":110510,"num_object_clones":0,"num_object_copies":221020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41452,"num_objects_misplaced":660,"num_objects_unfound":0,"num_objects_dirty":110510,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":179949,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7363,"num_bytes_recovered":30138387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160757,"ondisk_log_size":160757,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2077300,"kb_used_data":1053496,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":281038220,"statfs":{"total":289910292480,"available":287783137280,"internally_reserved":0,"allocated":1078779904,"data_stored":1077916364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1048300547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,5,5,12,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28286976,"num_objects":-6906,"num_object_clones":0,"num_object_copies":-13812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":141,"num_objects_misplaced":-461,"num_objects_unfound":0,"num_objects_dirty":-6906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6906,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004345"},"pg_stats":[{"pgid":"2.f","version":"45'14005","reported_seq":14081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.874443+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:32.874443+0000","last_peered":"2024-09-15T21:26:32.874443+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:32.874443+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29274112,"num_objects":7147,"num_object_clones":0,"num_object_copies":14294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14005,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3015},{"shards":"0,2","objects":703}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13724","reported_seq":13830,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:33.806706+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:33.806706+0000","last_peered":"2024-09-15T21:26:33.806706+0000","last_clean":"2024-09-15T21:26:33.806706+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:33.806706+0000","last_undegraded":"2024-09-15T21:26:33.806706+0000","last_fullsized":"2024-09-15T21:26:33.806706+0000","mapping_epoch":35,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28468166,"num_objects":6952,"num_object_clones":0,"num_object_copies":13904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12005,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14143","reported_seq":14270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:33.999673+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:33.999673+0000","last_peered":"2024-09-15T21:26:33.999673+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:33.999673+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28516376,"num_objects":6963,"num_object_clones":0,"num_object_copies":13926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12243,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":679},{"shards":"2","objects":2696}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13544","reported_seq":13615,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:34.003291+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:34.003291+0000","last_peered":"2024-09-15T21:26:34.003291+0000","last_clean":"2024-09-15T21:26:34.003291+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:34.003291+0000","last_undegraded":"2024-09-15T21:26:34.003291+0000","last_fullsized":"2024-09-15T21:26:34.003291+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13544,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13620","reported_seq":13753,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:32.886665+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:32.886665+0000","last_peered":"2024-09-15T21:26:32.886665+0000","last_clean":"2024-09-15T21:26:32.886665+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:32.886665+0000","last_undegraded":"2024-09-15T21:26:32.886665+0000","last_fullsized":"2024-09-15T21:26:32.886665+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28201443,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10075,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13713","reported_seq":13848,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.943778+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:32.943778+0000","last_peered":"2024-09-15T21:26:32.943778+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:32.943778+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28352995,"num_objects":6923,"num_object_clones":0,"num_object_copies":13846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6923,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10389,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2847},{"shards":"0,2","objects":683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13796","reported_seq":13933,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.941782+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:32.941782+0000","last_peered":"2024-09-15T21:26:32.941782+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:32.941782+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28156870,"num_objects":6876,"num_object_clones":0,"num_object_copies":13752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10107,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2715},{"shards":"0,2","objects":703}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13898","reported_seq":14027,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:32.911127+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:32.911127+0000","last_peered":"2024-09-15T21:26:32.911127+0000","last_clean":"2024-09-15T21:26:32.911127+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:32.911127+0000","last_undegraded":"2024-09-15T21:26:32.911127+0000","last_fullsized":"2024-09-15T21:26:32.911127+0000","mapping_epoch":34,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28549120,"num_objects":6970,"num_object_clones":0,"num_object_copies":13940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10670,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13666","reported_seq":13793,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:33.989698+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:33.989698+0000","last_peered":"2024-09-15T21:26:33.989698+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:33.989698+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28045795,"num_objects":6848,"num_object_clones":0,"num_object_copies":13696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11598,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":679},{"shards":"2","objects":2761}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13537","reported_seq":13669,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.933916+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:32.933916+0000","last_peered":"2024-09-15T21:26:32.933916+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:32.933916+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28446720,"num_objects":6945,"num_object_clones":0,"num_object_copies":13890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10210,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2925},{"shards":"0,2","objects":728}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13744","reported_seq":13864,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:33.949200+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:33.949200+0000","last_peered":"2024-09-15T21:26:33.949200+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:33.949200+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28258787,"num_objects":6900,"num_object_clones":0,"num_object_copies":13800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6900,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11603,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":670},{"shards":"2","objects":2810}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13779","reported_seq":22238,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:33.900117+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:33.900117+0000","last_peered":"2024-09-15T21:26:33.900117+0000","last_clean":"2024-09-15T21:26:33.900117+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:33.900117+0000","last_undegraded":"2024-09-15T21:26:33.900117+0000","last_fullsized":"2024-09-15T21:26:33.900117+0000","mapping_epoch":44,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27774976,"num_objects":6781,"num_object_clones":0,"num_object_copies":13562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11472,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13716","reported_seq":13850,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.922164+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:32.922164+0000","last_peered":"2024-09-15T21:26:32.922164+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:32.922164+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28549120,"num_objects":6970,"num_object_clones":0,"num_object_copies":13940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10660,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2876},{"shards":"0,2","objects":722}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13794","reported_seq":13927,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.913148+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:32.913148+0000","last_peered":"2024-09-15T21:26:32.913148+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:32.913148+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27836416,"num_objects":6796,"num_object_clones":0,"num_object_copies":13592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10703,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2657},{"shards":"0,2","objects":644}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13618","reported_seq":20147,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.913653+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:32.913653+0000","last_peered":"2024-09-15T21:26:32.913653+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:32.913653+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":227,"num_objects_misplaced":660,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10102,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3197,"num_bytes_recovered":13086720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5883},{"shards":"0,2","objects":660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13857","reported_seq":13997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.944135+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:32.944135+0000","last_peered":"2024-09-15T21:26:32.944135+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:32.944135+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28709830,"num_objects":7011,"num_object_clones":0,"num_object_copies":14022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7011,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10561,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2864},{"shards":"0,2","objects":729}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":452600566,"num_objects":110508,"num_object_clones":0,"num_object_copies":221016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41452,"num_objects_misplaced":660,"num_objects_unfound":0,"num_objects_dirty":110508,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":179947,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7362,"num_bytes_recovered":30138368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1076736000,"data_stored":1076586949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659048,"kb_used_data":352472,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93712792,"statfs":{"total":96636764160,"available":95961899008,"internally_reserved":0,"allocated":360931328,"data_stored":360655681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":795984,"kb_used_data":399224,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93575856,"statfs":{"total":96636764160,"available":95821676544,"internally_reserved":0,"allocated":408805376,"data_stored":408529724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622268,"kb_used_data":301800,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93749572,"statfs":{"total":96636764160,"available":95999561728,"internally_reserved":0,"allocated":309043200,"data_stored":308730959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328134319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":408260608,"data_stored":408224478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308621312,"data_stored":308548593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":359854080,"data_stored":359813878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:35.924 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:36.461 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:36.461 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:36.492 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-15T21:26:34.792582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":452600585,"num_objects":110510,"num_object_clones":0,"num_object_copies":221020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41452,"num_objects_misplaced":660,"num_objects_unfound":0,"num_objects_dirty":110510,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":179949,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7363,"num_bytes_recovered":30138387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160757,"ondisk_log_size":160757,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2077300,"kb_used_data":1053496,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":281038220,"statfs":{"total":289910292480,"available":287783137280,"internally_reserved":0,"allocated":1078779904,"data_stored":1077916364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1048300547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,5,5,12,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28286976,"num_objects":-6906,"num_object_clones":0,"num_object_copies":-13812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":141,"num_objects_misplaced":-461,"num_objects_unfound":0,"num_objects_dirty":-6906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6906,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":1310720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004345"},"pg_stats":[{"pgid":"2.f","version":"45'14005","reported_seq":14081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.874443+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:32.874443+0000","last_peered":"2024-09-15T21:26:32.874443+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:32.874443+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29274112,"num_objects":7147,"num_object_clones":0,"num_object_copies":14294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14005,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3015},{"shards":"0,2","objects":703}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13724","reported_seq":13830,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:33.806706+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:33.806706+0000","last_peered":"2024-09-15T21:26:33.806706+0000","last_clean":"2024-09-15T21:26:33.806706+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:33.806706+0000","last_undegraded":"2024-09-15T21:26:33.806706+0000","last_fullsized":"2024-09-15T21:26:33.806706+0000","mapping_epoch":35,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28468166,"num_objects":6952,"num_object_clones":0,"num_object_copies":13904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12005,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14143","reported_seq":14270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:33.999673+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:33.999673+0000","last_peered":"2024-09-15T21:26:33.999673+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:33.999673+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28516376,"num_objects":6963,"num_object_clones":0,"num_object_copies":13926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12243,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":679},{"shards":"2","objects":2696}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13544","reported_seq":13615,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:34.003291+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:34.003291+0000","last_peered":"2024-09-15T21:26:34.003291+0000","last_clean":"2024-09-15T21:26:34.003291+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:34.003291+0000","last_undegraded":"2024-09-15T21:26:34.003291+0000","last_fullsized":"2024-09-15T21:26:34.003291+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13544,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13620","reported_seq":13753,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:32.886665+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:32.886665+0000","last_peered":"2024-09-15T21:26:32.886665+0000","last_clean":"2024-09-15T21:26:32.886665+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:32.886665+0000","last_undegraded":"2024-09-15T21:26:32.886665+0000","last_fullsized":"2024-09-15T21:26:32.886665+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28201443,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10075,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13713","reported_seq":13848,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.943778+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:32.943778+0000","last_peered":"2024-09-15T21:26:32.943778+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:32.943778+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28352995,"num_objects":6923,"num_object_clones":0,"num_object_copies":13846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6923,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10389,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2847},{"shards":"0,2","objects":683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13796","reported_seq":13933,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.941782+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:32.941782+0000","last_peered":"2024-09-15T21:26:32.941782+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:32.941782+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28156870,"num_objects":6876,"num_object_clones":0,"num_object_copies":13752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10107,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2715},{"shards":"0,2","objects":703}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13898","reported_seq":14027,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:32.911127+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:32.911127+0000","last_peered":"2024-09-15T21:26:32.911127+0000","last_clean":"2024-09-15T21:26:32.911127+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:32.911127+0000","last_undegraded":"2024-09-15T21:26:32.911127+0000","last_fullsized":"2024-09-15T21:26:32.911127+0000","mapping_epoch":34,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28549120,"num_objects":6970,"num_object_clones":0,"num_object_copies":13940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10670,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13666","reported_seq":13793,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:33.989698+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:33.989698+0000","last_peered":"2024-09-15T21:26:33.989698+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:33.989698+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28045795,"num_objects":6848,"num_object_clones":0,"num_object_copies":13696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11598,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":679},{"shards":"2","objects":2761}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13537","reported_seq":13669,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.933916+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:32.933916+0000","last_peered":"2024-09-15T21:26:32.933916+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:32.933916+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28446720,"num_objects":6945,"num_object_clones":0,"num_object_copies":13890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10210,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2925},{"shards":"0,2","objects":728}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13744","reported_seq":13864,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:33.949200+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:33.949200+0000","last_peered":"2024-09-15T21:26:33.949200+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:33.949200+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28258787,"num_objects":6900,"num_object_clones":0,"num_object_copies":13800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6900,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11603,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":670},{"shards":"2","objects":2810}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13779","reported_seq":22238,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:33.900117+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:33.900117+0000","last_peered":"2024-09-15T21:26:33.900117+0000","last_clean":"2024-09-15T21:26:33.900117+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:33.900117+0000","last_undegraded":"2024-09-15T21:26:33.900117+0000","last_fullsized":"2024-09-15T21:26:33.900117+0000","mapping_epoch":44,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27774976,"num_objects":6781,"num_object_clones":0,"num_object_copies":13562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11472,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13716","reported_seq":13850,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.922164+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:32.922164+0000","last_peered":"2024-09-15T21:26:32.922164+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:32.922164+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28549120,"num_objects":6970,"num_object_clones":0,"num_object_copies":13940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10660,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2876},{"shards":"0,2","objects":722}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13794","reported_seq":13927,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.913148+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:32.913148+0000","last_peered":"2024-09-15T21:26:32.913148+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:32.913148+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27836416,"num_objects":6796,"num_object_clones":0,"num_object_copies":13592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10703,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2657},{"shards":"0,2","objects":644}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13618","reported_seq":20147,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.913653+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:32.913653+0000","last_peered":"2024-09-15T21:26:32.913653+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:32.913653+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":227,"num_objects_misplaced":660,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10102,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3197,"num_bytes_recovered":13086720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5883},{"shards":"0,2","objects":660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13857","reported_seq":13997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.944135+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:32.944135+0000","last_peered":"2024-09-15T21:26:32.944135+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:32.944135+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28709830,"num_objects":7011,"num_object_clones":0,"num_object_copies":14022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7011,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10561,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2864},{"shards":"0,2","objects":729}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":452600566,"num_objects":110508,"num_object_clones":0,"num_object_copies":221016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41452,"num_objects_misplaced":660,"num_objects_unfound":0,"num_objects_dirty":110508,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":179947,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7362,"num_bytes_recovered":30138368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1076736000,"data_stored":1076586949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659048,"kb_used_data":352472,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93712792,"statfs":{"total":96636764160,"available":95961899008,"internally_reserved":0,"allocated":360931328,"data_stored":360655681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":795984,"kb_used_data":399224,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93575856,"statfs":{"total":96636764160,"available":95821676544,"internally_reserved":0,"allocated":408805376,"data_stored":408529724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622268,"kb_used_data":301800,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93749572,"statfs":{"total":96636764160,"available":95999561728,"internally_reserved":0,"allocated":309043200,"data_stored":308730959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328134319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":408260608,"data_stored":408224478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308621312,"data_stored":308548593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":359854080,"data_stored":359813878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:36.493 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:36.493 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14005", 'reported_seq': 14081, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:32.874443+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:26:32.874443+0000', 'last_peered': '2024-09-15T21:26:32.874443+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:26:32.874443+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "40'4000", 'ondisk_log_start': "40'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29274112, 'num_objects': 7147, 'num_object_clones': 0, 'num_object_copies': 14294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7147, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14005, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3015}, {'shards': '0,2', 'objects': 703}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:36.493 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:26:36.493 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'14143", 'reported_seq': 14270, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:33.999673+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:26:33.999673+0000', 'last_peered': '2024-09-15T21:26:33.999673+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:26:33.999673+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "39'4100", 'ondisk_log_start': "39'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28516376, 'num_objects': 6963, 'num_object_clones': 0, 'num_object_copies': 13926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6963, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12243, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 679}, {'shards': '2', 'objects': 2696}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:36.494 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:36.494 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13713", 'reported_seq': 13848, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:32.943778+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:26:32.943778+0000', 'last_peered': '2024-09-15T21:26:32.943778+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:26:32.943778+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "39'3700", 'ondisk_log_start': "39'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28352995, 'num_objects': 6923, 'num_object_clones': 0, 'num_object_copies': 13846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6923, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10389, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2847}, {'shards': '0,2', 'objects': 683}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:36.494 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:36.494 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13796", 'reported_seq': 13933, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:32.941782+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:26:32.941782+0000', 'last_peered': '2024-09-15T21:26:32.941782+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:26:32.941782+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28156870, 'num_objects': 6876, 'num_object_clones': 0, 'num_object_copies': 13752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6876, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10107, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2715}, {'shards': '0,2', 'objects': 703}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:36.494 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:26:36.494 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'13666", 'reported_seq': 13793, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:33.989698+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:26:33.989698+0000', 'last_peered': '2024-09-15T21:26:33.989698+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:26:33.989698+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28045795, 'num_objects': 6848, 'num_object_clones': 0, 'num_object_copies': 13696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6848, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11598, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 679}, {'shards': '2', 'objects': 2761}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:36.494 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:36.494 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13537", 'reported_seq': 13669, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:32.933916+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:26:32.933916+0000', 'last_peered': '2024-09-15T21:26:32.933916+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:26:32.933916+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28446720, 'num_objects': 6945, 'num_object_clones': 0, 'num_object_copies': 13890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6945, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10210, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2925}, {'shards': '0,2', 'objects': 728}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:36.494 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:36.494 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13744", 'reported_seq': 13864, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:33.949200+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:26:33.949200+0000', 'last_peered': '2024-09-15T21:26:33.949200+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:26:33.949200+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "39'3700", 'ondisk_log_start': "39'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28258787, 'num_objects': 6900, 'num_object_clones': 0, 'num_object_copies': 13800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6900, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11603, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 670}, {'shards': '2', 'objects': 2810}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:36.494 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:36.495 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13716", 'reported_seq': 13850, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:32.922164+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:26:32.922164+0000', 'last_peered': '2024-09-15T21:26:32.922164+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:26:32.922164+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "39'3700", 'ondisk_log_start': "39'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28549120, 'num_objects': 6970, 'num_object_clones': 0, 'num_object_copies': 13940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6970, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10660, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2876}, {'shards': '0,2', 'objects': 722}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:36.495 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:26:36.495 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13794", 'reported_seq': 13927, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:32.913148+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:26:32.913148+0000', 'last_peered': '2024-09-15T21:26:32.913148+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:26:32.913148+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27836416, 'num_objects': 6796, 'num_object_clones': 0, 'num_object_copies': 13592, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6796, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10703, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2657}, {'shards': '0,2', 'objects': 644}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:36.495 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:36.495 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13618", 'reported_seq': 20147, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:32.913653+0000', 'last_change': '2024-09-15T21:26:05.146366+0000', 'last_active': '2024-09-15T21:26:32.913653+0000', 'last_peered': '2024-09-15T21:26:32.913653+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:26:32.913653+0000', 'last_undegraded': '2024-09-15T21:26:05.134624+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27729920, 'num_objects': 6770, 'num_object_clones': 0, 'num_object_copies': 13540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 227, 'num_objects_misplaced': 660, 'num_objects_unfound': 0, 'num_objects_dirty': 6770, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10102, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3197, 'num_bytes_recovered': 13086720, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5883}, {'shards': '0,2', 'objects': 660}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:36.495 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:26:36.495 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'13857", 'reported_seq': 13997, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:32.944135+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:26:32.944135+0000', 'last_peered': '2024-09-15T21:26:32.944135+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:26:32.944135+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28709830, 'num_objects': 7011, 'num_object_clones': 0, 'num_object_copies': 14022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7011, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10561, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2864}, {'shards': '0,2', 'objects': 729}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:36.495 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:37.189 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:37.189 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41452/221020 objects degraded (18.755%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":448,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":110510,"data_bytes":452600585,"bytes_used":2127155200,"bytes_avail":287783137280,"bytes_total":289910292480,"degraded_objects":41452,"degraded_total":221020,"degraded_ratio":0.18754863813229572,"misplaced_objects":660,"misplaced_total":221020,"misplaced_ratio":0.0029861550990860557,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":109187,"recovering_keys_per_sec":0,"num_objects_recovered":320,"num_bytes_recovered":1310720,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":575},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:37.216 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:37.216 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:37.771 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:37.773 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:37.794 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-15T21:26:36.793092+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":452600585,"num_objects":110510,"num_object_clones":0,"num_object_copies":221020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41452,"num_objects_misplaced":660,"num_objects_unfound":0,"num_objects_dirty":110510,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":179949,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7363,"num_bytes_recovered":30138387,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160757,"ondisk_log_size":160757,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2077300,"kb_used_data":1053496,"kb_used_omap":12,"kb_used_meta":1023731,"kb_avail":281038220,"statfs":{"total":289910292480,"available":287783137280,"internally_reserved":0,"allocated":1078779904,"data_stored":1077916364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1048300547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,5,5,12,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18997248,"num_objects":-4638,"num_object_clones":0,"num_object_copies":-9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":81,"num_objects_misplaced":-301,"num_objects_unfound":0,"num_objects_dirty":-4638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4638,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":901120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004439"},"pg_stats":[{"pgid":"2.f","version":"45'14005","reported_seq":14081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.874443+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:32.874443+0000","last_peered":"2024-09-15T21:26:32.874443+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:32.874443+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29274112,"num_objects":7147,"num_object_clones":0,"num_object_copies":14294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14005,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3015},{"shards":"0,2","objects":703}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13724","reported_seq":13830,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:33.806706+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:33.806706+0000","last_peered":"2024-09-15T21:26:33.806706+0000","last_clean":"2024-09-15T21:26:33.806706+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:33.806706+0000","last_undegraded":"2024-09-15T21:26:33.806706+0000","last_fullsized":"2024-09-15T21:26:33.806706+0000","mapping_epoch":35,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28468166,"num_objects":6952,"num_object_clones":0,"num_object_copies":13904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12005,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14143","reported_seq":14270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:33.999673+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:33.999673+0000","last_peered":"2024-09-15T21:26:33.999673+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:33.999673+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"39'4100","ondisk_log_start":"39'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28516376,"num_objects":6963,"num_object_clones":0,"num_object_copies":13926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12243,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":679},{"shards":"2","objects":2696}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13544","reported_seq":13615,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:34.003291+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:34.003291+0000","last_peered":"2024-09-15T21:26:34.003291+0000","last_clean":"2024-09-15T21:26:34.003291+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:34.003291+0000","last_undegraded":"2024-09-15T21:26:34.003291+0000","last_fullsized":"2024-09-15T21:26:34.003291+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13544,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13620","reported_seq":13753,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:32.886665+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:32.886665+0000","last_peered":"2024-09-15T21:26:32.886665+0000","last_clean":"2024-09-15T21:26:32.886665+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:32.886665+0000","last_undegraded":"2024-09-15T21:26:32.886665+0000","last_fullsized":"2024-09-15T21:26:32.886665+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28201443,"num_objects":6886,"num_object_clones":0,"num_object_copies":13772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10075,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13713","reported_seq":13848,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.943778+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:32.943778+0000","last_peered":"2024-09-15T21:26:32.943778+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:32.943778+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28352995,"num_objects":6923,"num_object_clones":0,"num_object_copies":13846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6923,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10389,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2847},{"shards":"0,2","objects":683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13796","reported_seq":13933,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.941782+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:32.941782+0000","last_peered":"2024-09-15T21:26:32.941782+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:32.941782+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28156870,"num_objects":6876,"num_object_clones":0,"num_object_copies":13752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10107,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2715},{"shards":"0,2","objects":703}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'13898","reported_seq":14027,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:32.911127+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:32.911127+0000","last_peered":"2024-09-15T21:26:32.911127+0000","last_clean":"2024-09-15T21:26:32.911127+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:32.911127+0000","last_undegraded":"2024-09-15T21:26:32.911127+0000","last_fullsized":"2024-09-15T21:26:32.911127+0000","mapping_epoch":34,"log_start":"36'3800","ondisk_log_start":"36'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28549120,"num_objects":6970,"num_object_clones":0,"num_object_copies":13940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10670,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13666","reported_seq":13793,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:33.989698+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:33.989698+0000","last_peered":"2024-09-15T21:26:33.989698+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:33.989698+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28045795,"num_objects":6848,"num_object_clones":0,"num_object_copies":13696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6848,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11598,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":679},{"shards":"2","objects":2761}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13537","reported_seq":13669,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.933916+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:32.933916+0000","last_peered":"2024-09-15T21:26:32.933916+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:32.933916+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28446720,"num_objects":6945,"num_object_clones":0,"num_object_copies":13890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10210,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2925},{"shards":"0,2","objects":728}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13744","reported_seq":13864,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:33.949200+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:33.949200+0000","last_peered":"2024-09-15T21:26:33.949200+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:33.949200+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28258787,"num_objects":6900,"num_object_clones":0,"num_object_copies":13800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6900,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11603,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":670},{"shards":"2","objects":2810}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13779","reported_seq":22238,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:33.900117+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:33.900117+0000","last_peered":"2024-09-15T21:26:33.900117+0000","last_clean":"2024-09-15T21:26:33.900117+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:33.900117+0000","last_undegraded":"2024-09-15T21:26:33.900117+0000","last_fullsized":"2024-09-15T21:26:33.900117+0000","mapping_epoch":44,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27774976,"num_objects":6781,"num_object_clones":0,"num_object_copies":13562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11472,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13716","reported_seq":13850,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.922164+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:32.922164+0000","last_peered":"2024-09-15T21:26:32.922164+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:32.922164+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28549120,"num_objects":6970,"num_object_clones":0,"num_object_copies":13940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10660,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2876},{"shards":"0,2","objects":722}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13794","reported_seq":13927,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.913148+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:32.913148+0000","last_peered":"2024-09-15T21:26:32.913148+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:32.913148+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27836416,"num_objects":6796,"num_object_clones":0,"num_object_copies":13592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10703,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2657},{"shards":"0,2","objects":644}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13618","reported_seq":20147,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.913653+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:32.913653+0000","last_peered":"2024-09-15T21:26:32.913653+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:32.913653+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27729920,"num_objects":6770,"num_object_clones":0,"num_object_copies":13540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":227,"num_objects_misplaced":660,"num_objects_unfound":0,"num_objects_dirty":6770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10102,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3197,"num_bytes_recovered":13086720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5883},{"shards":"0,2","objects":660}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'13857","reported_seq":13997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:32.944135+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:32.944135+0000","last_peered":"2024-09-15T21:26:32.944135+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:32.944135+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28709830,"num_objects":7011,"num_object_clones":0,"num_object_copies":14022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7011,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10561,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2864},{"shards":"0,2","objects":729}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":452600566,"num_objects":110508,"num_object_clones":0,"num_object_copies":221016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41452,"num_objects_misplaced":660,"num_objects_unfound":0,"num_objects_dirty":110508,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":179947,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7362,"num_bytes_recovered":30138368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1076736000,"data_stored":1076586949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659048,"kb_used_data":352472,"kb_used_omap":4,"kb_used_meta":306555,"kb_avail":93712792,"statfs":{"total":96636764160,"available":95961899008,"internally_reserved":0,"allocated":360931328,"data_stored":360655681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":313913001},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":795984,"kb_used_data":399224,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93575856,"statfs":{"total":96636764160,"available":95821676544,"internally_reserved":0,"allocated":408805376,"data_stored":408529724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622268,"kb_used_data":301800,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93749572,"statfs":{"total":96636764160,"available":95999561728,"internally_reserved":0,"allocated":309043200,"data_stored":308730959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328134319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":408260608,"data_stored":408224478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308621312,"data_stored":308548593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":359854080,"data_stored":359813878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:40.796 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:41.369 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:41.370 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:41.396 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-15T21:26:40.794001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":443073289,"num_objects":108184,"num_object_clones":0,"num_object_copies":216368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41475,"num_objects_misplaced":519,"num_objects_unfound":0,"num_objects_dirty":108184,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":182275,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7481,"num_bytes_recovered":30621715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2095408,"kb_used_data":1041084,"kb_used_omap":12,"kb_used_meta":1054259,"kb_avail":281020112,"statfs":{"total":289910292480,"available":287764594688,"internally_reserved":0,"allocated":1066070016,"data_stored":1065206476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1079561219},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,10,6,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22294528,"num_objects":-5443,"num_object_clones":0,"num_object_copies":-10886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-5443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5443,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":950272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003877"},"pg_stats":[{"pgid":"2.f","version":"45'14138","reported_seq":14214,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.927171+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:37.927171+0000","last_peered":"2024-09-15T21:26:37.927171+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:37.927171+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"40'4100","ondisk_log_start":"40'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14138,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2882},{"shards":"0,2","objects":570}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13891","reported_seq":13997,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:38.877940+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:38.877940+0000","last_peered":"2024-09-15T21:26:38.877940+0000","last_clean":"2024-09-15T21:26:38.877940+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:38.877940+0000","last_undegraded":"2024-09-15T21:26:38.877940+0000","last_fullsized":"2024-09-15T21:26:38.877940+0000","mapping_epoch":35,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27784134,"num_objects":6785,"num_object_clones":0,"num_object_copies":13570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6785,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12172,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14280","reported_seq":14407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.009915+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:39.009915+0000","last_peered":"2024-09-15T21:26:39.009915+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:39.009915+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27955224,"num_objects":6826,"num_object_clones":0,"num_object_copies":13652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12380,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":542},{"shards":"2","objects":2559}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13693","reported_seq":13764,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:38.935272+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:38.935272+0000","last_peered":"2024-09-15T21:26:38.935272+0000","last_clean":"2024-09-15T21:26:38.935272+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:38.935272+0000","last_undegraded":"2024-09-15T21:26:38.935272+0000","last_fullsized":"2024-09-15T21:26:38.935272+0000","mapping_epoch":35,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27119616,"num_objects":6621,"num_object_clones":0,"num_object_copies":13242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13781","reported_seq":13915,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:37.940859+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:37.940859+0000","last_peered":"2024-09-15T21:26:37.940859+0000","last_clean":"2024-09-15T21:26:37.940859+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:37.940859+0000","last_undegraded":"2024-09-15T21:26:37.940859+0000","last_fullsized":"2024-09-15T21:26:37.940859+0000","mapping_epoch":34,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541987,"num_objects":6725,"num_object_clones":0,"num_object_copies":13450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10236,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13852","reported_seq":13987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.952856+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:37.952856+0000","last_peered":"2024-09-15T21:26:37.952856+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:37.952856+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27783651,"num_objects":6784,"num_object_clones":0,"num_object_copies":13568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6784,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10528,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2708},{"shards":"0,2","objects":544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13928","reported_seq":14065,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.958748+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:37.958748+0000","last_peered":"2024-09-15T21:26:37.958748+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:37.958748+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27616198,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2583},{"shards":"0,2","objects":571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14066","reported_seq":14196,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:37.884006+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:37.884006+0000","last_peered":"2024-09-15T21:26:37.884006+0000","last_clean":"2024-09-15T21:26:37.884006+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:37.884006+0000","last_undegraded":"2024-09-15T21:26:37.884006+0000","last_fullsized":"2024-09-15T21:26:37.884006+0000","mapping_epoch":34,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27860992,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10838,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13802","reported_seq":13929,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:38.979554+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:38.979554+0000","last_peered":"2024-09-15T21:26:38.979554+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:38.979554+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27488739,"num_objects":6712,"num_object_clones":0,"num_object_copies":13424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6712,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11734,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":543},{"shards":"2","objects":2625}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13704","reported_seq":13836,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.904539+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:37.904539+0000","last_peered":"2024-09-15T21:26:37.904539+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:37.904539+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27762688,"num_objects":6778,"num_object_clones":0,"num_object_copies":13556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10377,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2758},{"shards":"0,2","objects":561}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13881","reported_seq":14000,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:38.980085+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:38.980085+0000","last_peered":"2024-09-15T21:26:38.980085+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:38.980085+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27697635,"num_objects":6763,"num_object_clones":0,"num_object_copies":13526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11740,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":533},{"shards":"2","objects":2673}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13926","reported_seq":22385,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:38.925655+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:38.925655+0000","last_peered":"2024-09-15T21:26:38.925655+0000","last_clean":"2024-09-15T21:26:38.925655+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:38.925655+0000","last_undegraded":"2024-09-15T21:26:38.925655+0000","last_fullsized":"2024-09-15T21:26:38.925655+0000","mapping_epoch":44,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27172864,"num_objects":6634,"num_object_clones":0,"num_object_copies":13268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11619,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13863","reported_seq":13997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.880033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:37.880033+0000","last_peered":"2024-09-15T21:26:37.880033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:37.880033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27947008,"num_objects":6823,"num_object_clones":0,"num_object_copies":13646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10807,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2729},{"shards":"0,2","objects":575}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13912","reported_seq":14045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.950779+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:37.950779+0000","last_peered":"2024-09-15T21:26:37.950779+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:37.950779+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27353088,"num_objects":6678,"num_object_clones":0,"num_object_copies":13356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10821,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2539},{"shards":"0,2","objects":526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13751","reported_seq":20519,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.904841+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:37.904841+0000","last_peered":"2024-09-15T21:26:37.904841+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:37.904841+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27185152,"num_objects":6637,"num_object_clones":0,"num_object_copies":13274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":250,"num_objects_misplaced":519,"num_objects_unfound":0,"num_objects_dirty":6637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10235,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3315,"num_bytes_recovered":13570048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5868},{"shards":"0,2","objects":519}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14012","reported_seq":14152,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.919590+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:37.919590+0000","last_peered":"2024-09-15T21:26:37.919590+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:37.919590+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28074950,"num_objects":6856,"num_object_clones":0,"num_object_copies":13712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10716,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2709},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":443073270,"num_objects":108182,"num_object_clones":0,"num_object_copies":216364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41475,"num_objects_misplaced":519,"num_objects_unfound":0,"num_objects_dirty":108182,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":182273,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7480,"num_bytes_recovered":30621696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064005632,"data_stored":1063856581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658464,"kb_used_data":347728,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93713376,"statfs":{"total":96636764160,"available":95962497024,"internally_reserved":0,"allocated":356073472,"data_stored":355797825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":793020,"kb_used_data":392104,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93578820,"statfs":{"total":96636764160,"available":95824711680,"internally_reserved":0,"allocated":401514496,"data_stored":401238844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410513067},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,3,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643924,"kb_used_data":301252,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93727916,"statfs":{"total":96636764160,"available":95977385984,"internally_reserved":0,"allocated":308482048,"data_stored":308169807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401006592,"data_stored":400970462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308080640,"data_stored":308007921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":354918400,"data_stored":354878198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:41.398 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:41.979 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:41.979 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:42.012 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-15T21:26:40.794001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":443073289,"num_objects":108184,"num_object_clones":0,"num_object_copies":216368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41475,"num_objects_misplaced":519,"num_objects_unfound":0,"num_objects_dirty":108184,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":182275,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7481,"num_bytes_recovered":30621715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2095408,"kb_used_data":1041084,"kb_used_omap":12,"kb_used_meta":1054259,"kb_avail":281020112,"statfs":{"total":289910292480,"available":287764594688,"internally_reserved":0,"allocated":1066070016,"data_stored":1065206476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1079561219},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,10,6,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22294528,"num_objects":-5443,"num_object_clones":0,"num_object_copies":-10886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-5443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5443,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":950272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003877"},"pg_stats":[{"pgid":"2.f","version":"45'14138","reported_seq":14214,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.927171+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:37.927171+0000","last_peered":"2024-09-15T21:26:37.927171+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:37.927171+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"40'4100","ondisk_log_start":"40'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14138,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2882},{"shards":"0,2","objects":570}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13891","reported_seq":13997,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:38.877940+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:38.877940+0000","last_peered":"2024-09-15T21:26:38.877940+0000","last_clean":"2024-09-15T21:26:38.877940+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:38.877940+0000","last_undegraded":"2024-09-15T21:26:38.877940+0000","last_fullsized":"2024-09-15T21:26:38.877940+0000","mapping_epoch":35,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27784134,"num_objects":6785,"num_object_clones":0,"num_object_copies":13570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6785,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12172,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14280","reported_seq":14407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.009915+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:39.009915+0000","last_peered":"2024-09-15T21:26:39.009915+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:39.009915+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27955224,"num_objects":6826,"num_object_clones":0,"num_object_copies":13652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12380,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":542},{"shards":"2","objects":2559}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13693","reported_seq":13764,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:38.935272+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:38.935272+0000","last_peered":"2024-09-15T21:26:38.935272+0000","last_clean":"2024-09-15T21:26:38.935272+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:38.935272+0000","last_undegraded":"2024-09-15T21:26:38.935272+0000","last_fullsized":"2024-09-15T21:26:38.935272+0000","mapping_epoch":35,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27119616,"num_objects":6621,"num_object_clones":0,"num_object_copies":13242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13781","reported_seq":13915,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:37.940859+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:37.940859+0000","last_peered":"2024-09-15T21:26:37.940859+0000","last_clean":"2024-09-15T21:26:37.940859+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:37.940859+0000","last_undegraded":"2024-09-15T21:26:37.940859+0000","last_fullsized":"2024-09-15T21:26:37.940859+0000","mapping_epoch":34,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541987,"num_objects":6725,"num_object_clones":0,"num_object_copies":13450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10236,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13852","reported_seq":13987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.952856+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:37.952856+0000","last_peered":"2024-09-15T21:26:37.952856+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:37.952856+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27783651,"num_objects":6784,"num_object_clones":0,"num_object_copies":13568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6784,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10528,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2708},{"shards":"0,2","objects":544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13928","reported_seq":14065,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.958748+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:37.958748+0000","last_peered":"2024-09-15T21:26:37.958748+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:37.958748+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27616198,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2583},{"shards":"0,2","objects":571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14066","reported_seq":14196,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:37.884006+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:37.884006+0000","last_peered":"2024-09-15T21:26:37.884006+0000","last_clean":"2024-09-15T21:26:37.884006+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:37.884006+0000","last_undegraded":"2024-09-15T21:26:37.884006+0000","last_fullsized":"2024-09-15T21:26:37.884006+0000","mapping_epoch":34,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27860992,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10838,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13802","reported_seq":13929,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:38.979554+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:38.979554+0000","last_peered":"2024-09-15T21:26:38.979554+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:38.979554+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27488739,"num_objects":6712,"num_object_clones":0,"num_object_copies":13424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6712,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11734,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":543},{"shards":"2","objects":2625}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13704","reported_seq":13836,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.904539+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:37.904539+0000","last_peered":"2024-09-15T21:26:37.904539+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:37.904539+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27762688,"num_objects":6778,"num_object_clones":0,"num_object_copies":13556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10377,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2758},{"shards":"0,2","objects":561}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13881","reported_seq":14000,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:38.980085+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:38.980085+0000","last_peered":"2024-09-15T21:26:38.980085+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:38.980085+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27697635,"num_objects":6763,"num_object_clones":0,"num_object_copies":13526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11740,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":533},{"shards":"2","objects":2673}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13926","reported_seq":22385,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:38.925655+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:38.925655+0000","last_peered":"2024-09-15T21:26:38.925655+0000","last_clean":"2024-09-15T21:26:38.925655+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:38.925655+0000","last_undegraded":"2024-09-15T21:26:38.925655+0000","last_fullsized":"2024-09-15T21:26:38.925655+0000","mapping_epoch":44,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27172864,"num_objects":6634,"num_object_clones":0,"num_object_copies":13268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11619,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13863","reported_seq":13997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.880033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:37.880033+0000","last_peered":"2024-09-15T21:26:37.880033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:37.880033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27947008,"num_objects":6823,"num_object_clones":0,"num_object_copies":13646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10807,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2729},{"shards":"0,2","objects":575}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13912","reported_seq":14045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.950779+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:37.950779+0000","last_peered":"2024-09-15T21:26:37.950779+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:37.950779+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27353088,"num_objects":6678,"num_object_clones":0,"num_object_copies":13356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10821,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2539},{"shards":"0,2","objects":526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13751","reported_seq":20519,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.904841+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:37.904841+0000","last_peered":"2024-09-15T21:26:37.904841+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:37.904841+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27185152,"num_objects":6637,"num_object_clones":0,"num_object_copies":13274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":250,"num_objects_misplaced":519,"num_objects_unfound":0,"num_objects_dirty":6637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10235,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3315,"num_bytes_recovered":13570048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5868},{"shards":"0,2","objects":519}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14012","reported_seq":14152,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.919590+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:37.919590+0000","last_peered":"2024-09-15T21:26:37.919590+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:37.919590+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28074950,"num_objects":6856,"num_object_clones":0,"num_object_copies":13712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10716,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2709},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":443073270,"num_objects":108182,"num_object_clones":0,"num_object_copies":216364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41475,"num_objects_misplaced":519,"num_objects_unfound":0,"num_objects_dirty":108182,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":182273,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7480,"num_bytes_recovered":30621696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064005632,"data_stored":1063856581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658464,"kb_used_data":347728,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93713376,"statfs":{"total":96636764160,"available":95962497024,"internally_reserved":0,"allocated":356073472,"data_stored":355797825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":793020,"kb_used_data":392104,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93578820,"statfs":{"total":96636764160,"available":95824711680,"internally_reserved":0,"allocated":401514496,"data_stored":401238844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410513067},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,3,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643924,"kb_used_data":301252,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93727916,"statfs":{"total":96636764160,"available":95977385984,"internally_reserved":0,"allocated":308482048,"data_stored":308169807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401006592,"data_stored":400970462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308080640,"data_stored":308007921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":354918400,"data_stored":354878198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:42.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:42.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14138", 'reported_seq': 14214, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:37.927171+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:26:37.927171+0000', 'last_peered': '2024-09-15T21:26:37.927171+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:26:37.927171+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "40'4100", 'ondisk_log_start': "40'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28729344, 'num_objects': 7014, 'num_object_clones': 0, 'num_object_copies': 14028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7014, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14138, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2882}, {'shards': '0,2', 'objects': 570}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:26:42.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'14280", 'reported_seq': 14407, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:39.009915+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:26:39.009915+0000', 'last_peered': '2024-09-15T21:26:39.009915+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:26:39.009915+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "40'4200", 'ondisk_log_start': "40'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27955224, 'num_objects': 6826, 'num_object_clones': 0, 'num_object_copies': 13652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6826, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12380, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 542}, {'shards': '2', 'objects': 2559}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:42.013 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:42.013 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13852", 'reported_seq': 13987, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:37.952856+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:26:37.952856+0000', 'last_peered': '2024-09-15T21:26:37.952856+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:26:37.952856+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27783651, 'num_objects': 6784, 'num_object_clones': 0, 'num_object_copies': 13568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6784, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10528, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2708}, {'shards': '0,2', 'objects': 544}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:42.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'13928", 'reported_seq': 14065, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:37.958748+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:26:37.958748+0000', 'last_peered': '2024-09-15T21:26:37.958748+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:26:37.958748+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "39'3900", 'ondisk_log_start': "39'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27616198, 'num_objects': 6744, 'num_object_clones': 0, 'num_object_copies': 13488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6744, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10239, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2583}, {'shards': '0,2', 'objects': 571}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:26:42.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'13802", 'reported_seq': 13929, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:38.979554+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:26:38.979554+0000', 'last_peered': '2024-09-15T21:26:38.979554+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:26:38.979554+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27488739, 'num_objects': 6712, 'num_object_clones': 0, 'num_object_copies': 13424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6712, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11734, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 543}, {'shards': '2', 'objects': 2625}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:42.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:42.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13704", 'reported_seq': 13836, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:37.904539+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:26:37.904539+0000', 'last_peered': '2024-09-15T21:26:37.904539+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:26:37.904539+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "39'3700", 'ondisk_log_start': "39'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27762688, 'num_objects': 6778, 'num_object_clones': 0, 'num_object_copies': 13556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6778, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10377, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2758}, {'shards': '0,2', 'objects': 561}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:42.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'13881", 'reported_seq': 14000, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:38.980085+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:26:38.980085+0000', 'last_peered': '2024-09-15T21:26:38.980085+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:26:38.980085+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27697635, 'num_objects': 6763, 'num_object_clones': 0, 'num_object_copies': 13526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6763, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11740, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 533}, {'shards': '2', 'objects': 2673}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:42.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:42.014 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'13863", 'reported_seq': 13997, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:37.880033+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:26:37.880033+0000', 'last_peered': '2024-09-15T21:26:37.880033+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:26:37.880033+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27947008, 'num_objects': 6823, 'num_object_clones': 0, 'num_object_copies': 13646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6823, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10807, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2729}, {'shards': '0,2', 'objects': 575}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.014 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:26:42.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'13912", 'reported_seq': 14045, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:37.950779+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:26:37.950779+0000', 'last_peered': '2024-09-15T21:26:37.950779+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:26:37.950779+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "39'3900", 'ondisk_log_start': "39'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27353088, 'num_objects': 6678, 'num_object_clones': 0, 'num_object_copies': 13356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6678, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10821, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2539}, {'shards': '0,2', 'objects': 526}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:42.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13751", 'reported_seq': 20519, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:37.904841+0000', 'last_change': '2024-09-15T21:26:05.146366+0000', 'last_active': '2024-09-15T21:26:37.904841+0000', 'last_peered': '2024-09-15T21:26:37.904841+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:26:37.904841+0000', 'last_undegraded': '2024-09-15T21:26:05.134624+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "36'3700", 'ondisk_log_start': "36'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27185152, 'num_objects': 6637, 'num_object_clones': 0, 'num_object_copies': 13274, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 250, 'num_objects_misplaced': 519, 'num_objects_unfound': 0, 'num_objects_dirty': 6637, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10235, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3315, 'num_bytes_recovered': 13570048, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5868}, {'shards': '0,2', 'objects': 519}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.015 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:26:42.015 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'14012", 'reported_seq': 14152, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:37.919590+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:26:37.919590+0000', 'last_peered': '2024-09-15T21:26:37.919590+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:26:37.919590+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "39'4000", 'ondisk_log_start': "39'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28074950, 'num_objects': 6856, 'num_object_clones': 0, 'num_object_copies': 13712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6856, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10716, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2709}, {'shards': '0,2', 'objects': 574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:42.015 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:42.694 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:42.694 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41475/216368 objects degraded (19.169%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":454,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":108184,"data_bytes":443073289,"bytes_used":2145697792,"bytes_avail":287764594688,"bytes_total":289910292480,"degraded_objects":41475,"degraded_total":216368,"degraded_ratio":0.19168731050802335,"misplaced_objects":519,"misplaced_total":216368,"misplaced_ratio":0.0023986911188345781,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":79163,"recovering_keys_per_sec":0,"num_objects_recovered":232,"num_bytes_recovered":950272,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":453},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:42.713 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:42.713 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:43.269 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:43.269 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:43.301 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-15T21:26:42.794467+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":443073289,"num_objects":108184,"num_object_clones":0,"num_object_copies":216368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41475,"num_objects_misplaced":519,"num_objects_unfound":0,"num_objects_dirty":108184,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":182275,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7481,"num_bytes_recovered":30621715,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2095408,"kb_used_data":1041084,"kb_used_omap":12,"kb_used_meta":1054259,"kb_avail":281020112,"statfs":{"total":289910292480,"available":287764594688,"internally_reserved":0,"allocated":1066070016,"data_stored":1065206476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1079561219},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,10,6,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18903040,"num_objects":-4615,"num_object_clones":0,"num_object_copies":-9230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-4615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4615,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":950272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003645"},"pg_stats":[{"pgid":"2.f","version":"45'14138","reported_seq":14214,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.927171+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:37.927171+0000","last_peered":"2024-09-15T21:26:37.927171+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:37.927171+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"40'4100","ondisk_log_start":"40'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14138,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2882},{"shards":"0,2","objects":570}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'13891","reported_seq":13997,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:38.877940+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:38.877940+0000","last_peered":"2024-09-15T21:26:38.877940+0000","last_clean":"2024-09-15T21:26:38.877940+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:38.877940+0000","last_undegraded":"2024-09-15T21:26:38.877940+0000","last_fullsized":"2024-09-15T21:26:38.877940+0000","mapping_epoch":35,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27784134,"num_objects":6785,"num_object_clones":0,"num_object_copies":13570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6785,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12172,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14280","reported_seq":14407,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:39.009915+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:39.009915+0000","last_peered":"2024-09-15T21:26:39.009915+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:39.009915+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27955224,"num_objects":6826,"num_object_clones":0,"num_object_copies":13652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12380,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":542},{"shards":"2","objects":2559}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13693","reported_seq":13764,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:38.935272+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:38.935272+0000","last_peered":"2024-09-15T21:26:38.935272+0000","last_clean":"2024-09-15T21:26:38.935272+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:38.935272+0000","last_undegraded":"2024-09-15T21:26:38.935272+0000","last_fullsized":"2024-09-15T21:26:38.935272+0000","mapping_epoch":35,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27119616,"num_objects":6621,"num_object_clones":0,"num_object_copies":13242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6621,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13781","reported_seq":13915,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:37.940859+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:37.940859+0000","last_peered":"2024-09-15T21:26:37.940859+0000","last_clean":"2024-09-15T21:26:37.940859+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:37.940859+0000","last_undegraded":"2024-09-15T21:26:37.940859+0000","last_fullsized":"2024-09-15T21:26:37.940859+0000","mapping_epoch":34,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27541987,"num_objects":6725,"num_object_clones":0,"num_object_copies":13450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10236,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13852","reported_seq":13987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.952856+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:37.952856+0000","last_peered":"2024-09-15T21:26:37.952856+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:37.952856+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27783651,"num_objects":6784,"num_object_clones":0,"num_object_copies":13568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6784,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10528,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2708},{"shards":"0,2","objects":544}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'13928","reported_seq":14065,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.958748+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:37.958748+0000","last_peered":"2024-09-15T21:26:37.958748+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:37.958748+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27616198,"num_objects":6744,"num_object_clones":0,"num_object_copies":13488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10239,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2583},{"shards":"0,2","objects":571}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14066","reported_seq":14196,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:37.884006+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:37.884006+0000","last_peered":"2024-09-15T21:26:37.884006+0000","last_clean":"2024-09-15T21:26:37.884006+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:37.884006+0000","last_undegraded":"2024-09-15T21:26:37.884006+0000","last_fullsized":"2024-09-15T21:26:37.884006+0000","mapping_epoch":34,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27860992,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10838,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13802","reported_seq":13929,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:38.979554+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:38.979554+0000","last_peered":"2024-09-15T21:26:38.979554+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:38.979554+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27488739,"num_objects":6712,"num_object_clones":0,"num_object_copies":13424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6712,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11734,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":543},{"shards":"2","objects":2625}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13704","reported_seq":13836,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.904539+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:37.904539+0000","last_peered":"2024-09-15T21:26:37.904539+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:37.904539+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27762688,"num_objects":6778,"num_object_clones":0,"num_object_copies":13556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10377,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2758},{"shards":"0,2","objects":561}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'13881","reported_seq":14000,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:38.980085+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:38.980085+0000","last_peered":"2024-09-15T21:26:38.980085+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:38.980085+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27697635,"num_objects":6763,"num_object_clones":0,"num_object_copies":13526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11740,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":533},{"shards":"2","objects":2673}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'13926","reported_seq":22385,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:38.925655+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:38.925655+0000","last_peered":"2024-09-15T21:26:38.925655+0000","last_clean":"2024-09-15T21:26:38.925655+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:38.925655+0000","last_undegraded":"2024-09-15T21:26:38.925655+0000","last_fullsized":"2024-09-15T21:26:38.925655+0000","mapping_epoch":44,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27172864,"num_objects":6634,"num_object_clones":0,"num_object_copies":13268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11619,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'13863","reported_seq":13997,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.880033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:37.880033+0000","last_peered":"2024-09-15T21:26:37.880033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:37.880033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27947008,"num_objects":6823,"num_object_clones":0,"num_object_copies":13646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10807,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2729},{"shards":"0,2","objects":575}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'13912","reported_seq":14045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.950779+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:37.950779+0000","last_peered":"2024-09-15T21:26:37.950779+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:37.950779+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27353088,"num_objects":6678,"num_object_clones":0,"num_object_copies":13356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10821,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2539},{"shards":"0,2","objects":526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13751","reported_seq":20519,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.904841+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:37.904841+0000","last_peered":"2024-09-15T21:26:37.904841+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:37.904841+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"36'3700","ondisk_log_start":"36'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27185152,"num_objects":6637,"num_object_clones":0,"num_object_copies":13274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":250,"num_objects_misplaced":519,"num_objects_unfound":0,"num_objects_dirty":6637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10235,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3315,"num_bytes_recovered":13570048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5868},{"shards":"0,2","objects":519}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14012","reported_seq":14152,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:37.919590+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:37.919590+0000","last_peered":"2024-09-15T21:26:37.919590+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:37.919590+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28074950,"num_objects":6856,"num_object_clones":0,"num_object_copies":13712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6856,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10716,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2709},{"shards":"0,2","objects":574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":443073270,"num_objects":108182,"num_object_clones":0,"num_object_copies":216364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41475,"num_objects_misplaced":519,"num_objects_unfound":0,"num_objects_dirty":108182,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":182273,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7480,"num_bytes_recovered":30621696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064005632,"data_stored":1063856581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658464,"kb_used_data":347728,"kb_used_omap":4,"kb_used_meta":310715,"kb_avail":93713376,"statfs":{"total":96636764160,"available":95962497024,"internally_reserved":0,"allocated":356073472,"data_stored":355797825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":318172841},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":793020,"kb_used_data":392104,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93578820,"statfs":{"total":96636764160,"available":95824711680,"internally_reserved":0,"allocated":401514496,"data_stored":401238844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410513067},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,3,1,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643924,"kb_used_data":301252,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93727916,"statfs":{"total":96636764160,"available":95977385984,"internally_reserved":0,"allocated":308482048,"data_stored":308169807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":401006592,"data_stored":400970462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":308080640,"data_stored":308007921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":354918400,"data_stored":354878198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:46.304 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:46.870 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:46.871 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:46.902 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-15T21:26:46.795399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":433521417,"num_objects":105852,"num_object_clones":0,"num_object_copies":211704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41486,"num_objects_misplaced":386,"num_objects_unfound":0,"num_objects_dirty":105852,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":184607,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7603,"num_bytes_recovered":31121427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2072032,"kb_used_data":1028772,"kb_used_omap":12,"kb_used_meta":1043187,"kb_avail":281043488,"statfs":{"total":289910292480,"available":287788531712,"internally_reserved":0,"allocated":1053462528,"data_stored":1052598988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1068223491},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,7,4,5,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19079168,"num_objects":-4658,"num_object_clones":0,"num_object_copies":-9316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":-4658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4658,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002816"},"pg_stats":[{"pgid":"2.f","version":"45'14285","reported_seq":14361,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.950333+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:42.950333+0000","last_peered":"2024-09-15T21:26:42.950333+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:42.950333+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28127232,"num_objects":6867,"num_object_clones":0,"num_object_copies":13734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14285,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2735},{"shards":"0,2","objects":423}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14025","reported_seq":14131,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:43.859062+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:43.859062+0000","last_peered":"2024-09-15T21:26:43.859062+0000","last_clean":"2024-09-15T21:26:43.859062+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:43.859062+0000","last_undegraded":"2024-09-15T21:26:43.859062+0000","last_fullsized":"2024-09-15T21:26:43.859062+0000","mapping_epoch":35,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27235270,"num_objects":6651,"num_object_clones":0,"num_object_copies":13302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6651,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12306,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14420","reported_seq":14547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.032619+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:44.032619+0000","last_peered":"2024-09-15T21:26:44.032619+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:44.032619+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27381784,"num_objects":6686,"num_object_clones":0,"num_object_copies":13372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6686,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12520,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":402},{"shards":"2","objects":2419}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13830","reported_seq":13901,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:43.929009+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:43.929009+0000","last_peered":"2024-09-15T21:26:43.929009+0000","last_clean":"2024-09-15T21:26:43.929009+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:43.929009+0000","last_undegraded":"2024-09-15T21:26:43.929009+0000","last_fullsized":"2024-09-15T21:26:43.929009+0000","mapping_epoch":35,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26558464,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13830,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13921","reported_seq":14055,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:42.870469+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:42.870469+0000","last_peered":"2024-09-15T21:26:42.870469+0000","last_clean":"2024-09-15T21:26:42.870469+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:42.870469+0000","last_undegraded":"2024-09-15T21:26:42.870469+0000","last_fullsized":"2024-09-15T21:26:42.870469+0000","mapping_epoch":34,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26968547,"num_objects":6585,"num_object_clones":0,"num_object_copies":13170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10376,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13987","reported_seq":14124,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.937233+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:42.937233+0000","last_peered":"2024-09-15T21:26:42.937233+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:42.937233+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27230691,"num_objects":6649,"num_object_clones":0,"num_object_copies":13298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10663,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2573},{"shards":"0,2","objects":409}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14071","reported_seq":14207,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.904422+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:42.904422+0000","last_peered":"2024-09-15T21:26:42.904422+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:42.904422+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27030470,"num_objects":6601,"num_object_clones":0,"num_object_copies":13202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10382,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2440},{"shards":"0,2","objects":428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14233","reported_seq":14363,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:42.922161+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:42.922161+0000","last_peered":"2024-09-15T21:26:42.922161+0000","last_clean":"2024-09-15T21:26:42.922161+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:42.922161+0000","last_undegraded":"2024-09-15T21:26:42.922161+0000","last_fullsized":"2024-09-15T21:26:42.922161+0000","mapping_epoch":34,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27176960,"num_objects":6635,"num_object_clones":0,"num_object_copies":13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11005,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13967","reported_seq":14093,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:43.949514+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:43.949514+0000","last_peered":"2024-09-15T21:26:43.949514+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:43.949514+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26812899,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11899,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":378},{"shards":"2","objects":2460}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13855","reported_seq":13987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.938230+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:42.938230+0000","last_peered":"2024-09-15T21:26:42.938230+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:42.938230+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27144192,"num_objects":6627,"num_object_clones":0,"num_object_copies":13254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10528,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2607},{"shards":"0,2","objects":410}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14024","reported_seq":14144,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.037103+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:44.037103+0000","last_peered":"2024-09-15T21:26:44.037103+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:44.037103+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27111907,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11883,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":390},{"shards":"2","objects":2530}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14077","reported_seq":22536,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:44.017911+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:44.017911+0000","last_peered":"2024-09-15T21:26:44.017911+0000","last_clean":"2024-09-15T21:26:44.017911+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:44.017911+0000","last_undegraded":"2024-09-15T21:26:44.017911+0000","last_fullsized":"2024-09-15T21:26:44.017911+0000","mapping_epoch":44,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26554368,"num_objects":6483,"num_object_clones":0,"num_object_copies":12966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11770,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14014","reported_seq":14148,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.948827+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:42.948827+0000","last_peered":"2024-09-15T21:26:42.948827+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:42.948827+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27328512,"num_objects":6672,"num_object_clones":0,"num_object_copies":13344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10958,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2578},{"shards":"0,2","objects":424}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14049","reported_seq":14182,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.986790+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:42.986790+0000","last_peered":"2024-09-15T21:26:42.986790+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:42.986790+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26791936,"num_objects":6541,"num_object_clones":0,"num_object_copies":13082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10958,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2402},{"shards":"0,2","objects":389}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13890","reported_seq":20900,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.978228+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:42.978228+0000","last_peered":"2024-09-15T21:26:42.978228+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:42.978228+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26615808,"num_objects":6498,"num_object_clones":0,"num_object_copies":12996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":261,"num_objects_misplaced":386,"num_objects_unfound":0,"num_objects_dirty":6498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10374,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3437,"num_bytes_recovered":14069760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5851},{"shards":"0,2","objects":386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14164","reported_seq":14304,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.865647+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:42.865647+0000","last_peered":"2024-09-15T21:26:42.865647+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:42.865647+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"40'4100","ondisk_log_start":"40'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27452358,"num_objects":6704,"num_object_clones":0,"num_object_copies":13408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10868,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2557},{"shards":"0,2","objects":422}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":433521398,"num_objects":105850,"num_object_clones":0,"num_object_copies":211700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41486,"num_objects_misplaced":386,"num_objects_unfound":0,"num_objects_dirty":105850,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":184605,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7602,"num_bytes_recovered":31121408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1051402240,"data_stored":1051253189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659272,"kb_used_data":342964,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93712568,"statfs":{"total":96636764160,"available":95961669632,"internally_reserved":0,"allocated":351195136,"data_stored":350919489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182485,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":777780,"kb_used_data":385180,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93594060,"statfs":{"total":96636764160,"available":95840317440,"internally_reserved":0,"allocated":394424320,"data_stored":394148668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,0,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634980,"kb_used_data":300628,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93736860,"statfs":{"total":96636764160,"available":95986544640,"internally_reserved":0,"allocated":307843072,"data_stored":307530831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":393908224,"data_stored":393872094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307417088,"data_stored":307344369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350076928,"data_stored":350036726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:46.903 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:47.457 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:47.458 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:47.483 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-15T21:26:46.795399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":433521417,"num_objects":105852,"num_object_clones":0,"num_object_copies":211704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41486,"num_objects_misplaced":386,"num_objects_unfound":0,"num_objects_dirty":105852,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":184607,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7603,"num_bytes_recovered":31121427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2072032,"kb_used_data":1028772,"kb_used_omap":12,"kb_used_meta":1043187,"kb_avail":281043488,"statfs":{"total":289910292480,"available":287788531712,"internally_reserved":0,"allocated":1053462528,"data_stored":1052598988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1068223491},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,7,4,5,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19079168,"num_objects":-4658,"num_object_clones":0,"num_object_copies":-9316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":-4658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4658,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002816"},"pg_stats":[{"pgid":"2.f","version":"45'14285","reported_seq":14361,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.950333+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:42.950333+0000","last_peered":"2024-09-15T21:26:42.950333+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:42.950333+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28127232,"num_objects":6867,"num_object_clones":0,"num_object_copies":13734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14285,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2735},{"shards":"0,2","objects":423}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14025","reported_seq":14131,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:43.859062+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:43.859062+0000","last_peered":"2024-09-15T21:26:43.859062+0000","last_clean":"2024-09-15T21:26:43.859062+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:43.859062+0000","last_undegraded":"2024-09-15T21:26:43.859062+0000","last_fullsized":"2024-09-15T21:26:43.859062+0000","mapping_epoch":35,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27235270,"num_objects":6651,"num_object_clones":0,"num_object_copies":13302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6651,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12306,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14420","reported_seq":14547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.032619+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:44.032619+0000","last_peered":"2024-09-15T21:26:44.032619+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:44.032619+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27381784,"num_objects":6686,"num_object_clones":0,"num_object_copies":13372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6686,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12520,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":402},{"shards":"2","objects":2419}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13830","reported_seq":13901,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:43.929009+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:43.929009+0000","last_peered":"2024-09-15T21:26:43.929009+0000","last_clean":"2024-09-15T21:26:43.929009+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:43.929009+0000","last_undegraded":"2024-09-15T21:26:43.929009+0000","last_fullsized":"2024-09-15T21:26:43.929009+0000","mapping_epoch":35,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26558464,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13830,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13921","reported_seq":14055,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:42.870469+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:42.870469+0000","last_peered":"2024-09-15T21:26:42.870469+0000","last_clean":"2024-09-15T21:26:42.870469+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:42.870469+0000","last_undegraded":"2024-09-15T21:26:42.870469+0000","last_fullsized":"2024-09-15T21:26:42.870469+0000","mapping_epoch":34,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26968547,"num_objects":6585,"num_object_clones":0,"num_object_copies":13170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10376,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13987","reported_seq":14124,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.937233+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:42.937233+0000","last_peered":"2024-09-15T21:26:42.937233+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:42.937233+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27230691,"num_objects":6649,"num_object_clones":0,"num_object_copies":13298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10663,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2573},{"shards":"0,2","objects":409}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14071","reported_seq":14207,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.904422+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:42.904422+0000","last_peered":"2024-09-15T21:26:42.904422+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:42.904422+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27030470,"num_objects":6601,"num_object_clones":0,"num_object_copies":13202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10382,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2440},{"shards":"0,2","objects":428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14233","reported_seq":14363,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:42.922161+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:42.922161+0000","last_peered":"2024-09-15T21:26:42.922161+0000","last_clean":"2024-09-15T21:26:42.922161+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:42.922161+0000","last_undegraded":"2024-09-15T21:26:42.922161+0000","last_fullsized":"2024-09-15T21:26:42.922161+0000","mapping_epoch":34,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27176960,"num_objects":6635,"num_object_clones":0,"num_object_copies":13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11005,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13967","reported_seq":14093,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:43.949514+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:43.949514+0000","last_peered":"2024-09-15T21:26:43.949514+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:43.949514+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26812899,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11899,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":378},{"shards":"2","objects":2460}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13855","reported_seq":13987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.938230+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:42.938230+0000","last_peered":"2024-09-15T21:26:42.938230+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:42.938230+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27144192,"num_objects":6627,"num_object_clones":0,"num_object_copies":13254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10528,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2607},{"shards":"0,2","objects":410}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14024","reported_seq":14144,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.037103+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:44.037103+0000","last_peered":"2024-09-15T21:26:44.037103+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:44.037103+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27111907,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11883,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":390},{"shards":"2","objects":2530}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14077","reported_seq":22536,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:44.017911+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:44.017911+0000","last_peered":"2024-09-15T21:26:44.017911+0000","last_clean":"2024-09-15T21:26:44.017911+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:44.017911+0000","last_undegraded":"2024-09-15T21:26:44.017911+0000","last_fullsized":"2024-09-15T21:26:44.017911+0000","mapping_epoch":44,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26554368,"num_objects":6483,"num_object_clones":0,"num_object_copies":12966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11770,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14014","reported_seq":14148,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.948827+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:42.948827+0000","last_peered":"2024-09-15T21:26:42.948827+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:42.948827+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27328512,"num_objects":6672,"num_object_clones":0,"num_object_copies":13344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10958,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2578},{"shards":"0,2","objects":424}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14049","reported_seq":14182,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.986790+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:42.986790+0000","last_peered":"2024-09-15T21:26:42.986790+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:42.986790+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26791936,"num_objects":6541,"num_object_clones":0,"num_object_copies":13082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10958,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2402},{"shards":"0,2","objects":389}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13890","reported_seq":20900,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.978228+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:42.978228+0000","last_peered":"2024-09-15T21:26:42.978228+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:42.978228+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26615808,"num_objects":6498,"num_object_clones":0,"num_object_copies":12996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":261,"num_objects_misplaced":386,"num_objects_unfound":0,"num_objects_dirty":6498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10374,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3437,"num_bytes_recovered":14069760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5851},{"shards":"0,2","objects":386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14164","reported_seq":14304,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.865647+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:42.865647+0000","last_peered":"2024-09-15T21:26:42.865647+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:42.865647+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"40'4100","ondisk_log_start":"40'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27452358,"num_objects":6704,"num_object_clones":0,"num_object_copies":13408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10868,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2557},{"shards":"0,2","objects":422}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":433521398,"num_objects":105850,"num_object_clones":0,"num_object_copies":211700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41486,"num_objects_misplaced":386,"num_objects_unfound":0,"num_objects_dirty":105850,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":184605,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7602,"num_bytes_recovered":31121408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1051402240,"data_stored":1051253189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659272,"kb_used_data":342964,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93712568,"statfs":{"total":96636764160,"available":95961669632,"internally_reserved":0,"allocated":351195136,"data_stored":350919489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182485,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":777780,"kb_used_data":385180,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93594060,"statfs":{"total":96636764160,"available":95840317440,"internally_reserved":0,"allocated":394424320,"data_stored":394148668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,0,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634980,"kb_used_data":300628,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93736860,"statfs":{"total":96636764160,"available":95986544640,"internally_reserved":0,"allocated":307843072,"data_stored":307530831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":393908224,"data_stored":393872094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307417088,"data_stored":307344369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350076928,"data_stored":350036726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:47.485 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:47.485 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14285", 'reported_seq': 14361, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:42.950333+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:26:42.950333+0000', 'last_peered': '2024-09-15T21:26:42.950333+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:26:42.950333+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28127232, 'num_objects': 6867, 'num_object_clones': 0, 'num_object_copies': 13734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6867, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14285, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2735}, {'shards': '0,2', 'objects': 423}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:47.485 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:26:47.485 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'14420", 'reported_seq': 14547, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:44.032619+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:26:44.032619+0000', 'last_peered': '2024-09-15T21:26:44.032619+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:26:44.032619+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27381784, 'num_objects': 6686, 'num_object_clones': 0, 'num_object_copies': 13372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6686, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12520, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 402}, {'shards': '2', 'objects': 2419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:47.485 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:47.485 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'13987", 'reported_seq': 14124, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:42.937233+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:26:42.937233+0000', 'last_peered': '2024-09-15T21:26:42.937233+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:26:42.937233+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "39'3900", 'ondisk_log_start': "39'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27230691, 'num_objects': 6649, 'num_object_clones': 0, 'num_object_copies': 13298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6649, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10663, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2573}, {'shards': '0,2', 'objects': 409}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:47.485 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:47.485 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14071", 'reported_seq': 14207, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:42.904422+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:26:42.904422+0000', 'last_peered': '2024-09-15T21:26:42.904422+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:26:42.904422+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "39'4000", 'ondisk_log_start': "39'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27030470, 'num_objects': 6601, 'num_object_clones': 0, 'num_object_copies': 13202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6601, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10382, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2440}, {'shards': '0,2', 'objects': 428}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:47.486 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:26:47.486 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'13967", 'reported_seq': 14093, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:43.949514+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:26:43.949514+0000', 'last_peered': '2024-09-15T21:26:43.949514+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:26:43.949514+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "39'3900", 'ondisk_log_start': "39'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26812899, 'num_objects': 6547, 'num_object_clones': 0, 'num_object_copies': 13094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6547, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11899, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 378}, {'shards': '2', 'objects': 2460}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:47.486 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:47.486 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'13855", 'reported_seq': 13987, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:42.938230+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:26:42.938230+0000', 'last_peered': '2024-09-15T21:26:42.938230+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:26:42.938230+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27144192, 'num_objects': 6627, 'num_object_clones': 0, 'num_object_copies': 13254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6627, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10528, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2607}, {'shards': '0,2', 'objects': 410}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:47.486 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:47.486 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14024", 'reported_seq': 14144, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:44.037103+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:26:44.037103+0000', 'last_peered': '2024-09-15T21:26:44.037103+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:26:44.037103+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "40'4000", 'ondisk_log_start': "40'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27111907, 'num_objects': 6620, 'num_object_clones': 0, 'num_object_copies': 13240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6620, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11883, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 390}, {'shards': '2', 'objects': 2530}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:47.486 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:47.486 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14014", 'reported_seq': 14148, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:42.948827+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:26:42.948827+0000', 'last_peered': '2024-09-15T21:26:42.948827+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:26:42.948827+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "40'4000", 'ondisk_log_start': "40'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27328512, 'num_objects': 6672, 'num_object_clones': 0, 'num_object_copies': 13344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10958, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2578}, {'shards': '0,2', 'objects': 424}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:47.486 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:26:47.486 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14049", 'reported_seq': 14182, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:42.986790+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:26:42.986790+0000', 'last_peered': '2024-09-15T21:26:42.986790+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:26:42.986790+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "39'4000", 'ondisk_log_start': "39'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26791936, 'num_objects': 6541, 'num_object_clones': 0, 'num_object_copies': 13082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6541, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10958, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2402}, {'shards': '0,2', 'objects': 389}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:47.487 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:47.487 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'13890", 'reported_seq': 20900, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:42.978228+0000', 'last_change': '2024-09-15T21:26:05.146366+0000', 'last_active': '2024-09-15T21:26:42.978228+0000', 'last_peered': '2024-09-15T21:26:42.978228+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:26:42.978228+0000', 'last_undegraded': '2024-09-15T21:26:05.134624+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26615808, 'num_objects': 6498, 'num_object_clones': 0, 'num_object_copies': 12996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 261, 'num_objects_misplaced': 386, 'num_objects_unfound': 0, 'num_objects_dirty': 6498, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10374, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3437, 'num_bytes_recovered': 14069760, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5851}, {'shards': '0,2', 'objects': 386}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:47.487 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:26:47.487 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'14164", 'reported_seq': 14304, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:42.865647+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:26:42.865647+0000', 'last_peered': '2024-09-15T21:26:42.865647+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:26:42.865647+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "40'4100", 'ondisk_log_start': "40'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27452358, 'num_objects': 6704, 'num_object_clones': 0, 'num_object_copies': 13408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6704, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10868, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2557}, {'shards': '0,2', 'objects': 422}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:47.487 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:48.217 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:48.217 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41486/211704 objects degraded (19.596%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":459,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":105852,"data_bytes":433521417,"bytes_used":2121760768,"bytes_avail":287788531712,"bytes_total":289910292480,"degraded_objects":41486,"degraded_total":211704,"degraded_ratio":0.19596228696670825,"misplaced_objects":386,"misplaced_total":211704,"misplaced_ratio":0.0018233004572421872,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":81900,"recovering_keys_per_sec":0,"num_objects_recovered":240,"num_bytes_recovered":983040,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":388},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:48.244 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:48.244 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:48.791 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:48.792 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:48.821 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-15T21:26:46.795399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":433521417,"num_objects":105852,"num_object_clones":0,"num_object_copies":211704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41486,"num_objects_misplaced":386,"num_objects_unfound":0,"num_objects_dirty":105852,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":184607,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7603,"num_bytes_recovered":31121427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2072032,"kb_used_data":1028772,"kb_used_omap":12,"kb_used_meta":1043187,"kb_avail":281043488,"statfs":{"total":289910292480,"available":287788531712,"internally_reserved":0,"allocated":1053462528,"data_stored":1052598988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1068223491},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,7,4,5,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19079168,"num_objects":-4658,"num_object_clones":0,"num_object_copies":-9316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":-4658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4658,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002816"},"pg_stats":[{"pgid":"2.f","version":"45'14285","reported_seq":14361,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.950333+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:42.950333+0000","last_peered":"2024-09-15T21:26:42.950333+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:42.950333+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28127232,"num_objects":6867,"num_object_clones":0,"num_object_copies":13734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14285,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2735},{"shards":"0,2","objects":423}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14025","reported_seq":14131,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:43.859062+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:43.859062+0000","last_peered":"2024-09-15T21:26:43.859062+0000","last_clean":"2024-09-15T21:26:43.859062+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:43.859062+0000","last_undegraded":"2024-09-15T21:26:43.859062+0000","last_fullsized":"2024-09-15T21:26:43.859062+0000","mapping_epoch":35,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27235270,"num_objects":6651,"num_object_clones":0,"num_object_copies":13302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6651,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12306,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14420","reported_seq":14547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.032619+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:44.032619+0000","last_peered":"2024-09-15T21:26:44.032619+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:44.032619+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27381784,"num_objects":6686,"num_object_clones":0,"num_object_copies":13372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6686,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12520,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":402},{"shards":"2","objects":2419}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13830","reported_seq":13901,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:43.929009+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:43.929009+0000","last_peered":"2024-09-15T21:26:43.929009+0000","last_clean":"2024-09-15T21:26:43.929009+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:43.929009+0000","last_undegraded":"2024-09-15T21:26:43.929009+0000","last_fullsized":"2024-09-15T21:26:43.929009+0000","mapping_epoch":35,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26558464,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13830,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'13921","reported_seq":14055,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:42.870469+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:42.870469+0000","last_peered":"2024-09-15T21:26:42.870469+0000","last_clean":"2024-09-15T21:26:42.870469+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:42.870469+0000","last_undegraded":"2024-09-15T21:26:42.870469+0000","last_fullsized":"2024-09-15T21:26:42.870469+0000","mapping_epoch":34,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26968547,"num_objects":6585,"num_object_clones":0,"num_object_copies":13170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10376,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'13987","reported_seq":14124,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.937233+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:42.937233+0000","last_peered":"2024-09-15T21:26:42.937233+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:42.937233+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27230691,"num_objects":6649,"num_object_clones":0,"num_object_copies":13298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10663,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2573},{"shards":"0,2","objects":409}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14071","reported_seq":14207,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.904422+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:42.904422+0000","last_peered":"2024-09-15T21:26:42.904422+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:42.904422+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27030470,"num_objects":6601,"num_object_clones":0,"num_object_copies":13202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10382,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2440},{"shards":"0,2","objects":428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14233","reported_seq":14363,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:42.922161+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:42.922161+0000","last_peered":"2024-09-15T21:26:42.922161+0000","last_clean":"2024-09-15T21:26:42.922161+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:42.922161+0000","last_undegraded":"2024-09-15T21:26:42.922161+0000","last_fullsized":"2024-09-15T21:26:42.922161+0000","mapping_epoch":34,"log_start":"40'4200","ondisk_log_start":"40'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27176960,"num_objects":6635,"num_object_clones":0,"num_object_copies":13270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11005,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'13967","reported_seq":14093,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:43.949514+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:43.949514+0000","last_peered":"2024-09-15T21:26:43.949514+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:43.949514+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26812899,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11899,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":378},{"shards":"2","objects":2460}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'13855","reported_seq":13987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.938230+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:42.938230+0000","last_peered":"2024-09-15T21:26:42.938230+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:42.938230+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27144192,"num_objects":6627,"num_object_clones":0,"num_object_copies":13254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10528,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2607},{"shards":"0,2","objects":410}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14024","reported_seq":14144,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:44.037103+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:44.037103+0000","last_peered":"2024-09-15T21:26:44.037103+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:44.037103+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27111907,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11883,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":390},{"shards":"2","objects":2530}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14077","reported_seq":22536,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:44.017911+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:44.017911+0000","last_peered":"2024-09-15T21:26:44.017911+0000","last_clean":"2024-09-15T21:26:44.017911+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:44.017911+0000","last_undegraded":"2024-09-15T21:26:44.017911+0000","last_fullsized":"2024-09-15T21:26:44.017911+0000","mapping_epoch":44,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26554368,"num_objects":6483,"num_object_clones":0,"num_object_copies":12966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11770,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14014","reported_seq":14148,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.948827+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:42.948827+0000","last_peered":"2024-09-15T21:26:42.948827+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:42.948827+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27328512,"num_objects":6672,"num_object_clones":0,"num_object_copies":13344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10958,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2578},{"shards":"0,2","objects":424}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14049","reported_seq":14182,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.986790+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:42.986790+0000","last_peered":"2024-09-15T21:26:42.986790+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:42.986790+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"39'4000","ondisk_log_start":"39'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26791936,"num_objects":6541,"num_object_clones":0,"num_object_copies":13082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10958,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2402},{"shards":"0,2","objects":389}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'13890","reported_seq":20900,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.978228+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:42.978228+0000","last_peered":"2024-09-15T21:26:42.978228+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:42.978228+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26615808,"num_objects":6498,"num_object_clones":0,"num_object_copies":12996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":261,"num_objects_misplaced":386,"num_objects_unfound":0,"num_objects_dirty":6498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10374,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3437,"num_bytes_recovered":14069760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5851},{"shards":"0,2","objects":386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14164","reported_seq":14304,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:42.865647+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:42.865647+0000","last_peered":"2024-09-15T21:26:42.865647+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:42.865647+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"40'4100","ondisk_log_start":"40'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27452358,"num_objects":6704,"num_object_clones":0,"num_object_copies":13408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10868,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2557},{"shards":"0,2","objects":422}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":433521398,"num_objects":105850,"num_object_clones":0,"num_object_copies":211700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41486,"num_objects_misplaced":386,"num_objects_unfound":0,"num_objects_dirty":105850,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":184605,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7602,"num_bytes_recovered":31121408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1051402240,"data_stored":1051253189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":659272,"kb_used_data":342964,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93712568,"statfs":{"total":96636764160,"available":95961669632,"internally_reserved":0,"allocated":351195136,"data_stored":350919489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182485,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":777780,"kb_used_data":385180,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93594060,"statfs":{"total":96636764160,"available":95840317440,"internally_reserved":0,"allocated":394424320,"data_stored":394148668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,4,0,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634980,"kb_used_data":300628,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93736860,"statfs":{"total":96636764160,"available":95986544640,"internally_reserved":0,"allocated":307843072,"data_stored":307530831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":393908224,"data_stored":393872094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307417088,"data_stored":307344369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350076928,"data_stored":350036726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:51.824 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:52.397 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:52.398 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:52.428 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":269,"stamp":"2024-09-15T21:26:50.796294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":424030985,"num_objects":103535,"num_object_clones":0,"num_object_copies":207070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41488,"num_objects_misplaced":248,"num_objects_unfound":0,"num_objects_dirty":103535,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":186924,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7739,"num_bytes_recovered":31678483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160832,"ondisk_log_size":160832,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2072236,"kb_used_data":1016504,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281043284,"statfs":{"total":289910292480,"available":287788322816,"internally_reserved":0,"allocated":1040900096,"data_stored":1040036556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1081003011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,3,6,14,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22618112,"num_objects":-5522,"num_object_clones":0,"num_object_copies":-11044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":-271,"num_objects_unfound":0,"num_objects_dirty":-5522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5522,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":1056768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002783"},"pg_stats":[{"pgid":"2.f","version":"45'14450","reported_seq":14526,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.963243+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:47.963243+0000","last_peered":"2024-09-15T21:26:47.963243+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:47.963243+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27451392,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14450,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2570},{"shards":"0,2","objects":258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14161","reported_seq":14267,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:48.999243+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:48.999243+0000","last_peered":"2024-09-15T21:26:48.999243+0000","last_clean":"2024-09-15T21:26:48.999243+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:48.999243+0000","last_undegraded":"2024-09-15T21:26:48.999243+0000","last_fullsized":"2024-09-15T21:26:48.999243+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26678214,"num_objects":6515,"num_object_clones":0,"num_object_copies":13030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6515,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12442,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14565","reported_seq":14692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.041000+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:49.041000+0000","last_peered":"2024-09-15T21:26:49.041000+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:49.041000+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26787864,"num_objects":6541,"num_object_clones":0,"num_object_copies":13082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6541,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12665,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":257},{"shards":"2","objects":2274}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13967","reported_seq":14038,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:48.939715+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:48.939715+0000","last_peered":"2024-09-15T21:26:48.939715+0000","last_clean":"2024-09-15T21:26:48.939715+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:48.939715+0000","last_undegraded":"2024-09-15T21:26:48.939715+0000","last_fullsized":"2024-09-15T21:26:48.939715+0000","mapping_epoch":35,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25997312,"num_objects":6347,"num_object_clones":0,"num_object_copies":12694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13967,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14068","reported_seq":14201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:47.946948+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:47.946948+0000","last_peered":"2024-09-15T21:26:47.946948+0000","last_clean":"2024-09-15T21:26:47.946948+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:47.946948+0000","last_undegraded":"2024-09-15T21:26:47.946948+0000","last_fullsized":"2024-09-15T21:26:47.946948+0000","mapping_epoch":34,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26366435,"num_objects":6438,"num_object_clones":0,"num_object_copies":12876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10523,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14136","reported_seq":14273,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.955226+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:47.955226+0000","last_peered":"2024-09-15T21:26:47.955226+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:47.955226+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26620387,"num_objects":6500,"num_object_clones":0,"num_object_copies":13000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6500,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10812,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2424},{"shards":"0,2","objects":260}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14217","reported_seq":14354,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.888113+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:47.888113+0000","last_peered":"2024-09-15T21:26:47.888113+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:47.888113+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26432454,"num_objects":6455,"num_object_clones":0,"num_object_copies":12910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10528,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2294},{"shards":"0,2","objects":282}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14393","reported_seq":14523,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:47.893882+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:47.893882+0000","last_peered":"2024-09-15T21:26:47.893882+0000","last_clean":"2024-09-15T21:26:47.893882+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:47.893882+0000","last_undegraded":"2024-09-15T21:26:47.893882+0000","last_fullsized":"2024-09-15T21:26:47.893882+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26521600,"num_objects":6475,"num_object_clones":0,"num_object_copies":12950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11165,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14104","reported_seq":14232,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:48.999437+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:48.999437+0000","last_peered":"2024-09-15T21:26:48.999437+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:48.999437+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26251747,"num_objects":6410,"num_object_clones":0,"num_object_copies":12820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6410,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12036,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":241},{"shards":"2","objects":2323}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14000","reported_seq":14132,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.959176+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:47.959176+0000","last_peered":"2024-09-15T21:26:47.959176+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:47.959176+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26550272,"num_objects":6482,"num_object_clones":0,"num_object_copies":12964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10673,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2462},{"shards":"0,2","objects":265}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14161","reported_seq":14281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.007629+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:49.007629+0000","last_peered":"2024-09-15T21:26:49.007629+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:49.007629+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26550755,"num_objects":6483,"num_object_clones":0,"num_object_copies":12966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12020,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":253},{"shards":"2","objects":2393}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14218","reported_seq":22678,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:49.053475+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:49.053475+0000","last_peered":"2024-09-15T21:26:49.053475+0000","last_clean":"2024-09-15T21:26:49.053475+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:49.053475+0000","last_undegraded":"2024-09-15T21:26:49.053475+0000","last_fullsized":"2024-09-15T21:26:49.053475+0000","mapping_epoch":44,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25976832,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11911,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14170","reported_seq":14305,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.956354+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:47.956354+0000","last_peered":"2024-09-15T21:26:47.956354+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:47.956354+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11114,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2422},{"shards":"0,2","objects":268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14174","reported_seq":14308,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.875972+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:47.875972+0000","last_peered":"2024-09-15T21:26:47.875972+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:47.875972+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"40'4100","ondisk_log_start":"40'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26279936,"num_objects":6416,"num_object_clones":0,"num_object_copies":12832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11083,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2277},{"shards":"0,2","objects":264}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14024","reported_seq":21306,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.985325+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:47.985325+0000","last_peered":"2024-09-15T21:26:47.985325+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:47.985325+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26066944,"num_objects":6364,"num_object_clones":0,"num_object_copies":12728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":263,"num_objects_misplaced":248,"num_objects_unfound":0,"num_objects_dirty":6364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10508,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3573,"num_bytes_recovered":14626816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5853},{"shards":"0,2","objects":248}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14321","reported_seq":14461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.931723+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:47.931723+0000","last_peered":"2024-09-15T21:26:47.931723+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:47.931723+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26809286,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11025,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2400},{"shards":"0,2","objects":265}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":424030966,"num_objects":103533,"num_object_clones":0,"num_object_copies":207066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41488,"num_objects_misplaced":248,"num_objects_unfound":0,"num_objects_dirty":103533,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":186922,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7738,"num_bytes_recovered":31678464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1039015936,"data_stored":1038866885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160829,"ondisk_log_size":160829,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658784,"kb_used_data":338320,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93713056,"statfs":{"total":96636764160,"available":95962169344,"internally_reserved":0,"allocated":346439680,"data_stored":346164033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":328134313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774872,"kb_used_data":378116,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93596968,"statfs":{"total":96636764160,"available":95843295232,"internally_reserved":0,"allocated":387190784,"data_stored":386915132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638580,"kb_used_data":300068,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93733260,"statfs":{"total":96636764160,"available":95982858240,"internally_reserved":0,"allocated":307269632,"data_stored":306957391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":386682880,"data_stored":386646750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306847744,"data_stored":306775025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":345485312,"data_stored":345445110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:52.429 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:52.994 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:52.995 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:53.017 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-15T21:26:52.799545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":424030985,"num_objects":103535,"num_object_clones":0,"num_object_copies":207070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41488,"num_objects_misplaced":248,"num_objects_unfound":0,"num_objects_dirty":103535,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":186924,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7739,"num_bytes_recovered":31678483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160832,"ondisk_log_size":160832,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2072236,"kb_used_data":1016504,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281043284,"statfs":{"total":289910292480,"available":287788322816,"internally_reserved":0,"allocated":1040900096,"data_stored":1040036556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1081003011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,3,6,14,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19042304,"num_objects":-4649,"num_object_clones":0,"num_object_copies":-9298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":-271,"num_objects_unfound":0,"num_objects_dirty":-4649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4649,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":1056768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005543"},"pg_stats":[{"pgid":"2.f","version":"45'14450","reported_seq":14526,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.963243+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:47.963243+0000","last_peered":"2024-09-15T21:26:47.963243+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:47.963243+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27451392,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14450,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2570},{"shards":"0,2","objects":258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14161","reported_seq":14267,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:48.999243+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:48.999243+0000","last_peered":"2024-09-15T21:26:48.999243+0000","last_clean":"2024-09-15T21:26:48.999243+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:48.999243+0000","last_undegraded":"2024-09-15T21:26:48.999243+0000","last_fullsized":"2024-09-15T21:26:48.999243+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26678214,"num_objects":6515,"num_object_clones":0,"num_object_copies":13030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6515,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12442,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14565","reported_seq":14692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.041000+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:49.041000+0000","last_peered":"2024-09-15T21:26:49.041000+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:49.041000+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26787864,"num_objects":6541,"num_object_clones":0,"num_object_copies":13082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6541,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12665,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":257},{"shards":"2","objects":2274}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13967","reported_seq":14038,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:48.939715+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:48.939715+0000","last_peered":"2024-09-15T21:26:48.939715+0000","last_clean":"2024-09-15T21:26:48.939715+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:48.939715+0000","last_undegraded":"2024-09-15T21:26:48.939715+0000","last_fullsized":"2024-09-15T21:26:48.939715+0000","mapping_epoch":35,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25997312,"num_objects":6347,"num_object_clones":0,"num_object_copies":12694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13967,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14068","reported_seq":14201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:47.946948+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:47.946948+0000","last_peered":"2024-09-15T21:26:47.946948+0000","last_clean":"2024-09-15T21:26:47.946948+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:47.946948+0000","last_undegraded":"2024-09-15T21:26:47.946948+0000","last_fullsized":"2024-09-15T21:26:47.946948+0000","mapping_epoch":34,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26366435,"num_objects":6438,"num_object_clones":0,"num_object_copies":12876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10523,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14136","reported_seq":14273,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.955226+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:47.955226+0000","last_peered":"2024-09-15T21:26:47.955226+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:47.955226+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26620387,"num_objects":6500,"num_object_clones":0,"num_object_copies":13000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6500,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10812,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2424},{"shards":"0,2","objects":260}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14217","reported_seq":14354,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.888113+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:47.888113+0000","last_peered":"2024-09-15T21:26:47.888113+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:47.888113+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26432454,"num_objects":6455,"num_object_clones":0,"num_object_copies":12910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10528,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2294},{"shards":"0,2","objects":282}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14393","reported_seq":14523,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:47.893882+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:47.893882+0000","last_peered":"2024-09-15T21:26:47.893882+0000","last_clean":"2024-09-15T21:26:47.893882+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:47.893882+0000","last_undegraded":"2024-09-15T21:26:47.893882+0000","last_fullsized":"2024-09-15T21:26:47.893882+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26521600,"num_objects":6475,"num_object_clones":0,"num_object_copies":12950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11165,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14104","reported_seq":14232,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:48.999437+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:48.999437+0000","last_peered":"2024-09-15T21:26:48.999437+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:48.999437+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26251747,"num_objects":6410,"num_object_clones":0,"num_object_copies":12820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6410,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12036,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":241},{"shards":"2","objects":2323}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14000","reported_seq":14132,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.959176+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:47.959176+0000","last_peered":"2024-09-15T21:26:47.959176+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:47.959176+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26550272,"num_objects":6482,"num_object_clones":0,"num_object_copies":12964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10673,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2462},{"shards":"0,2","objects":265}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14161","reported_seq":14281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.007629+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:49.007629+0000","last_peered":"2024-09-15T21:26:49.007629+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:49.007629+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26550755,"num_objects":6483,"num_object_clones":0,"num_object_copies":12966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12020,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":253},{"shards":"2","objects":2393}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14218","reported_seq":22678,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:49.053475+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:49.053475+0000","last_peered":"2024-09-15T21:26:49.053475+0000","last_clean":"2024-09-15T21:26:49.053475+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:49.053475+0000","last_undegraded":"2024-09-15T21:26:49.053475+0000","last_fullsized":"2024-09-15T21:26:49.053475+0000","mapping_epoch":44,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25976832,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11911,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14170","reported_seq":14305,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.956354+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:47.956354+0000","last_peered":"2024-09-15T21:26:47.956354+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:47.956354+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11114,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2422},{"shards":"0,2","objects":268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14174","reported_seq":14308,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.875972+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:47.875972+0000","last_peered":"2024-09-15T21:26:47.875972+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:47.875972+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"40'4100","ondisk_log_start":"40'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26279936,"num_objects":6416,"num_object_clones":0,"num_object_copies":12832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11083,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2277},{"shards":"0,2","objects":264}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14024","reported_seq":21306,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.985325+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:47.985325+0000","last_peered":"2024-09-15T21:26:47.985325+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:47.985325+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26066944,"num_objects":6364,"num_object_clones":0,"num_object_copies":12728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":263,"num_objects_misplaced":248,"num_objects_unfound":0,"num_objects_dirty":6364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10508,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3573,"num_bytes_recovered":14626816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5853},{"shards":"0,2","objects":248}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14321","reported_seq":14461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.931723+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:47.931723+0000","last_peered":"2024-09-15T21:26:47.931723+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:47.931723+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26809286,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11025,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2400},{"shards":"0,2","objects":265}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":424030966,"num_objects":103533,"num_object_clones":0,"num_object_copies":207066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41488,"num_objects_misplaced":248,"num_objects_unfound":0,"num_objects_dirty":103533,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":186922,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7738,"num_bytes_recovered":31678464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1039015936,"data_stored":1038866885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160829,"ondisk_log_size":160829,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658784,"kb_used_data":338320,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93713056,"statfs":{"total":96636764160,"available":95962169344,"internally_reserved":0,"allocated":346439680,"data_stored":346164033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":328134313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774872,"kb_used_data":378116,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93596968,"statfs":{"total":96636764160,"available":95843295232,"internally_reserved":0,"allocated":387190784,"data_stored":386915132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638580,"kb_used_data":300068,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93733260,"statfs":{"total":96636764160,"available":95982858240,"internally_reserved":0,"allocated":307269632,"data_stored":306957391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":386682880,"data_stored":386646750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306847744,"data_stored":306775025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":345485312,"data_stored":345445110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:53.018 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:53.018 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14450", 'reported_seq': 14526, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:47.963243+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:26:47.963243+0000', 'last_peered': '2024-09-15T21:26:47.963243+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:26:47.963243+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27451392, 'num_objects': 6702, 'num_object_clones': 0, 'num_object_copies': 13404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6702, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14450, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2570}, {'shards': '0,2', 'objects': 258}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:53.019 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:26:53.019 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'14565", 'reported_seq': 14692, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:49.041000+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:26:49.041000+0000', 'last_peered': '2024-09-15T21:26:49.041000+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:26:49.041000+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26787864, 'num_objects': 6541, 'num_object_clones': 0, 'num_object_copies': 13082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6541, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12665, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 257}, {'shards': '2', 'objects': 2274}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:53.019 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:53.019 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14136", 'reported_seq': 14273, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:47.955226+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:26:47.955226+0000', 'last_peered': '2024-09-15T21:26:47.955226+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:26:47.955226+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26620387, 'num_objects': 6500, 'num_object_clones': 0, 'num_object_copies': 13000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6500, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10812, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2424}, {'shards': '0,2', 'objects': 260}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:53.019 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:53.019 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14217", 'reported_seq': 14354, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:47.888113+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:26:47.888113+0000', 'last_peered': '2024-09-15T21:26:47.888113+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:26:47.888113+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26432454, 'num_objects': 6455, 'num_object_clones': 0, 'num_object_copies': 12910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6455, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10528, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2294}, {'shards': '0,2', 'objects': 282}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:53.019 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:26:53.019 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'14104", 'reported_seq': 14232, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:48.999437+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:26:48.999437+0000', 'last_peered': '2024-09-15T21:26:48.999437+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:26:48.999437+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26251747, 'num_objects': 6410, 'num_object_clones': 0, 'num_object_copies': 12820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6410, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12036, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 241}, {'shards': '2', 'objects': 2323}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:53.019 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:53.019 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14000", 'reported_seq': 14132, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:47.959176+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:26:47.959176+0000', 'last_peered': '2024-09-15T21:26:47.959176+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:26:47.959176+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "40'3900", 'ondisk_log_start': "40'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26550272, 'num_objects': 6482, 'num_object_clones': 0, 'num_object_copies': 12964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6482, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10673, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2462}, {'shards': '0,2', 'objects': 265}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:53.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:53.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14161", 'reported_seq': 14281, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:49.007629+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:26:49.007629+0000', 'last_peered': '2024-09-15T21:26:49.007629+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:26:49.007629+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26550755, 'num_objects': 6483, 'num_object_clones': 0, 'num_object_copies': 12966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6483, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12020, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 253}, {'shards': '2', 'objects': 2393}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:53.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:53.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14170", 'reported_seq': 14305, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:47.956354+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:26:47.956354+0000', 'last_peered': '2024-09-15T21:26:47.956354+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:26:47.956354+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26689536, 'num_objects': 6516, 'num_object_clones': 0, 'num_object_copies': 13032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6516, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11114, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2422}, {'shards': '0,2', 'objects': 268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:53.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:26:53.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14174", 'reported_seq': 14308, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:47.875972+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:26:47.875972+0000', 'last_peered': '2024-09-15T21:26:47.875972+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:26:47.875972+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "40'4100", 'ondisk_log_start': "40'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26279936, 'num_objects': 6416, 'num_object_clones': 0, 'num_object_copies': 12832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6416, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11083, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2277}, {'shards': '0,2', 'objects': 264}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:53.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:53.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14024", 'reported_seq': 21306, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:47.985325+0000', 'last_change': '2024-09-15T21:26:05.146366+0000', 'last_active': '2024-09-15T21:26:47.985325+0000', 'last_peered': '2024-09-15T21:26:47.985325+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:26:47.985325+0000', 'last_undegraded': '2024-09-15T21:26:05.134624+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "40'4000", 'ondisk_log_start': "40'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26066944, 'num_objects': 6364, 'num_object_clones': 0, 'num_object_copies': 12728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 263, 'num_objects_misplaced': 248, 'num_objects_unfound': 0, 'num_objects_dirty': 6364, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10508, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3573, 'num_bytes_recovered': 14626816, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5853}, {'shards': '0,2', 'objects': 248}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:53.020 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:26:53.020 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'14321", 'reported_seq': 14461, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:47.931723+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:26:47.931723+0000', 'last_peered': '2024-09-15T21:26:47.931723+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:26:47.931723+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26809286, 'num_objects': 6547, 'num_object_clones': 0, 'num_object_copies': 13094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6547, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11025, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2400}, {'shards': '0,2', 'objects': 265}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:53.021 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:53.683 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:53.684 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41488/207070 objects degraded (20.036%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":465,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":103535,"data_bytes":424030985,"bytes_used":2121969664,"bytes_avail":287788322816,"bytes_total":289910292480,"degraded_objects":41488,"degraded_total":207070,"degraded_ratio":0.20035736707393637,"misplaced_objects":248,"misplaced_total":207070,"misplaced_ratio":0.0011976626261650651,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":88023,"recovering_keys_per_sec":0,"num_objects_recovered":258,"num_bytes_recovered":1056768,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":387},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:53.702 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:53.702 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:54.268 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:54.271 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:54.297 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-15T21:26:52.799545+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":424030985,"num_objects":103535,"num_object_clones":0,"num_object_copies":207070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41488,"num_objects_misplaced":248,"num_objects_unfound":0,"num_objects_dirty":103535,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":186924,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7739,"num_bytes_recovered":31678483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160832,"ondisk_log_size":160832,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2072236,"kb_used_data":1016504,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281043284,"statfs":{"total":289910292480,"available":287788322816,"internally_reserved":0,"allocated":1040900096,"data_stored":1040036556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1081003011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,3,6,14,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19042304,"num_objects":-4649,"num_object_clones":0,"num_object_copies":-9298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13,"num_objects_misplaced":-271,"num_objects_unfound":0,"num_objects_dirty":-4649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4649,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":258,"num_bytes_recovered":1056768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005543"},"pg_stats":[{"pgid":"2.f","version":"45'14450","reported_seq":14526,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.963243+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:47.963243+0000","last_peered":"2024-09-15T21:26:47.963243+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:47.963243+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27451392,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14450,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2570},{"shards":"0,2","objects":258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14161","reported_seq":14267,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:48.999243+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:48.999243+0000","last_peered":"2024-09-15T21:26:48.999243+0000","last_clean":"2024-09-15T21:26:48.999243+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:48.999243+0000","last_undegraded":"2024-09-15T21:26:48.999243+0000","last_fullsized":"2024-09-15T21:26:48.999243+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26678214,"num_objects":6515,"num_object_clones":0,"num_object_copies":13030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6515,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12442,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14565","reported_seq":14692,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.041000+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:49.041000+0000","last_peered":"2024-09-15T21:26:49.041000+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:49.041000+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26787864,"num_objects":6541,"num_object_clones":0,"num_object_copies":13082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6541,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12665,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":257},{"shards":"2","objects":2274}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'13967","reported_seq":14038,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:48.939715+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:48.939715+0000","last_peered":"2024-09-15T21:26:48.939715+0000","last_clean":"2024-09-15T21:26:48.939715+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:48.939715+0000","last_undegraded":"2024-09-15T21:26:48.939715+0000","last_fullsized":"2024-09-15T21:26:48.939715+0000","mapping_epoch":35,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25997312,"num_objects":6347,"num_object_clones":0,"num_object_copies":12694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13967,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14068","reported_seq":14201,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:47.946948+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:47.946948+0000","last_peered":"2024-09-15T21:26:47.946948+0000","last_clean":"2024-09-15T21:26:47.946948+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:47.946948+0000","last_undegraded":"2024-09-15T21:26:47.946948+0000","last_fullsized":"2024-09-15T21:26:47.946948+0000","mapping_epoch":34,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26366435,"num_objects":6438,"num_object_clones":0,"num_object_copies":12876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10523,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14136","reported_seq":14273,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.955226+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:47.955226+0000","last_peered":"2024-09-15T21:26:47.955226+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:47.955226+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26620387,"num_objects":6500,"num_object_clones":0,"num_object_copies":13000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6500,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10812,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2424},{"shards":"0,2","objects":260}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14217","reported_seq":14354,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.888113+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:47.888113+0000","last_peered":"2024-09-15T21:26:47.888113+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:47.888113+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26432454,"num_objects":6455,"num_object_clones":0,"num_object_copies":12910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10528,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2294},{"shards":"0,2","objects":282}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14393","reported_seq":14523,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:47.893882+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:47.893882+0000","last_peered":"2024-09-15T21:26:47.893882+0000","last_clean":"2024-09-15T21:26:47.893882+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:47.893882+0000","last_undegraded":"2024-09-15T21:26:47.893882+0000","last_fullsized":"2024-09-15T21:26:47.893882+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26521600,"num_objects":6475,"num_object_clones":0,"num_object_copies":12950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11165,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14104","reported_seq":14232,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:48.999437+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:48.999437+0000","last_peered":"2024-09-15T21:26:48.999437+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:48.999437+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26251747,"num_objects":6410,"num_object_clones":0,"num_object_copies":12820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6410,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12036,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":241},{"shards":"2","objects":2323}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14000","reported_seq":14132,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.959176+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:47.959176+0000","last_peered":"2024-09-15T21:26:47.959176+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:47.959176+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26550272,"num_objects":6482,"num_object_clones":0,"num_object_copies":12964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10673,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2462},{"shards":"0,2","objects":265}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14161","reported_seq":14281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:49.007629+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:49.007629+0000","last_peered":"2024-09-15T21:26:49.007629+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:49.007629+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26550755,"num_objects":6483,"num_object_clones":0,"num_object_copies":12966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12020,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":253},{"shards":"2","objects":2393}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14218","reported_seq":22678,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:49.053475+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:49.053475+0000","last_peered":"2024-09-15T21:26:49.053475+0000","last_clean":"2024-09-15T21:26:49.053475+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:49.053475+0000","last_undegraded":"2024-09-15T21:26:49.053475+0000","last_fullsized":"2024-09-15T21:26:49.053475+0000","mapping_epoch":44,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25976832,"num_objects":6342,"num_object_clones":0,"num_object_copies":12684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11911,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14170","reported_seq":14305,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.956354+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:47.956354+0000","last_peered":"2024-09-15T21:26:47.956354+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:47.956354+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11114,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2422},{"shards":"0,2","objects":268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14174","reported_seq":14308,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.875972+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:47.875972+0000","last_peered":"2024-09-15T21:26:47.875972+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:47.875972+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"40'4100","ondisk_log_start":"40'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26279936,"num_objects":6416,"num_object_clones":0,"num_object_copies":12832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11083,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2277},{"shards":"0,2","objects":264}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14024","reported_seq":21306,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.985325+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:47.985325+0000","last_peered":"2024-09-15T21:26:47.985325+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:47.985325+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26066944,"num_objects":6364,"num_object_clones":0,"num_object_copies":12728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":263,"num_objects_misplaced":248,"num_objects_unfound":0,"num_objects_dirty":6364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10508,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3573,"num_bytes_recovered":14626816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5853},{"shards":"0,2","objects":248}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14321","reported_seq":14461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:47.931723+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:47.931723+0000","last_peered":"2024-09-15T21:26:47.931723+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:47.931723+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26809286,"num_objects":6547,"num_object_clones":0,"num_object_copies":13094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6547,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11025,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2400},{"shards":"0,2","objects":265}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":424030966,"num_objects":103533,"num_object_clones":0,"num_object_copies":207066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41488,"num_objects_misplaced":248,"num_objects_unfound":0,"num_objects_dirty":103533,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":186922,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7738,"num_bytes_recovered":31678464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1039015936,"data_stored":1038866885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160829,"ondisk_log_size":160829,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658784,"kb_used_data":338320,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93713056,"statfs":{"total":96636764160,"available":95962169344,"internally_reserved":0,"allocated":346439680,"data_stored":346164033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":328134313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":774872,"kb_used_data":378116,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93596968,"statfs":{"total":96636764160,"available":95843295232,"internally_reserved":0,"allocated":387190784,"data_stored":386915132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":638580,"kb_used_data":300068,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93733260,"statfs":{"total":96636764160,"available":95982858240,"internally_reserved":0,"allocated":307269632,"data_stored":306957391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":386682880,"data_stored":386646750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306847744,"data_stored":306775025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":345485312,"data_stored":345445110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:57.300 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:57.854 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:57.857 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:57.877 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-15T21:26:56.800480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":414638857,"num_objects":101242,"num_object_clones":0,"num_object_copies":202484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41494,"num_objects_misplaced":104,"num_objects_unfound":0,"num_objects_dirty":101242,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":189217,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7877,"num_bytes_recovered":32243731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160725,"ondisk_log_size":160725,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2081024,"kb_used_data":1004484,"kb_used_omap":12,"kb_used_meta":1076467,"kb_avail":281034496,"statfs":{"total":289910292480,"available":287779323904,"internally_reserved":0,"allocated":1028591616,"data_stored":1027728076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1102302211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,5,7,9,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18882560,"num_objects":-4610,"num_object_clones":0,"num_object_copies":-9220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-4610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4610,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005600"},"pg_stats":[{"pgid":"2.f","version":"45'14586","reported_seq":14663,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.964039+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:52.964039+0000","last_peered":"2024-09-15T21:26:52.964039+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:52.964039+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26894336,"num_objects":6566,"num_object_clones":0,"num_object_copies":13132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14586,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2434},{"shards":"0,2","objects":122}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14294","reported_seq":14400,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:53.924867+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:53.924867+0000","last_peered":"2024-09-15T21:26:53.924867+0000","last_clean":"2024-09-15T21:26:53.924867+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:53.924867+0000","last_undegraded":"2024-09-15T21:26:53.924867+0000","last_fullsized":"2024-09-15T21:26:53.924867+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26133446,"num_objects":6382,"num_object_clones":0,"num_object_copies":12764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6382,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12575,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14713","reported_seq":14840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:54.020762+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:54.020762+0000","last_peered":"2024-09-15T21:26:54.020762+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:54.020762+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26181656,"num_objects":6393,"num_object_clones":0,"num_object_copies":12786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12813,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":109},{"shards":"2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14112","reported_seq":14183,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:53.931185+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:53.931185+0000","last_peered":"2024-09-15T21:26:53.931185+0000","last_clean":"2024-09-15T21:26:53.931185+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:53.931185+0000","last_undegraded":"2024-09-15T21:26:53.931185+0000","last_fullsized":"2024-09-15T21:26:53.931185+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25403392,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14112,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14214","reported_seq":14348,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:52.940742+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:52.940742+0000","last_peered":"2024-09-15T21:26:52.940742+0000","last_clean":"2024-09-15T21:26:52.940742+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:52.940742+0000","last_undegraded":"2024-09-15T21:26:52.940742+0000","last_fullsized":"2024-09-15T21:26:52.940742+0000","mapping_epoch":34,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25768419,"num_objects":6292,"num_object_clones":0,"num_object_copies":12584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10669,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14267","reported_seq":14404,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.890185+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:52.890185+0000","last_peered":"2024-09-15T21:26:52.890185+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:52.890185+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26083811,"num_objects":6369,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10943,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2293},{"shards":"0,2","objects":129}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14385","reported_seq":14521,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.936259+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:52.936259+0000","last_peered":"2024-09-15T21:26:52.936259+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:52.936259+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25744326,"num_objects":6287,"num_object_clones":0,"num_object_copies":12574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10696,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2126},{"shards":"0,2","objects":114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14534","reported_seq":14664,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:52.887886+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:52.887886+0000","last_peered":"2024-09-15T21:26:52.887886+0000","last_clean":"2024-09-15T21:26:52.887886+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:52.887886+0000","last_undegraded":"2024-09-15T21:26:52.887886+0000","last_fullsized":"2024-09-15T21:26:52.887886+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25944064,"num_objects":6334,"num_object_clones":0,"num_object_copies":12668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11306,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14253","reported_seq":14379,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:54.039473+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:54.039473+0000","last_peered":"2024-09-15T21:26:54.039473+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:54.039473+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25641443,"num_objects":6261,"num_object_clones":0,"num_object_copies":12522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6261,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12185,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":92},{"shards":"2","objects":2174}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14149","reported_seq":14281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.895526+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:52.895526+0000","last_peered":"2024-09-15T21:26:52.895526+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:52.895526+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25939968,"num_objects":6333,"num_object_clones":0,"num_object_copies":12666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10822,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2313},{"shards":"0,2","objects":116}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14303","reported_seq":14423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:53.886983+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:53.886983+0000","last_peered":"2024-09-15T21:26:53.886983+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:53.886983+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25969123,"num_objects":6341,"num_object_clones":0,"num_object_copies":12682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12162,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":111},{"shards":"2","objects":2251}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14358","reported_seq":22817,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:54.002530+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:54.002530+0000","last_peered":"2024-09-15T21:26:54.002530+0000","last_clean":"2024-09-15T21:26:54.002530+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:54.002530+0000","last_undegraded":"2024-09-15T21:26:54.002530+0000","last_fullsized":"2024-09-15T21:26:54.002530+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25403392,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12051,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14313","reported_seq":14448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.910477+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:52.910477+0000","last_peered":"2024-09-15T21:26:52.910477+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:52.910477+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26103808,"num_objects":6373,"num_object_clones":0,"num_object_copies":12746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11257,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2279},{"shards":"0,2","objects":125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14320","reported_seq":14453,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.951307+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:52.951307+0000","last_peered":"2024-09-15T21:26:52.951307+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:52.951307+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25681920,"num_objects":6270,"num_object_clones":0,"num_object_copies":12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11229,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2131},{"shards":"0,2","objects":118}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14165","reported_seq":21725,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.958455+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:52.958455+0000","last_peered":"2024-09-15T21:26:52.958455+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:52.958455+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25489408,"num_objects":6223,"num_object_clones":0,"num_object_copies":12446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":269,"num_objects_misplaced":104,"num_objects_unfound":0,"num_objects_dirty":6223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10649,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3711,"num_bytes_recovered":15192064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5850},{"shards":"0,2","objects":104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14456","reported_seq":14596,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.999375+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:52.999375+0000","last_peered":"2024-09-15T21:26:52.999375+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:52.999375+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26256326,"num_objects":6412,"num_object_clones":0,"num_object_copies":12824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6412,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11160,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2265},{"shards":"0,2","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":414638838,"num_objects":101240,"num_object_clones":0,"num_object_copies":202480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41494,"num_objects_misplaced":104,"num_objects_unfound":0,"num_objects_dirty":101240,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":189215,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7876,"num_bytes_recovered":32243712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1026699264,"data_stored":1026550213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160722,"ondisk_log_size":160722,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658388,"kb_used_data":333756,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93713452,"statfs":{"total":96636764160,"available":95962574848,"internally_reserved":0,"allocated":341766144,"data_stored":341490497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":332394153},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":776300,"kb_used_data":371228,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93595540,"statfs":{"total":96636764160,"available":95841832960,"internally_reserved":0,"allocated":380137472,"data_stored":379861820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":414772907},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,3,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646336,"kb_used_data":299500,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93725504,"statfs":{"total":96636764160,"available":95974916096,"internally_reserved":0,"allocated":306688000,"data_stored":306375759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":379666432,"data_stored":379630302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306245632,"data_stored":306172913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340787200,"data_stored":340746998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:57.879 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:58.455 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:58.455 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:58.486 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-15T21:26:56.800480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":414638857,"num_objects":101242,"num_object_clones":0,"num_object_copies":202484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41494,"num_objects_misplaced":104,"num_objects_unfound":0,"num_objects_dirty":101242,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":189217,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7877,"num_bytes_recovered":32243731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160725,"ondisk_log_size":160725,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2081024,"kb_used_data":1004484,"kb_used_omap":12,"kb_used_meta":1076467,"kb_avail":281034496,"statfs":{"total":289910292480,"available":287779323904,"internally_reserved":0,"allocated":1028591616,"data_stored":1027728076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1102302211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,5,7,9,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18882560,"num_objects":-4610,"num_object_clones":0,"num_object_copies":-9220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":-282,"num_objects_unfound":0,"num_objects_dirty":-4610,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4610,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005600"},"pg_stats":[{"pgid":"2.f","version":"45'14586","reported_seq":14663,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.964039+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:52.964039+0000","last_peered":"2024-09-15T21:26:52.964039+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:52.964039+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26894336,"num_objects":6566,"num_object_clones":0,"num_object_copies":13132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14586,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2434},{"shards":"0,2","objects":122}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14294","reported_seq":14400,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:53.924867+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:53.924867+0000","last_peered":"2024-09-15T21:26:53.924867+0000","last_clean":"2024-09-15T21:26:53.924867+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:53.924867+0000","last_undegraded":"2024-09-15T21:26:53.924867+0000","last_fullsized":"2024-09-15T21:26:53.924867+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26133446,"num_objects":6382,"num_object_clones":0,"num_object_copies":12764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6382,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12575,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14713","reported_seq":14840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:54.020762+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:54.020762+0000","last_peered":"2024-09-15T21:26:54.020762+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:54.020762+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26181656,"num_objects":6393,"num_object_clones":0,"num_object_copies":12786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12813,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":109},{"shards":"2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14112","reported_seq":14183,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:53.931185+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:53.931185+0000","last_peered":"2024-09-15T21:26:53.931185+0000","last_clean":"2024-09-15T21:26:53.931185+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:53.931185+0000","last_undegraded":"2024-09-15T21:26:53.931185+0000","last_fullsized":"2024-09-15T21:26:53.931185+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25403392,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14112,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14214","reported_seq":14348,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:52.940742+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:52.940742+0000","last_peered":"2024-09-15T21:26:52.940742+0000","last_clean":"2024-09-15T21:26:52.940742+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:52.940742+0000","last_undegraded":"2024-09-15T21:26:52.940742+0000","last_fullsized":"2024-09-15T21:26:52.940742+0000","mapping_epoch":34,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25768419,"num_objects":6292,"num_object_clones":0,"num_object_copies":12584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10669,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14267","reported_seq":14404,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.890185+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:52.890185+0000","last_peered":"2024-09-15T21:26:52.890185+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:52.890185+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26083811,"num_objects":6369,"num_object_clones":0,"num_object_copies":12738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6369,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10943,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2293},{"shards":"0,2","objects":129}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14385","reported_seq":14521,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.936259+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:52.936259+0000","last_peered":"2024-09-15T21:26:52.936259+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:52.936259+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25744326,"num_objects":6287,"num_object_clones":0,"num_object_copies":12574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10696,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2126},{"shards":"0,2","objects":114}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14534","reported_seq":14664,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:52.887886+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:52.887886+0000","last_peered":"2024-09-15T21:26:52.887886+0000","last_clean":"2024-09-15T21:26:52.887886+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:52.887886+0000","last_undegraded":"2024-09-15T21:26:52.887886+0000","last_fullsized":"2024-09-15T21:26:52.887886+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25944064,"num_objects":6334,"num_object_clones":0,"num_object_copies":12668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11306,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14253","reported_seq":14379,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:54.039473+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:54.039473+0000","last_peered":"2024-09-15T21:26:54.039473+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:54.039473+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25641443,"num_objects":6261,"num_object_clones":0,"num_object_copies":12522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6261,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12185,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":92},{"shards":"2","objects":2174}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14149","reported_seq":14281,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.895526+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:52.895526+0000","last_peered":"2024-09-15T21:26:52.895526+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:52.895526+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25939968,"num_objects":6333,"num_object_clones":0,"num_object_copies":12666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10822,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2313},{"shards":"0,2","objects":116}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14303","reported_seq":14423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:53.886983+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:53.886983+0000","last_peered":"2024-09-15T21:26:53.886983+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:53.886983+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25969123,"num_objects":6341,"num_object_clones":0,"num_object_copies":12682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12162,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":111},{"shards":"2","objects":2251}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14358","reported_seq":22817,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:54.002530+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:54.002530+0000","last_peered":"2024-09-15T21:26:54.002530+0000","last_clean":"2024-09-15T21:26:54.002530+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:54.002530+0000","last_undegraded":"2024-09-15T21:26:54.002530+0000","last_fullsized":"2024-09-15T21:26:54.002530+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25403392,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12051,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14313","reported_seq":14448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.910477+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:52.910477+0000","last_peered":"2024-09-15T21:26:52.910477+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:52.910477+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26103808,"num_objects":6373,"num_object_clones":0,"num_object_copies":12746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11257,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2279},{"shards":"0,2","objects":125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14320","reported_seq":14453,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.951307+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:52.951307+0000","last_peered":"2024-09-15T21:26:52.951307+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:52.951307+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25681920,"num_objects":6270,"num_object_clones":0,"num_object_copies":12540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11229,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2131},{"shards":"0,2","objects":118}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14165","reported_seq":21725,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.958455+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:52.958455+0000","last_peered":"2024-09-15T21:26:52.958455+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:52.958455+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25489408,"num_objects":6223,"num_object_clones":0,"num_object_copies":12446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":269,"num_objects_misplaced":104,"num_objects_unfound":0,"num_objects_dirty":6223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10649,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3711,"num_bytes_recovered":15192064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5850},{"shards":"0,2","objects":104}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14456","reported_seq":14596,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:52.999375+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:52.999375+0000","last_peered":"2024-09-15T21:26:52.999375+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:52.999375+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26256326,"num_objects":6412,"num_object_clones":0,"num_object_copies":12824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6412,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11160,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2265},{"shards":"0,2","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":414638838,"num_objects":101240,"num_object_clones":0,"num_object_copies":202480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41494,"num_objects_misplaced":104,"num_objects_unfound":0,"num_objects_dirty":101240,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":189215,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7876,"num_bytes_recovered":32243712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1026699264,"data_stored":1026550213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160722,"ondisk_log_size":160722,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658388,"kb_used_data":333756,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93713452,"statfs":{"total":96636764160,"available":95962574848,"internally_reserved":0,"allocated":341766144,"data_stored":341490497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":332394153},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":776300,"kb_used_data":371228,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93595540,"statfs":{"total":96636764160,"available":95841832960,"internally_reserved":0,"allocated":380137472,"data_stored":379861820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":414772907},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,3,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646336,"kb_used_data":299500,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93725504,"statfs":{"total":96636764160,"available":95974916096,"internally_reserved":0,"allocated":306688000,"data_stored":306375759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":379666432,"data_stored":379630302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306245632,"data_stored":306172913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340787200,"data_stored":340746998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:26:58.488 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:26:58.488 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14586", 'reported_seq': 14663, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:52.964039+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:26:52.964039+0000', 'last_peered': '2024-09-15T21:26:52.964039+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:26:52.964039+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26894336, 'num_objects': 6566, 'num_object_clones': 0, 'num_object_copies': 13132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14586, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2434}, {'shards': '0,2', 'objects': 122}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:58.488 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:26:58.488 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'14713", 'reported_seq': 14840, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:54.020762+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:26:54.020762+0000', 'last_peered': '2024-09-15T21:26:54.020762+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:26:54.020762+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26181656, 'num_objects': 6393, 'num_object_clones': 0, 'num_object_copies': 12786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6393, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12813, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 109}, {'shards': '2', 'objects': 2126}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:58.488 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:26:58.488 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14267", 'reported_seq': 14404, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:52.890185+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:26:52.890185+0000', 'last_peered': '2024-09-15T21:26:52.890185+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:26:52.890185+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26083811, 'num_objects': 6369, 'num_object_clones': 0, 'num_object_copies': 12738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6369, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10943, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2293}, {'shards': '0,2', 'objects': 129}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:58.488 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:26:58.488 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14385", 'reported_seq': 14521, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:52.936259+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:26:52.936259+0000', 'last_peered': '2024-09-15T21:26:52.936259+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:26:52.936259+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25744326, 'num_objects': 6287, 'num_object_clones': 0, 'num_object_copies': 12574, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6287, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10696, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2126}, {'shards': '0,2', 'objects': 114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:58.489 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:26:58.489 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'14253", 'reported_seq': 14379, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:54.039473+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:26:54.039473+0000', 'last_peered': '2024-09-15T21:26:54.039473+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:26:54.039473+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25641443, 'num_objects': 6261, 'num_object_clones': 0, 'num_object_copies': 12522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6261, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12185, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 92}, {'shards': '2', 'objects': 2174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:58.489 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:26:58.489 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14149", 'reported_seq': 14281, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:52.895526+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:26:52.895526+0000', 'last_peered': '2024-09-15T21:26:52.895526+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:26:52.895526+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25939968, 'num_objects': 6333, 'num_object_clones': 0, 'num_object_copies': 12666, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6333, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10822, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2313}, {'shards': '0,2', 'objects': 116}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:58.489 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:26:58.489 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14303", 'reported_seq': 14423, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:53.886983+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:26:53.886983+0000', 'last_peered': '2024-09-15T21:26:53.886983+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:26:53.886983+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25969123, 'num_objects': 6341, 'num_object_clones': 0, 'num_object_copies': 12682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6341, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12162, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 111}, {'shards': '2', 'objects': 2251}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:26:58.489 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:26:58.489 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14313", 'reported_seq': 14448, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:52.910477+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:26:52.910477+0000', 'last_peered': '2024-09-15T21:26:52.910477+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:26:52.910477+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26103808, 'num_objects': 6373, 'num_object_clones': 0, 'num_object_copies': 12746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6373, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11257, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2279}, {'shards': '0,2', 'objects': 125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:58.489 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:26:58.489 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14320", 'reported_seq': 14453, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:52.951307+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:26:52.951307+0000', 'last_peered': '2024-09-15T21:26:52.951307+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:26:52.951307+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25681920, 'num_objects': 6270, 'num_object_clones': 0, 'num_object_copies': 12540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6270, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11229, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2131}, {'shards': '0,2', 'objects': 118}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:58.490 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:26:58.490 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14165", 'reported_seq': 21725, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:52.958455+0000', 'last_change': '2024-09-15T21:26:05.146366+0000', 'last_active': '2024-09-15T21:26:52.958455+0000', 'last_peered': '2024-09-15T21:26:52.958455+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:26:52.958455+0000', 'last_undegraded': '2024-09-15T21:26:05.134624+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25489408, 'num_objects': 6223, 'num_object_clones': 0, 'num_object_copies': 12446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 269, 'num_objects_misplaced': 104, 'num_objects_unfound': 0, 'num_objects_dirty': 6223, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10649, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3711, 'num_bytes_recovered': 15192064, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5850}, {'shards': '0,2', 'objects': 104}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:58.490 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:26:58.490 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'14456", 'reported_seq': 14596, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:52.999375+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:26:52.999375+0000', 'last_peered': '2024-09-15T21:26:52.999375+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:26:52.999375+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26256326, 'num_objects': 6412, 'num_object_clones': 0, 'num_object_copies': 12824, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6412, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11160, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2265}, {'shards': '0,2', 'objects': 130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:26:58.490 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:26:59.222 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:59.222 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41494/202484 objects degraded (20.492%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":470,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":101242,"data_bytes":414638857,"bytes_used":2130968576,"bytes_avail":287779323904,"bytes_total":289910292480,"degraded_objects":41494,"degraded_total":202484,"degraded_ratio":0.20492483356709668,"misplaced_objects":104,"misplaced_total":202484,"misplaced_ratio":0.0005136208293000928,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":93481,"recovering_keys_per_sec":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":383},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:26:59.242 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:26:59.243 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:26:59.789 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:26:59.790 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:26:59.813 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-09-15T21:26:58.800913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":408605449,"num_objects":99769,"num_object_clones":0,"num_object_copies":199538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99769,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":190690,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8002,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160698,"ondisk_log_size":160698,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2061632,"kb_used_data":997572,"kb_used_omap":12,"kb_used_meta":1063987,"kb_avail":281053888,"statfs":{"total":289910292480,"available":287799181312,"internally_reserved":0,"allocated":1021513728,"data_stored":1020650188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1089522691},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,3,5,8,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24915968,"num_objects":-6083,"num_object_clones":0,"num_object_copies":-12166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-13,"num_objects_misplaced":-386,"num_objects_unfound":0,"num_objects_dirty":-6083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6083,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":399,"num_bytes_recovered":1507328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005514"},"pg_stats":[{"pgid":"2.f","version":"45'14732","reported_seq":14807,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.023168+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:58.023168+0000","last_peered":"2024-09-15T21:26:58.023168+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:58.023168+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26296320,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14732,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2288}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14294","reported_seq":14400,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:53.924867+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:53.924867+0000","last_peered":"2024-09-15T21:26:53.924867+0000","last_clean":"2024-09-15T21:26:53.924867+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:53.924867+0000","last_undegraded":"2024-09-15T21:26:53.924867+0000","last_fullsized":"2024-09-15T21:26:53.924867+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26133446,"num_objects":6382,"num_object_clones":0,"num_object_copies":12764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6382,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12575,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14713","reported_seq":14840,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:54.020762+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:54.020762+0000","last_peered":"2024-09-15T21:26:54.020762+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:54.020762+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26181656,"num_objects":6393,"num_object_clones":0,"num_object_copies":12786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12813,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":109},{"shards":"2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14112","reported_seq":14183,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:53.931185+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:53.931185+0000","last_peered":"2024-09-15T21:26:53.931185+0000","last_clean":"2024-09-15T21:26:53.931185+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:53.931185+0000","last_undegraded":"2024-09-15T21:26:53.931185+0000","last_fullsized":"2024-09-15T21:26:53.931185+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25403392,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14112,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14357","reported_seq":14491,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.029399+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:58.029399+0000","last_peered":"2024-09-15T21:26:58.029399+0000","last_clean":"2024-09-15T21:26:58.029399+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:58.029399+0000","last_undegraded":"2024-09-15T21:26:58.029399+0000","last_fullsized":"2024-09-15T21:26:58.029399+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25182691,"num_objects":6149,"num_object_clones":0,"num_object_copies":12298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10812,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14429","reported_seq":14566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.032601+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:58.032601+0000","last_peered":"2024-09-15T21:26:58.032601+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:58.032601+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25420259,"num_objects":6207,"num_object_clones":0,"num_object_copies":12414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11105,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2131},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14519","reported_seq":14656,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:57.886205+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:57.886205+0000","last_peered":"2024-09-15T21:26:57.886205+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:57.886205+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25195462,"num_objects":6153,"num_object_clones":0,"num_object_copies":12306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10830,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1992},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14687","reported_seq":14817,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:57.951961+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:57.951961+0000","last_peered":"2024-09-15T21:26:57.951961+0000","last_clean":"2024-09-15T21:26:57.951961+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:57.951961+0000","last_undegraded":"2024-09-15T21:26:57.951961+0000","last_fullsized":"2024-09-15T21:26:57.951961+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25317376,"num_objects":6181,"num_object_clones":0,"num_object_copies":12362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11459,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14253","reported_seq":14379,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:54.039473+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:54.039473+0000","last_peered":"2024-09-15T21:26:54.039473+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:54.039473+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25641443,"num_objects":6261,"num_object_clones":0,"num_object_copies":12522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6261,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12185,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":92},{"shards":"2","objects":2174}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14290","reported_seq":14422,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.015318+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:58.015318+0000","last_peered":"2024-09-15T21:26:58.015318+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:58.015318+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362432,"num_objects":6192,"num_object_clones":0,"num_object_copies":12384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10963,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2172}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14303","reported_seq":14423,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:53.886983+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:53.886983+0000","last_peered":"2024-09-15T21:26:53.886983+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:53.886983+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25969123,"num_objects":6341,"num_object_clones":0,"num_object_copies":12682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12162,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":111},{"shards":"2","objects":2251}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14358","reported_seq":22817,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:54.002530+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:54.002530+0000","last_peered":"2024-09-15T21:26:54.002530+0000","last_clean":"2024-09-15T21:26:54.002530+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:54.002530+0000","last_undegraded":"2024-09-15T21:26:54.002530+0000","last_fullsized":"2024-09-15T21:26:54.002530+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25403392,"num_objects":6202,"num_object_clones":0,"num_object_copies":12404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12051,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14467","reported_seq":14600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.042218+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:58.042218+0000","last_peered":"2024-09-15T21:26:58.042218+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:58.042218+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25473024,"num_objects":6219,"num_object_clones":0,"num_object_copies":12438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11411,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14459","reported_seq":14593,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:57.972187+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:57.972187+0000","last_peered":"2024-09-15T21:26:57.972187+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:57.972187+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25112576,"num_objects":6131,"num_object_clones":0,"num_object_copies":12262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11368,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1992}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14302","reported_seq":22112,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.034347+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:58.034347+0000","last_peered":"2024-09-15T21:26:58.034347+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:58.034347+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24928256,"num_objects":6086,"num_object_clones":0,"num_object_copies":12172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10786,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3836,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14620","reported_seq":14760,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:57.959452+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:57.959452+0000","last_peered":"2024-09-15T21:26:57.959452+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:57.959452+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25584582,"num_objects":6248,"num_object_clones":0,"num_object_copies":12496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6248,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11324,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2101},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":408605430,"num_objects":99767,"num_object_clones":0,"num_object_copies":199534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99767,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":190688,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8001,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1019490304,"data_stored":1019341253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160695,"ondisk_log_size":160695,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":658388,"kb_used_data":333756,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93713452,"statfs":{"total":96636764160,"available":95962574848,"internally_reserved":0,"allocated":341766144,"data_stored":341490497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":332394153},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":756908,"kb_used_data":364316,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93614932,"statfs":{"total":96636764160,"available":95861690368,"internally_reserved":0,"allocated":373059584,"data_stored":372783932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,3,2,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646336,"kb_used_data":299500,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93725504,"statfs":{"total":96636764160,"available":95974916096,"internally_reserved":0,"allocated":306688000,"data_stored":306375759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":372457472,"data_stored":372421342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306245632,"data_stored":306172913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":340787200,"data_stored":340746998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:02.816 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:03.375 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:03.377 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:03.397 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-15T21:27:02.801999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":404943625,"num_objects":98875,"num_object_clones":0,"num_object_copies":197750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98875,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":191584,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8002,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2045876,"kb_used_data":990148,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281069644,"statfs":{"total":289910292480,"available":287815315456,"internally_reserved":0,"allocated":1013911552,"data_stored":1013048012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1081003011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,5,12,2,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19087360,"num_objects":-4660,"num_object_clones":0,"num_object_copies":-9320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-15,"num_objects_misplaced":-248,"num_objects_unfound":0,"num_objects_dirty":-4660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4660,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":950272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005704"},"pg_stats":[{"pgid":"2.f","version":"45'14732","reported_seq":14807,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.023168+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:58.023168+0000","last_peered":"2024-09-15T21:26:58.023168+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:58.023168+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26296320,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14732,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2288}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14433","reported_seq":14539,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:59.039624+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:59.039624+0000","last_peered":"2024-09-15T21:26:59.039624+0000","last_clean":"2024-09-15T21:26:59.039624+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:59.039624+0000","last_undegraded":"2024-09-15T21:26:59.039624+0000","last_fullsized":"2024-09-15T21:26:59.039624+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25564102,"num_objects":6243,"num_object_clones":0,"num_object_copies":12486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6243,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12714,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14883","reported_seq":15009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:59.022821+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:59.022821+0000","last_peered":"2024-09-15T21:26:59.022821+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:59.022821+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25485336,"num_objects":6223,"num_object_clones":0,"num_object_copies":12446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12983,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1956}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14260","reported_seq":14331,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:59.031196+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:59.031196+0000","last_peered":"2024-09-15T21:26:59.031196+0000","last_clean":"2024-09-15T21:26:59.031196+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:59.031196+0000","last_undegraded":"2024-09-15T21:26:59.031196+0000","last_fullsized":"2024-09-15T21:26:59.031196+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24797184,"num_objects":6054,"num_object_clones":0,"num_object_copies":12108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14260,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14357","reported_seq":14491,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.029399+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:58.029399+0000","last_peered":"2024-09-15T21:26:58.029399+0000","last_clean":"2024-09-15T21:26:58.029399+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:58.029399+0000","last_undegraded":"2024-09-15T21:26:58.029399+0000","last_fullsized":"2024-09-15T21:26:58.029399+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25182691,"num_objects":6149,"num_object_clones":0,"num_object_copies":12298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10812,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14429","reported_seq":14566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.032601+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:58.032601+0000","last_peered":"2024-09-15T21:26:58.032601+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:58.032601+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25420259,"num_objects":6207,"num_object_clones":0,"num_object_copies":12414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11105,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2131},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14519","reported_seq":14656,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:57.886205+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:57.886205+0000","last_peered":"2024-09-15T21:26:57.886205+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:57.886205+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25195462,"num_objects":6153,"num_object_clones":0,"num_object_copies":12306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10830,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1992},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14687","reported_seq":14817,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:57.951961+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:57.951961+0000","last_peered":"2024-09-15T21:26:57.951961+0000","last_clean":"2024-09-15T21:26:57.951961+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:57.951961+0000","last_undegraded":"2024-09-15T21:26:57.951961+0000","last_fullsized":"2024-09-15T21:26:57.951961+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25317376,"num_objects":6181,"num_object_clones":0,"num_object_copies":12362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11459,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14397","reported_seq":14524,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:59.057903+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:59.057903+0000","last_peered":"2024-09-15T21:26:59.057903+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:59.057903+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051619,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12329,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":2030}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14290","reported_seq":14422,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.015318+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:58.015318+0000","last_peered":"2024-09-15T21:26:58.015318+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:58.015318+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362432,"num_objects":6192,"num_object_clones":0,"num_object_copies":12384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10963,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2172}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14460","reported_seq":14580,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:59.003681+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:59.003681+0000","last_peered":"2024-09-15T21:26:59.003681+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:59.003681+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25326051,"num_objects":6184,"num_object_clones":0,"num_object_copies":12368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12319,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":2094}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14494","reported_seq":22954,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:59.008838+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:59.008838+0000","last_peered":"2024-09-15T21:26:59.008838+0000","last_clean":"2024-09-15T21:26:59.008838+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:59.008838+0000","last_undegraded":"2024-09-15T21:26:59.008838+0000","last_fullsized":"2024-09-15T21:26:59.008838+0000","mapping_epoch":44,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24846336,"num_objects":6066,"num_object_clones":0,"num_object_copies":12132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12187,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14467","reported_seq":14600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.042218+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:58.042218+0000","last_peered":"2024-09-15T21:26:58.042218+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:58.042218+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25473024,"num_objects":6219,"num_object_clones":0,"num_object_copies":12438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11411,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14459","reported_seq":14593,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:57.972187+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:57.972187+0000","last_peered":"2024-09-15T21:26:57.972187+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:57.972187+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25112576,"num_objects":6131,"num_object_clones":0,"num_object_copies":12262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11368,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1992}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14302","reported_seq":22112,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.034347+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:58.034347+0000","last_peered":"2024-09-15T21:26:58.034347+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:58.034347+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24928256,"num_objects":6086,"num_object_clones":0,"num_object_copies":12172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10786,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3836,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14620","reported_seq":14760,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:57.959452+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:57.959452+0000","last_peered":"2024-09-15T21:26:57.959452+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:57.959452+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25584582,"num_objects":6248,"num_object_clones":0,"num_object_copies":12496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6248,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11324,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2101},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":404943606,"num_objects":98873,"num_object_clones":0,"num_object_copies":197746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98873,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":191582,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8001,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1011429376,"data_stored":1011280325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":657796,"kb_used_data":329012,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93714044,"statfs":{"total":96636764160,"available":95963181056,"internally_reserved":0,"allocated":336908288,"data_stored":336632641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":336653993},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":756908,"kb_used_data":364316,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93614932,"statfs":{"total":96636764160,"available":95861690368,"internally_reserved":0,"allocated":373059584,"data_stored":372783932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,3,2,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631172,"kb_used_data":296820,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93740668,"statfs":{"total":96636764160,"available":95990444032,"internally_reserved":0,"allocated":303943680,"data_stored":303631439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,4,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":372457472,"data_stored":372421342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303108096,"data_stored":303035377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335863808,"data_stored":335823606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:03.399 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:03.958 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:03.960 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:03.988 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-15T21:27:02.801999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":404943625,"num_objects":98875,"num_object_clones":0,"num_object_copies":197750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98875,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":191584,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8002,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2045876,"kb_used_data":990148,"kb_used_omap":12,"kb_used_meta":1055667,"kb_avail":281069644,"statfs":{"total":289910292480,"available":287815315456,"internally_reserved":0,"allocated":1013911552,"data_stored":1013048012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1081003011},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,5,12,2,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19087360,"num_objects":-4660,"num_object_clones":0,"num_object_copies":-9320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-15,"num_objects_misplaced":-248,"num_objects_unfound":0,"num_objects_dirty":-4660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4660,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":263,"num_bytes_recovered":950272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005704"},"pg_stats":[{"pgid":"2.f","version":"45'14732","reported_seq":14807,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.023168+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:26:58.023168+0000","last_peered":"2024-09-15T21:26:58.023168+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:26:58.023168+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26296320,"num_objects":6420,"num_object_clones":0,"num_object_copies":12840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14732,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2288}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14433","reported_seq":14539,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:59.039624+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:26:59.039624+0000","last_peered":"2024-09-15T21:26:59.039624+0000","last_clean":"2024-09-15T21:26:59.039624+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:26:59.039624+0000","last_undegraded":"2024-09-15T21:26:59.039624+0000","last_fullsized":"2024-09-15T21:26:59.039624+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25564102,"num_objects":6243,"num_object_clones":0,"num_object_copies":12486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6243,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12714,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'14883","reported_seq":15009,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:59.022821+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:26:59.022821+0000","last_peered":"2024-09-15T21:26:59.022821+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:26:59.022821+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25485336,"num_objects":6223,"num_object_clones":0,"num_object_copies":12446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6223,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12983,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1956}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14260","reported_seq":14331,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:59.031196+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:26:59.031196+0000","last_peered":"2024-09-15T21:26:59.031196+0000","last_clean":"2024-09-15T21:26:59.031196+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:26:59.031196+0000","last_undegraded":"2024-09-15T21:26:59.031196+0000","last_fullsized":"2024-09-15T21:26:59.031196+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24797184,"num_objects":6054,"num_object_clones":0,"num_object_copies":12108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14260,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14357","reported_seq":14491,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:58.029399+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:26:58.029399+0000","last_peered":"2024-09-15T21:26:58.029399+0000","last_clean":"2024-09-15T21:26:58.029399+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:26:58.029399+0000","last_undegraded":"2024-09-15T21:26:58.029399+0000","last_fullsized":"2024-09-15T21:26:58.029399+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25182691,"num_objects":6149,"num_object_clones":0,"num_object_copies":12298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10812,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14429","reported_seq":14566,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.032601+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:26:58.032601+0000","last_peered":"2024-09-15T21:26:58.032601+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:26:58.032601+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25420259,"num_objects":6207,"num_object_clones":0,"num_object_copies":12414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6207,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11105,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2131},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14519","reported_seq":14656,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:57.886205+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:26:57.886205+0000","last_peered":"2024-09-15T21:26:57.886205+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:26:57.886205+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25195462,"num_objects":6153,"num_object_clones":0,"num_object_copies":12306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10830,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1992},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14687","reported_seq":14817,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:57.951961+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:26:57.951961+0000","last_peered":"2024-09-15T21:26:57.951961+0000","last_clean":"2024-09-15T21:26:57.951961+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:26:57.951961+0000","last_undegraded":"2024-09-15T21:26:57.951961+0000","last_fullsized":"2024-09-15T21:26:57.951961+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25317376,"num_objects":6181,"num_object_clones":0,"num_object_copies":12362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11459,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14397","reported_seq":14524,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:59.057903+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:26:59.057903+0000","last_peered":"2024-09-15T21:26:59.057903+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:26:59.057903+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051619,"num_objects":6117,"num_object_clones":0,"num_object_copies":12234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6117,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12329,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":2030}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14290","reported_seq":14422,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.015318+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:26:58.015318+0000","last_peered":"2024-09-15T21:26:58.015318+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:26:58.015318+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25362432,"num_objects":6192,"num_object_clones":0,"num_object_copies":12384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10963,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2172}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14460","reported_seq":14580,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:59.003681+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:26:59.003681+0000","last_peered":"2024-09-15T21:26:59.003681+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:26:59.003681+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25326051,"num_objects":6184,"num_object_clones":0,"num_object_copies":12368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6184,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12319,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":2094}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14494","reported_seq":22954,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:26:59.008838+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:26:59.008838+0000","last_peered":"2024-09-15T21:26:59.008838+0000","last_clean":"2024-09-15T21:26:59.008838+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:26:59.008838+0000","last_undegraded":"2024-09-15T21:26:59.008838+0000","last_fullsized":"2024-09-15T21:26:59.008838+0000","mapping_epoch":44,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24846336,"num_objects":6066,"num_object_clones":0,"num_object_copies":12132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12187,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14467","reported_seq":14600,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.042218+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:26:58.042218+0000","last_peered":"2024-09-15T21:26:58.042218+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:26:58.042218+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25473024,"num_objects":6219,"num_object_clones":0,"num_object_copies":12438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11411,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2125}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14459","reported_seq":14593,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:57.972187+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:26:57.972187+0000","last_peered":"2024-09-15T21:26:57.972187+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:26:57.972187+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25112576,"num_objects":6131,"num_object_clones":0,"num_object_copies":12262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11368,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1992}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14302","reported_seq":22112,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:58.034347+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:26:58.034347+0000","last_peered":"2024-09-15T21:26:58.034347+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:26:58.034347+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24928256,"num_objects":6086,"num_object_clones":0,"num_object_copies":12172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10786,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3836,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5838}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14620","reported_seq":14760,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:26:57.959452+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:26:57.959452+0000","last_peered":"2024-09-15T21:26:57.959452+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:26:57.959452+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25584582,"num_objects":6248,"num_object_clones":0,"num_object_copies":12496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6248,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11324,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2101},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":404943606,"num_objects":98873,"num_object_clones":0,"num_object_copies":197746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41473,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98873,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":191582,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8001,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1011429376,"data_stored":1011280325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160889,"ondisk_log_size":160889,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":657796,"kb_used_data":329012,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93714044,"statfs":{"total":96636764160,"available":95963181056,"internally_reserved":0,"allocated":336908288,"data_stored":336632641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":336653993},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":756908,"kb_used_data":364316,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93614932,"statfs":{"total":96636764160,"available":95861690368,"internally_reserved":0,"allocated":373059584,"data_stored":372783932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,3,2,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631172,"kb_used_data":296820,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93740668,"statfs":{"total":96636764160,"available":95990444032,"internally_reserved":0,"allocated":303943680,"data_stored":303631439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342355631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,4,6],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":372457472,"data_stored":372421342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303108096,"data_stored":303035377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":335863808,"data_stored":335823606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:03.989 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:03.989 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'14732", 'reported_seq': 14807, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:58.023168+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:26:58.023168+0000', 'last_peered': '2024-09-15T21:26:58.023168+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:26:58.023168+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26296320, 'num_objects': 6420, 'num_object_clones': 0, 'num_object_copies': 12840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14732, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2288}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:03.990 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:27:03.990 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'14883", 'reported_seq': 15009, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:59.022821+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:26:59.022821+0000', 'last_peered': '2024-09-15T21:26:59.022821+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:26:59.022821+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25485336, 'num_objects': 6223, 'num_object_clones': 0, 'num_object_copies': 12446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6223, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12983, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1956}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:03.990 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:03.990 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14429", 'reported_seq': 14566, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:58.032601+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:26:58.032601+0000', 'last_peered': '2024-09-15T21:26:58.032601+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:26:58.032601+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25420259, 'num_objects': 6207, 'num_object_clones': 0, 'num_object_copies': 12414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6207, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11105, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2131}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:03.990 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:03.990 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14519", 'reported_seq': 14656, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:57.886205+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:26:57.886205+0000', 'last_peered': '2024-09-15T21:26:57.886205+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:26:57.886205+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25195462, 'num_objects': 6153, 'num_object_clones': 0, 'num_object_copies': 12306, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6153, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10830, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1992}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:03.990 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:27:03.990 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'14397", 'reported_seq': 14524, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:59.057903+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:26:59.057903+0000', 'last_peered': '2024-09-15T21:26:59.057903+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:26:59.057903+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25051619, 'num_objects': 6117, 'num_object_clones': 0, 'num_object_copies': 12234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6117, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12329, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 2030}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:03.990 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:03.990 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14290", 'reported_seq': 14422, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:58.015318+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:26:58.015318+0000', 'last_peered': '2024-09-15T21:26:58.015318+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:26:58.015318+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25362432, 'num_objects': 6192, 'num_object_clones': 0, 'num_object_copies': 12384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6192, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10963, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2172}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:03.990 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:27:03.991 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14460", 'reported_seq': 14580, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:59.003681+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:26:59.003681+0000', 'last_peered': '2024-09-15T21:26:59.003681+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:26:59.003681+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25326051, 'num_objects': 6184, 'num_object_clones': 0, 'num_object_copies': 12368, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6184, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12319, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 2094}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:03.991 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:03.991 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14467", 'reported_seq': 14600, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:58.042218+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:26:58.042218+0000', 'last_peered': '2024-09-15T21:26:58.042218+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:26:58.042218+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25473024, 'num_objects': 6219, 'num_object_clones': 0, 'num_object_copies': 12438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6219, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11411, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2125}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:03.991 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:27:03.991 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14459", 'reported_seq': 14593, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:57.972187+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:26:57.972187+0000', 'last_peered': '2024-09-15T21:26:57.972187+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:26:57.972187+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25112576, 'num_objects': 6131, 'num_object_clones': 0, 'num_object_copies': 12262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11368, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1992}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:03.991 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:03.992 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14302", 'reported_seq': 22112, 'reported_epoch': 45, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:58.034347+0000', 'last_change': '2024-09-15T21:26:05.146366+0000', 'last_active': '2024-09-15T21:26:58.034347+0000', 'last_peered': '2024-09-15T21:26:58.034347+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:26:58.034347+0000', 'last_undegraded': '2024-09-15T21:26:05.134624+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24928256, 'num_objects': 6086, 'num_object_clones': 0, 'num_object_copies': 12172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6086, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10786, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3836, 'num_bytes_recovered': 15577088, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5838}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:03.992 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:27:03.992 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'14620", 'reported_seq': 14760, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:26:57.959452+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:26:57.959452+0000', 'last_peered': '2024-09-15T21:26:57.959452+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:26:57.959452+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25584582, 'num_objects': 6248, 'num_object_clones': 0, 'num_object_copies': 12496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6248, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11324, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2101}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:03.992 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:04.669 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:04.669 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41473/197750 objects degraded (20.972%), 11 pgs degraded, 11 pgs undersized","count":22},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":476,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":98875,"data_bytes":404943625,"bytes_used":2094977024,"bytes_avail":287815315456,"bytes_total":289910292480,"degraded_objects":41473,"degraded_total":197750,"degraded_ratio":0.20972439949431099,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":79151,"recovering_keys_per_sec":0,"num_objects_recovered":263,"num_bytes_recovered":950272,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":388},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:04.694 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:04.695 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:05.280 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:05.282 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:05.308 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":276,"stamp":"2024-09-15T21:27:04.802589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":395174665,"num_objects":96490,"num_object_clones":0,"num_object_copies":192980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41348,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96490,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":193969,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8127,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160777,"ondisk_log_size":160777,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2045012,"kb_used_data":971228,"kb_used_omap":12,"kb_used_meta":1073715,"kb_avail":281070508,"statfs":{"total":289910292480,"available":287816200192,"internally_reserved":0,"allocated":994537472,"data_stored":993673932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1099484163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,8,9,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28856320,"num_objects":-7045,"num_object_clones":0,"num_object_copies":-14090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-140,"num_objects_misplaced":-248,"num_objects_unfound":0,"num_objects_dirty":-7045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7045,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":950272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003043"},"pg_stats":[{"pgid":"2.f","version":"45'14886","reported_seq":14963,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:03.046885+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:03.046885+0000","last_peered":"2024-09-15T21:27:03.046885+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:03.046885+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25665536,"num_objects":6266,"num_object_clones":0,"num_object_copies":12532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14886,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2134}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14573","reported_seq":14678,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:04.050412+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:04.050412+0000","last_peered":"2024-09-15T21:27:04.050412+0000","last_clean":"2024-09-15T21:27:04.050412+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:04.050412+0000","last_undegraded":"2024-09-15T21:27:04.050412+0000","last_fullsized":"2024-09-15T21:27:04.050412+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24990662,"num_objects":6103,"num_object_clones":0,"num_object_copies":12206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6103,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12854,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15025","reported_seq":15152,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:04.037155+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:27:04.037155+0000","last_peered":"2024-09-15T21:27:04.037155+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:04.037155+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24903704,"num_objects":6081,"num_object_clones":0,"num_object_copies":12162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13125,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1814}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14401","reported_seq":14472,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:04.073115+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:04.073115+0000","last_peered":"2024-09-15T21:27:04.073115+0000","last_clean":"2024-09-15T21:27:04.073115+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:04.073115+0000","last_undegraded":"2024-09-15T21:27:04.073115+0000","last_fullsized":"2024-09-15T21:27:04.073115+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24219648,"num_objects":5913,"num_object_clones":0,"num_object_copies":11826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14401,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14519","reported_seq":14653,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:03.094997+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:03.094997+0000","last_peered":"2024-09-15T21:27:03.094997+0000","last_clean":"2024-09-15T21:27:03.094997+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:03.094997+0000","last_undegraded":"2024-09-15T21:27:03.094997+0000","last_fullsized":"2024-09-15T21:27:03.094997+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24519139,"num_objects":5987,"num_object_clones":0,"num_object_copies":11974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10974,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14575","reported_seq":14712,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:03.067681+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:03.067681+0000","last_peered":"2024-09-15T21:27:03.067681+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:03.067681+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24822243,"num_objects":6061,"num_object_clones":0,"num_object_copies":12122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6061,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11251,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1985},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14680","reported_seq":14817,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:03.078010+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:03.078010+0000","last_peered":"2024-09-15T21:27:03.078010+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:03.078010+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24536006,"num_objects":5992,"num_object_clones":0,"num_object_copies":11984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10991,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1831},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14851","reported_seq":14981,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:03.051461+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:03.051461+0000","last_peered":"2024-09-15T21:27:03.051461+0000","last_clean":"2024-09-15T21:27:03.051461+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:03.051461+0000","last_undegraded":"2024-09-15T21:27:03.051461+0000","last_fullsized":"2024-09-15T21:27:03.051461+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24645632,"num_objects":6017,"num_object_clones":0,"num_object_copies":12034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11623,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14558","reported_seq":14686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:04.058438+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:04.058438+0000","last_peered":"2024-09-15T21:27:04.058438+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:04.058438+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24392163,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12490,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1869}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14433","reported_seq":14565,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:02.980609+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:02.980609+0000","last_peered":"2024-09-15T21:27:02.980609+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:02.980609+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24776704,"num_objects":6049,"num_object_clones":0,"num_object_copies":12098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11106,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2029}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14595","reported_seq":14715,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:04.087710+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:04.087710+0000","last_peered":"2024-09-15T21:27:04.087710+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:04.087710+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24773091,"num_objects":6049,"num_object_clones":0,"num_object_copies":12098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12454,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1959}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14628","reported_seq":23088,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:03.815492+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:03.815492+0000","last_peered":"2024-09-15T21:27:03.815492+0000","last_clean":"2024-09-15T21:27:03.815492+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:03.815492+0000","last_undegraded":"2024-09-15T21:27:03.815492+0000","last_fullsized":"2024-09-15T21:27:03.815492+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24297472,"num_objects":5932,"num_object_clones":0,"num_object_copies":11864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12321,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14604","reported_seq":14739,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:03.045917+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:03.045917+0000","last_peered":"2024-09-15T21:27:03.045917+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:03.045917+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24911872,"num_objects":6082,"num_object_clones":0,"num_object_copies":12164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11548,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1988}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14622","reported_seq":14756,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:03.093060+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:03.093060+0000","last_peered":"2024-09-15T21:27:03.093060+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:03.093060+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24444928,"num_objects":5968,"num_object_clones":0,"num_object_copies":11936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11531,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1829}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14448","reported_seq":22508,"reported_epoch":45,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:03.061004+0000","last_change":"2024-09-15T21:26:05.146366+0000","last_active":"2024-09-15T21:27:03.061004+0000","last_peered":"2024-09-15T21:27:03.061004+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:27:03.061004+0000","last_undegraded":"2024-09-15T21:26:05.134624+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24330240,"num_objects":5940,"num_object_clones":0,"num_object_copies":11880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10932,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3961,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5817}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14776","reported_seq":14914,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:02.970203+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:02.970203+0000","last_peered":"2024-09-15T21:27:02.970203+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:02.970203+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24945606,"num_objects":6092,"num_object_clones":0,"num_object_copies":12184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6092,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11480,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1945},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":395174646,"num_objects":96488,"num_object_clones":0,"num_object_copies":192976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41348,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96488,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":193967,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8126,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":991903744,"data_stored":991754693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160774,"ondisk_log_size":160774,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640644,"kb_used_data":324336,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93731196,"statfs":{"total":96636764160,"available":95980744704,"internally_reserved":0,"allocated":332120064,"data_stored":331844417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182489,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":771876,"kb_used_data":357072,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93599964,"statfs":{"total":96636764160,"available":95846363136,"internally_reserved":0,"allocated":365641728,"data_stored":365366076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632492,"kb_used_data":289820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93739348,"statfs":{"total":96636764160,"available":95989092352,"internally_reserved":0,"allocated":296775680,"data_stored":296463439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":364998656,"data_stored":364962526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":295829504,"data_stored":295756785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331075584,"data_stored":331035382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:08.310 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:08.802 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:08.802 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:08.820 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-15T21:27:08.317878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":389100297,"num_objects":95007,"num_object_clones":0,"num_object_copies":190014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95007,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":195452,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8250,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2046280,"kb_used_data":964172,"kb_used_omap":12,"kb_used_meta":1082035,"kb_avail":281069240,"statfs":{"total":289910292480,"available":287814901760,"internally_reserved":0,"allocated":987312128,"data_stored":986448588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1108003843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,8,13,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25538560,"num_objects":-6235,"num_object_clones":0,"num_object_copies":-12470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-269,"num_objects_misplaced":-104,"num_objects_unfound":0,"num_objects_dirty":-6235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6235,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":385024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.517398"},"pg_stats":[{"pgid":"2.f","version":"45'15020","reported_seq":15097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.085516+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:08.085516+0000","last_peered":"2024-09-15T21:27:08.085516+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:08.085516+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25116672,"num_objects":6132,"num_object_clones":0,"num_object_copies":12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2000}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14573","reported_seq":14678,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:04.050412+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:04.050412+0000","last_peered":"2024-09-15T21:27:04.050412+0000","last_clean":"2024-09-15T21:27:04.050412+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:04.050412+0000","last_undegraded":"2024-09-15T21:27:04.050412+0000","last_fullsized":"2024-09-15T21:27:04.050412+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24990662,"num_objects":6103,"num_object_clones":0,"num_object_copies":12206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6103,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12854,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15025","reported_seq":15152,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:04.037155+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:27:04.037155+0000","last_peered":"2024-09-15T21:27:04.037155+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:04.037155+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24903704,"num_objects":6081,"num_object_clones":0,"num_object_copies":12162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13125,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1814}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14401","reported_seq":14472,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:04.073115+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:04.073115+0000","last_peered":"2024-09-15T21:27:04.073115+0000","last_clean":"2024-09-15T21:27:04.073115+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:04.073115+0000","last_undegraded":"2024-09-15T21:27:04.073115+0000","last_fullsized":"2024-09-15T21:27:04.073115+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24219648,"num_objects":5913,"num_object_clones":0,"num_object_copies":11826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14401,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14649","reported_seq":14783,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.038281+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:08.038281+0000","last_peered":"2024-09-15T21:27:08.038281+0000","last_clean":"2024-09-15T21:27:08.038281+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:08.038281+0000","last_undegraded":"2024-09-15T21:27:08.038281+0000","last_fullsized":"2024-09-15T21:27:08.038281+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986659,"num_objects":5857,"num_object_clones":0,"num_object_copies":11714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11104,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14732","reported_seq":14868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.120020+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:08.120020+0000","last_peered":"2024-09-15T21:27:08.120020+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:08.120020+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24179171,"num_objects":5904,"num_object_clones":0,"num_object_copies":11808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5904,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11408,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1828},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14824","reported_seq":14961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.099431+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:08.099431+0000","last_peered":"2024-09-15T21:27:08.099431+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:08.099431+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23946182,"num_objects":5848,"num_object_clones":0,"num_object_copies":11696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11135,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1687},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14994","reported_seq":15124,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.077109+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:08.077109+0000","last_peered":"2024-09-15T21:27:08.077109+0000","last_clean":"2024-09-15T21:27:08.077109+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:08.077109+0000","last_undegraded":"2024-09-15T21:27:08.077109+0000","last_fullsized":"2024-09-15T21:27:08.077109+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24059904,"num_objects":5874,"num_object_clones":0,"num_object_copies":11748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11766,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14558","reported_seq":14686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:04.058438+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:04.058438+0000","last_peered":"2024-09-15T21:27:04.058438+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:04.058438+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24392163,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12490,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1869}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14589","reported_seq":14721,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.013412+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:08.013412+0000","last_peered":"2024-09-15T21:27:08.013412+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:08.013412+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24137728,"num_objects":5893,"num_object_clones":0,"num_object_copies":11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11262,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14595","reported_seq":14715,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:04.087710+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:04.087710+0000","last_peered":"2024-09-15T21:27:04.087710+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:04.087710+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24773091,"num_objects":6049,"num_object_clones":0,"num_object_copies":12098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12454,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1959}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14628","reported_seq":23088,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:03.815492+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:03.815492+0000","last_peered":"2024-09-15T21:27:03.815492+0000","last_clean":"2024-09-15T21:27:03.815492+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:03.815492+0000","last_undegraded":"2024-09-15T21:27:03.815492+0000","last_fullsized":"2024-09-15T21:27:03.815492+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24297472,"num_objects":5932,"num_object_clones":0,"num_object_copies":11864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12321,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14786","reported_seq":14919,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.081419+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:08.081419+0000","last_peered":"2024-09-15T21:27:08.081419+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:08.081419+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24166400,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11730,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14775","reported_seq":14909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.036587+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:08.036587+0000","last_peered":"2024-09-15T21:27:08.036587+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:08.036587+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23818240,"num_objects":5815,"num_object_clones":0,"num_object_copies":11630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11684,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14595","reported_seq":22902,"reported_epoch":45,"state":"active+undersized+remapped","last_fresh":"2024-09-15T21:27:08.113113+0000","last_change":"2024-09-15T21:27:07.367241+0000","last_active":"2024-09-15T21:27:08.113113+0000","last_peered":"2024-09-15T21:27:08.113113+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:27:08.113113+0000","last_undegraded":"2024-09-15T21:27:08.113113+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23728128,"num_objects":5793,"num_object_clones":0,"num_object_copies":11586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11079,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0","objects":5793}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14913","reported_seq":15053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.096609+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:08.096609+0000","last_peered":"2024-09-15T21:27:08.096609+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:08.096609+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24384454,"num_objects":5955,"num_object_clones":0,"num_object_copies":11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5955,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11617,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1808},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":389100278,"num_objects":95005,"num_object_clones":0,"num_object_copies":190010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95005,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":195450,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8249,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":984571904,"data_stored":984422853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640644,"kb_used_data":324336,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93731196,"statfs":{"total":96636764160,"available":95980744704,"internally_reserved":0,"allocated":332120064,"data_stored":331844417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":773144,"kb_used_data":350016,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93598696,"statfs":{"total":96636764160,"available":95845064704,"internally_reserved":0,"allocated":358416384,"data_stored":358140732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":433254059},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632492,"kb_used_data":289820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93739348,"statfs":{"total":96636764160,"available":95989092352,"internally_reserved":0,"allocated":296775680,"data_stored":296463439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357666816,"data_stored":357630686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":295829504,"data_stored":295756785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331075584,"data_stored":331035382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:08.821 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:09.179 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:09.193 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":279,"stamp":"2024-09-15T21:27:08.805850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":389100297,"num_objects":95007,"num_object_clones":0,"num_object_copies":190014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95007,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":195452,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8250,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2046280,"kb_used_data":964172,"kb_used_omap":12,"kb_used_meta":1082035,"kb_avail":281069240,"statfs":{"total":289910292480,"available":287814901760,"internally_reserved":0,"allocated":987312128,"data_stored":986448588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1108003843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,8,13,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19505152,"num_objects":-4762,"num_object_clones":0,"num_object_copies":-9524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4762,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004936"},"pg_stats":[{"pgid":"2.f","version":"45'15020","reported_seq":15097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.085516+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:08.085516+0000","last_peered":"2024-09-15T21:27:08.085516+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:08.085516+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25116672,"num_objects":6132,"num_object_clones":0,"num_object_copies":12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2000}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'14573","reported_seq":14678,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:04.050412+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:04.050412+0000","last_peered":"2024-09-15T21:27:04.050412+0000","last_clean":"2024-09-15T21:27:04.050412+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:04.050412+0000","last_undegraded":"2024-09-15T21:27:04.050412+0000","last_fullsized":"2024-09-15T21:27:04.050412+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24990662,"num_objects":6103,"num_object_clones":0,"num_object_copies":12206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6103,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":12854,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'15025","reported_seq":15152,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:04.037155+0000","last_change":"2024-09-15T21:21:14.819853+0000","last_active":"2024-09-15T21:27:04.037155+0000","last_peered":"2024-09-15T21:27:04.037155+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:04.037155+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24903704,"num_objects":6081,"num_object_clones":0,"num_object_copies":12162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6081,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13125,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1814}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'14401","reported_seq":14472,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:04.073115+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:04.073115+0000","last_peered":"2024-09-15T21:27:04.073115+0000","last_clean":"2024-09-15T21:27:04.073115+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:04.073115+0000","last_undegraded":"2024-09-15T21:27:04.073115+0000","last_fullsized":"2024-09-15T21:27:04.073115+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24219648,"num_objects":5913,"num_object_clones":0,"num_object_copies":11826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14401,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14649","reported_seq":14783,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.038281+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:08.038281+0000","last_peered":"2024-09-15T21:27:08.038281+0000","last_clean":"2024-09-15T21:27:08.038281+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:08.038281+0000","last_undegraded":"2024-09-15T21:27:08.038281+0000","last_fullsized":"2024-09-15T21:27:08.038281+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986659,"num_objects":5857,"num_object_clones":0,"num_object_copies":11714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11104,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14732","reported_seq":14868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.120020+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:08.120020+0000","last_peered":"2024-09-15T21:27:08.120020+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:08.120020+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24179171,"num_objects":5904,"num_object_clones":0,"num_object_copies":11808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5904,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11408,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1828},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14824","reported_seq":14961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.099431+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:08.099431+0000","last_peered":"2024-09-15T21:27:08.099431+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:08.099431+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23946182,"num_objects":5848,"num_object_clones":0,"num_object_copies":11696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11135,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1687},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14994","reported_seq":15124,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.077109+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:08.077109+0000","last_peered":"2024-09-15T21:27:08.077109+0000","last_clean":"2024-09-15T21:27:08.077109+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:08.077109+0000","last_undegraded":"2024-09-15T21:27:08.077109+0000","last_fullsized":"2024-09-15T21:27:08.077109+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24059904,"num_objects":5874,"num_object_clones":0,"num_object_copies":11748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11766,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.876957+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:23:35.876957+0000","last_peered":"2024-09-15T21:23:35.876957+0000","last_clean":"2024-09-15T21:23:35.876957+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:23:35.876957+0000","last_undegraded":"2024-09-15T21:23:35.876957+0000","last_fullsized":"2024-09-15T21:23:35.876957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'14558","reported_seq":14686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:04.058438+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:04.058438+0000","last_peered":"2024-09-15T21:27:04.058438+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:04.058438+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24392163,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12490,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1869}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.605851+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:24:28.605851+0000","last_peered":"2024-09-15T21:24:28.605851+0000","last_clean":"2024-09-15T21:24:28.605851+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:24:28.605851+0000","last_undegraded":"2024-09-15T21:24:28.605851+0000","last_fullsized":"2024-09-15T21:24:28.605851+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14589","reported_seq":14721,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.013412+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:08.013412+0000","last_peered":"2024-09-15T21:27:08.013412+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:08.013412+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24137728,"num_objects":5893,"num_object_clones":0,"num_object_copies":11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11262,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'14595","reported_seq":14715,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:04.087710+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:04.087710+0000","last_peered":"2024-09-15T21:27:04.087710+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:04.087710+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24773091,"num_objects":6049,"num_object_clones":0,"num_object_copies":12098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12454,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1959}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877189+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:23:35.877189+0000","last_peered":"2024-09-15T21:23:35.877189+0000","last_clean":"2024-09-15T21:23:35.877189+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:23:35.877189+0000","last_undegraded":"2024-09-15T21:23:35.877189+0000","last_fullsized":"2024-09-15T21:23:35.877189+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'14628","reported_seq":23088,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:03.815492+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:03.815492+0000","last_peered":"2024-09-15T21:27:03.815492+0000","last_clean":"2024-09-15T21:27:03.815492+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:03.815492+0000","last_undegraded":"2024-09-15T21:27:03.815492+0000","last_fullsized":"2024-09-15T21:27:03.815492+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24297472,"num_objects":5932,"num_object_clones":0,"num_object_copies":11864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5932,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12321,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14786","reported_seq":14919,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.081419+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:08.081419+0000","last_peered":"2024-09-15T21:27:08.081419+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:08.081419+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24166400,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11730,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877382+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:23:35.877382+0000","last_peered":"2024-09-15T21:23:35.877382+0000","last_clean":"2024-09-15T21:23:35.877382+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:23:35.877382+0000","last_undegraded":"2024-09-15T21:23:35.877382+0000","last_fullsized":"2024-09-15T21:23:35.877382+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14775","reported_seq":14909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.036587+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:08.036587+0000","last_peered":"2024-09-15T21:27:08.036587+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:08.036587+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23818240,"num_objects":5815,"num_object_clones":0,"num_object_copies":11630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11684,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:28.625267+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:24:28.625267+0000","last_peered":"2024-09-15T21:24:28.625267+0000","last_clean":"2024-09-15T21:24:28.625267+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:24:28.625267+0000","last_undegraded":"2024-09-15T21:24:28.625267+0000","last_fullsized":"2024-09-15T21:24:28.625267+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14595","reported_seq":22902,"reported_epoch":45,"state":"active+undersized+remapped","last_fresh":"2024-09-15T21:27:08.113113+0000","last_change":"2024-09-15T21:27:07.367241+0000","last_active":"2024-09-15T21:27:08.113113+0000","last_peered":"2024-09-15T21:27:08.113113+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:27:08.113113+0000","last_undegraded":"2024-09-15T21:27:08.113113+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23728128,"num_objects":5793,"num_object_clones":0,"num_object_copies":11586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11079,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0","objects":5793}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877328+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:23:35.877328+0000","last_peered":"2024-09-15T21:23:35.877328+0000","last_clean":"2024-09-15T21:23:35.877328+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:23:35.877328+0000","last_undegraded":"2024-09-15T21:23:35.877328+0000","last_fullsized":"2024-09-15T21:23:35.877328+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:23:35.877340+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:23:35.877340+0000","last_peered":"2024-09-15T21:23:35.877340+0000","last_clean":"2024-09-15T21:23:35.877340+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:23:35.877340+0000","last_undegraded":"2024-09-15T21:23:35.877340+0000","last_fullsized":"2024-09-15T21:23:35.877340+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14913","reported_seq":15053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.096609+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:08.096609+0000","last_peered":"2024-09-15T21:27:08.096609+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:08.096609+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24384454,"num_objects":5955,"num_object_clones":0,"num_object_copies":11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5955,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11617,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1808},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":389100278,"num_objects":95005,"num_object_clones":0,"num_object_copies":190010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95005,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":195450,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8249,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":984571904,"data_stored":984422853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160857,"ondisk_log_size":160857,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640644,"kb_used_data":324336,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93731196,"statfs":{"total":96636764160,"available":95980744704,"internally_reserved":0,"allocated":332120064,"data_stored":331844417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":773144,"kb_used_data":350016,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93598696,"statfs":{"total":96636764160,"available":95845064704,"internally_reserved":0,"allocated":358416384,"data_stored":358140732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":433254059},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":632492,"kb_used_data":289820,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93739348,"statfs":{"total":96636764160,"available":95989092352,"internally_reserved":0,"allocated":296775680,"data_stored":296463439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350875311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357666816,"data_stored":357630686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":295829504,"data_stored":295756785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":331075584,"data_stored":331035382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:09.195 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:09.195 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'15020", 'reported_seq': 15097, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.085516+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:27:08.085516+0000', 'last_peered': '2024-09-15T21:27:08.085516+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:27:08.085516+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25116672, 'num_objects': 6132, 'num_object_clones': 0, 'num_object_copies': 12264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6132, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15020, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2000}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:09.195 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:27:09.195 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "45'15025", 'reported_seq': 15152, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:04.037155+0000', 'last_change': '2024-09-15T21:21:14.819853+0000', 'last_active': '2024-09-15T21:27:04.037155+0000', 'last_peered': '2024-09-15T21:27:04.037155+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:27:04.037155+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24903704, 'num_objects': 6081, 'num_object_clones': 0, 'num_object_copies': 12162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4268, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6081, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13125, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1814}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:09.195 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:09.195 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14732", 'reported_seq': 14868, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.120020+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:27:08.120020+0000', 'last_peered': '2024-09-15T21:27:08.120020+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:27:08.120020+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24179171, 'num_objects': 5904, 'num_object_clones': 0, 'num_object_copies': 11808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5904, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11408, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1828}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:09.195 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:09.195 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14824", 'reported_seq': 14961, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.099431+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:27:08.099431+0000', 'last_peered': '2024-09-15T21:27:08.099431+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:27:08.099431+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23946182, 'num_objects': 5848, 'num_object_clones': 0, 'num_object_copies': 11696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5848, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11135, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1687}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:09.195 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:27:09.195 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "45'14558", 'reported_seq': 14686, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:04.058438+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:27:04.058438+0000', 'last_peered': '2024-09-15T21:27:04.058438+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:27:04.058438+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24392163, 'num_objects': 5956, 'num_object_clones': 0, 'num_object_copies': 11912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5956, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12490, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1869}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:09.196 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:09.196 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14589", 'reported_seq': 14721, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.013412+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:27:08.013412+0000', 'last_peered': '2024-09-15T21:27:08.013412+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:27:08.013412+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24137728, 'num_objects': 5893, 'num_object_clones': 0, 'num_object_copies': 11786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11262, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1873}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:09.196 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:27:09.196 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "45'14595", 'reported_seq': 14715, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:04.087710+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:27:04.087710+0000', 'last_peered': '2024-09-15T21:27:04.087710+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:27:04.087710+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24773091, 'num_objects': 6049, 'num_object_clones': 0, 'num_object_copies': 12098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6049, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12454, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1959}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:09.196 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:09.196 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14786", 'reported_seq': 14919, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.081419+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:27:08.081419+0000', 'last_peered': '2024-09-15T21:27:08.081419+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:27:08.081419+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24166400, 'num_objects': 5900, 'num_object_clones': 0, 'num_object_copies': 11800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11730, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1806}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:09.196 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:27:09.196 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14775", 'reported_seq': 14909, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.036587+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:27:08.036587+0000', 'last_peered': '2024-09-15T21:27:08.036587+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:27:08.036587+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23818240, 'num_objects': 5815, 'num_object_clones': 0, 'num_object_copies': 11630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5815, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11684, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1676}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:09.196 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:09.196 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14595", 'reported_seq': 22902, 'reported_epoch': 45, 'state': 'active+undersized+remapped', 'last_fresh': '2024-09-15T21:27:08.113113+0000', 'last_change': '2024-09-15T21:27:07.367241+0000', 'last_active': '2024-09-15T21:27:08.113113+0000', 'last_peered': '2024-09-15T21:27:08.113113+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:27:08.113113+0000', 'last_undegraded': '2024-09-15T21:27:08.113113+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23728128, 'num_objects': 5793, 'num_object_clones': 0, 'num_object_copies': 11586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5793, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11079, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4084, 'num_bytes_recovered': 15577088, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '1', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5793}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:09.197 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:27:09.197 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'14913", 'reported_seq': 15053, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.096609+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:27:08.096609+0000', 'last_peered': '2024-09-15T21:27:08.096609+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:27:08.096609+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24384454, 'num_objects': 5955, 'num_object_clones': 0, 'num_object_copies': 11910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5955, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11617, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1808}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:09.197 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:09.739 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:09.739 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41225/190014 objects degraded (21.696%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":481,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":95007,"data_bytes":389100297,"bytes_used":2095390720,"bytes_avail":287814901760,"bytes_total":289910292480,"degraded_objects":41225,"degraded_total":190014,"degraded_ratio":0.21695769785384234,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":248,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":475},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:09.758 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:09.759 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:10.310 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:10.311 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:10.347 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-15T21:27:09.316812+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":385594121,"num_objects":94151,"num_object_clones":0,"num_object_copies":188302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94151,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":196308,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8313,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160816,"ondisk_log_size":160816,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2042916,"kb_used_data":952484,"kb_used_omap":12,"kb_used_meta":1090355,"kb_avail":281072604,"statfs":{"total":289910292480,"available":287818346496,"internally_reserved":0,"allocated":975343616,"data_stored":974474099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1116523523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,7,18,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19349504,"num_objects":-4724,"num_object_clones":0,"num_object_copies":-9448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-311,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4724,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.515299"},"pg_stats":[{"pgid":"2.f","version":"45'15020","reported_seq":15097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.085516+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:08.085516+0000","last_peered":"2024-09-15T21:27:08.085516+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:08.085516+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25116672,"num_objects":6132,"num_object_clones":0,"num_object_copies":12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2000}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'14726","reported_seq":14834,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.799620+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:08.799620+0000","last_peered":"2024-09-15T21:27:08.799620+0000","last_clean":"2024-09-15T21:27:08.799620+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:08.799620+0000","last_undegraded":"2024-09-15T21:27:08.799620+0000","last_fullsized":"2024-09-15T21:27:08.799620+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24363974,"num_objects":5950,"num_object_clones":0,"num_object_copies":11900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5950,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13007,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'15166","reported_seq":15422,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:09.119101+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:09.119101+0000","last_peered":"2024-09-15T21:27:09.119101+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:09.119101+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24326168,"num_objects":5940,"num_object_clones":0,"num_object_copies":11880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4205,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5940,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13266,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1735}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'14533","reported_seq":14607,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.881346+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:08.881346+0000","last_peered":"2024-09-15T21:27:08.881346+0000","last_clean":"2024-09-15T21:27:08.881346+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:08.881346+0000","last_undegraded":"2024-09-15T21:27:08.881346+0000","last_fullsized":"2024-09-15T21:27:08.881346+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23678976,"num_objects":5781,"num_object_clones":0,"num_object_copies":11562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14533,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14649","reported_seq":14783,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.038281+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:08.038281+0000","last_peered":"2024-09-15T21:27:08.038281+0000","last_clean":"2024-09-15T21:27:08.038281+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:08.038281+0000","last_undegraded":"2024-09-15T21:27:08.038281+0000","last_fullsized":"2024-09-15T21:27:08.038281+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986659,"num_objects":5857,"num_object_clones":0,"num_object_copies":11714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11104,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14732","reported_seq":14868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.120020+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:08.120020+0000","last_peered":"2024-09-15T21:27:08.120020+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:08.120020+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24179171,"num_objects":5904,"num_object_clones":0,"num_object_copies":11808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5904,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11408,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1828},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14824","reported_seq":14961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.099431+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:08.099431+0000","last_peered":"2024-09-15T21:27:08.099431+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:08.099431+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23946182,"num_objects":5848,"num_object_clones":0,"num_object_copies":11696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11135,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1687},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14994","reported_seq":15124,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.077109+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:08.077109+0000","last_peered":"2024-09-15T21:27:08.077109+0000","last_clean":"2024-09-15T21:27:08.077109+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:08.077109+0000","last_undegraded":"2024-09-15T21:27:08.077109+0000","last_fullsized":"2024-09-15T21:27:08.077109+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24059904,"num_objects":5874,"num_object_clones":0,"num_object_copies":11748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11766,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.319792+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:08.319792+0000","last_peered":"2024-09-15T21:27:08.319792+0000","last_clean":"2024-09-15T21:27:08.319792+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:08.319792+0000","last_undegraded":"2024-09-15T21:27:08.319792+0000","last_fullsized":"2024-09-15T21:27:08.319792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'14694","reported_seq":14824,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.912711+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:08.912711+0000","last_peered":"2024-09-15T21:27:08.912711+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:08.912711+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23835107,"num_objects":5820,"num_object_clones":0,"num_object_copies":11640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5820,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12626,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1733}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.330448+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:08.330448+0000","last_peered":"2024-09-15T21:27:08.330448+0000","last_clean":"2024-09-15T21:27:08.330448+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:08.330448+0000","last_undegraded":"2024-09-15T21:27:08.330448+0000","last_fullsized":"2024-09-15T21:27:08.330448+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14589","reported_seq":14721,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.013412+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:08.013412+0000","last_peered":"2024-09-15T21:27:08.013412+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:08.013412+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24137728,"num_objects":5893,"num_object_clones":0,"num_object_copies":11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11262,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'14744","reported_seq":14866,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.875897+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:08.875897+0000","last_peered":"2024-09-15T21:27:08.875897+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:08.875897+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24162787,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12603,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1810}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":82,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.328955+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:08.328955+0000","last_peered":"2024-09-15T21:27:08.328955+0000","last_clean":"2024-09-15T21:27:08.328955+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:08.328955+0000","last_undegraded":"2024-09-15T21:27:08.328955+0000","last_fullsized":"2024-09-15T21:27:08.328955+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'14765","reported_seq":23227,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.837707+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:08.837707+0000","last_peered":"2024-09-15T21:27:08.837707+0000","last_clean":"2024-09-15T21:27:08.837707+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:08.837707+0000","last_undegraded":"2024-09-15T21:27:08.837707+0000","last_fullsized":"2024-09-15T21:27:08.837707+0000","mapping_epoch":44,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23736320,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12458,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14786","reported_seq":14919,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.081419+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:08.081419+0000","last_peered":"2024-09-15T21:27:08.081419+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:08.081419+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24166400,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11730,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.327424+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:08.327424+0000","last_peered":"2024-09-15T21:27:08.327424+0000","last_clean":"2024-09-15T21:27:08.327424+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:08.327424+0000","last_undegraded":"2024-09-15T21:27:08.327424+0000","last_fullsized":"2024-09-15T21:27:08.327424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14775","reported_seq":14909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.036587+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:08.036587+0000","last_peered":"2024-09-15T21:27:08.036587+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:08.036587+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23818240,"num_objects":5815,"num_object_clones":0,"num_object_copies":11630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11684,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.332729+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:08.332729+0000","last_peered":"2024-09-15T21:27:08.332729+0000","last_clean":"2024-09-15T21:27:08.332729+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:08.332729+0000","last_undegraded":"2024-09-15T21:27:08.332729+0000","last_fullsized":"2024-09-15T21:27:08.332729+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14603","reported_seq":22909,"reported_epoch":46,"state":"peering","last_fresh":"2024-09-15T21:27:08.211442+0000","last_change":"2024-09-15T21:27:09.168381+0000","last_active":"2024-09-15T21:27:09.168381+0000","last_peered":"2024-09-15T21:27:08.211442+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:27:08.211442+0000","last_undegraded":"2024-09-15T21:27:08.211442+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":46,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23695360,"num_objects":5785,"num_object_clones":0,"num_object_copies":11570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11087,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.327139+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:08.327139+0000","last_peered":"2024-09-15T21:27:08.327139+0000","last_clean":"2024-09-15T21:27:08.327139+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:08.327139+0000","last_undegraded":"2024-09-15T21:27:08.327139+0000","last_fullsized":"2024-09-15T21:27:08.327139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.336040+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:08.336040+0000","last_peered":"2024-09-15T21:27:08.336040+0000","last_clean":"2024-09-15T21:27:08.336040+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:08.336040+0000","last_undegraded":"2024-09-15T21:27:08.336040+0000","last_fullsized":"2024-09-15T21:27:08.336040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14913","reported_seq":15053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.096609+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:08.096609+0000","last_peered":"2024-09-15T21:27:08.096609+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:08.096609+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24384454,"num_objects":5955,"num_object_clones":0,"num_object_copies":11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5955,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11617,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1808},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":385594102,"num_objects":94149,"num_object_clones":0,"num_object_copies":188298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94149,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":196306,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8312,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":973230080,"data_stored":973084642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160813,"ondisk_log_size":160813,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640180,"kb_used_data":319712,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93731660,"statfs":{"total":96636764160,"available":95981219840,"internally_reserved":0,"allocated":327385088,"data_stored":327104646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":328134313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,1,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":773144,"kb_used_data":350016,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93598696,"statfs":{"total":96636764160,"available":95845064704,"internally_reserved":0,"allocated":358416384,"data_stored":358140732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":433254059},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629592,"kb_used_data":282756,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93742248,"statfs":{"total":96636764160,"available":95992061952,"internally_reserved":0,"allocated":289542144,"data_stored":289228721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,8,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357666816,"data_stored":357630686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289013760,"data_stored":288944654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326549504,"data_stored":326509302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:13.349 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:13.918 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:13.918 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:13.939 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-15T21:27:12.806709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":385594121,"num_objects":94151,"num_object_clones":0,"num_object_copies":188302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94151,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":196308,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8313,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160816,"ondisk_log_size":160816,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2042916,"kb_used_data":952484,"kb_used_omap":12,"kb_used_meta":1090355,"kb_avail":281072604,"statfs":{"total":289910292480,"available":287818346496,"internally_reserved":0,"allocated":975343616,"data_stored":974474099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1116523523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,7,18,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9580544,"num_objects":-2339,"num_object_clones":0,"num_object_copies":-4678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2339,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004119"},"pg_stats":[{"pgid":"2.f","version":"45'15020","reported_seq":15097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.085516+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:08.085516+0000","last_peered":"2024-09-15T21:27:08.085516+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:08.085516+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25116672,"num_objects":6132,"num_object_clones":0,"num_object_copies":12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2000}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'14726","reported_seq":14834,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.799620+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:08.799620+0000","last_peered":"2024-09-15T21:27:08.799620+0000","last_clean":"2024-09-15T21:27:08.799620+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:08.799620+0000","last_undegraded":"2024-09-15T21:27:08.799620+0000","last_fullsized":"2024-09-15T21:27:08.799620+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24363974,"num_objects":5950,"num_object_clones":0,"num_object_copies":11900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5950,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13007,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'15166","reported_seq":15422,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:09.119101+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:09.119101+0000","last_peered":"2024-09-15T21:27:09.119101+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:09.119101+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24326168,"num_objects":5940,"num_object_clones":0,"num_object_copies":11880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4205,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5940,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13266,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1735}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'14533","reported_seq":14607,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.881346+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:08.881346+0000","last_peered":"2024-09-15T21:27:08.881346+0000","last_clean":"2024-09-15T21:27:08.881346+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:08.881346+0000","last_undegraded":"2024-09-15T21:27:08.881346+0000","last_fullsized":"2024-09-15T21:27:08.881346+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23678976,"num_objects":5781,"num_object_clones":0,"num_object_copies":11562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14533,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14649","reported_seq":14783,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.038281+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:08.038281+0000","last_peered":"2024-09-15T21:27:08.038281+0000","last_clean":"2024-09-15T21:27:08.038281+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:08.038281+0000","last_undegraded":"2024-09-15T21:27:08.038281+0000","last_fullsized":"2024-09-15T21:27:08.038281+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986659,"num_objects":5857,"num_object_clones":0,"num_object_copies":11714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11104,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14732","reported_seq":14868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.120020+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:08.120020+0000","last_peered":"2024-09-15T21:27:08.120020+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:08.120020+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24179171,"num_objects":5904,"num_object_clones":0,"num_object_copies":11808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5904,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11408,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1828},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14824","reported_seq":14961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.099431+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:08.099431+0000","last_peered":"2024-09-15T21:27:08.099431+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:08.099431+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23946182,"num_objects":5848,"num_object_clones":0,"num_object_copies":11696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11135,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1687},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14994","reported_seq":15124,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.077109+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:08.077109+0000","last_peered":"2024-09-15T21:27:08.077109+0000","last_clean":"2024-09-15T21:27:08.077109+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:08.077109+0000","last_undegraded":"2024-09-15T21:27:08.077109+0000","last_fullsized":"2024-09-15T21:27:08.077109+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24059904,"num_objects":5874,"num_object_clones":0,"num_object_copies":11748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11766,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.319792+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:08.319792+0000","last_peered":"2024-09-15T21:27:08.319792+0000","last_clean":"2024-09-15T21:27:08.319792+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:08.319792+0000","last_undegraded":"2024-09-15T21:27:08.319792+0000","last_fullsized":"2024-09-15T21:27:08.319792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'14694","reported_seq":14824,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.912711+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:08.912711+0000","last_peered":"2024-09-15T21:27:08.912711+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:08.912711+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23835107,"num_objects":5820,"num_object_clones":0,"num_object_copies":11640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5820,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12626,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1733}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.330448+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:08.330448+0000","last_peered":"2024-09-15T21:27:08.330448+0000","last_clean":"2024-09-15T21:27:08.330448+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:08.330448+0000","last_undegraded":"2024-09-15T21:27:08.330448+0000","last_fullsized":"2024-09-15T21:27:08.330448+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14589","reported_seq":14721,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.013412+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:08.013412+0000","last_peered":"2024-09-15T21:27:08.013412+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:08.013412+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24137728,"num_objects":5893,"num_object_clones":0,"num_object_copies":11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11262,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'14744","reported_seq":14866,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.875897+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:08.875897+0000","last_peered":"2024-09-15T21:27:08.875897+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:08.875897+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24162787,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12603,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1810}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":82,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.328955+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:08.328955+0000","last_peered":"2024-09-15T21:27:08.328955+0000","last_clean":"2024-09-15T21:27:08.328955+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:08.328955+0000","last_undegraded":"2024-09-15T21:27:08.328955+0000","last_fullsized":"2024-09-15T21:27:08.328955+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'14765","reported_seq":23227,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.837707+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:08.837707+0000","last_peered":"2024-09-15T21:27:08.837707+0000","last_clean":"2024-09-15T21:27:08.837707+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:08.837707+0000","last_undegraded":"2024-09-15T21:27:08.837707+0000","last_fullsized":"2024-09-15T21:27:08.837707+0000","mapping_epoch":44,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23736320,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12458,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14786","reported_seq":14919,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.081419+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:08.081419+0000","last_peered":"2024-09-15T21:27:08.081419+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:08.081419+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24166400,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11730,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.327424+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:08.327424+0000","last_peered":"2024-09-15T21:27:08.327424+0000","last_clean":"2024-09-15T21:27:08.327424+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:08.327424+0000","last_undegraded":"2024-09-15T21:27:08.327424+0000","last_fullsized":"2024-09-15T21:27:08.327424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14775","reported_seq":14909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.036587+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:08.036587+0000","last_peered":"2024-09-15T21:27:08.036587+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:08.036587+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23818240,"num_objects":5815,"num_object_clones":0,"num_object_copies":11630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11684,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.332729+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:08.332729+0000","last_peered":"2024-09-15T21:27:08.332729+0000","last_clean":"2024-09-15T21:27:08.332729+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:08.332729+0000","last_undegraded":"2024-09-15T21:27:08.332729+0000","last_fullsized":"2024-09-15T21:27:08.332729+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14603","reported_seq":22909,"reported_epoch":46,"state":"peering","last_fresh":"2024-09-15T21:27:08.211442+0000","last_change":"2024-09-15T21:27:09.168381+0000","last_active":"2024-09-15T21:27:09.168381+0000","last_peered":"2024-09-15T21:27:08.211442+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:27:08.211442+0000","last_undegraded":"2024-09-15T21:27:08.211442+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":46,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23695360,"num_objects":5785,"num_object_clones":0,"num_object_copies":11570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11087,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.327139+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:08.327139+0000","last_peered":"2024-09-15T21:27:08.327139+0000","last_clean":"2024-09-15T21:27:08.327139+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:08.327139+0000","last_undegraded":"2024-09-15T21:27:08.327139+0000","last_fullsized":"2024-09-15T21:27:08.327139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.336040+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:08.336040+0000","last_peered":"2024-09-15T21:27:08.336040+0000","last_clean":"2024-09-15T21:27:08.336040+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:08.336040+0000","last_undegraded":"2024-09-15T21:27:08.336040+0000","last_fullsized":"2024-09-15T21:27:08.336040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14913","reported_seq":15053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.096609+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:08.096609+0000","last_peered":"2024-09-15T21:27:08.096609+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:08.096609+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24384454,"num_objects":5955,"num_object_clones":0,"num_object_copies":11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5955,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11617,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1808},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":385594102,"num_objects":94149,"num_object_clones":0,"num_object_copies":188298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94149,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":196306,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8312,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":973230080,"data_stored":973084642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160813,"ondisk_log_size":160813,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640180,"kb_used_data":319712,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93731660,"statfs":{"total":96636764160,"available":95981219840,"internally_reserved":0,"allocated":327385088,"data_stored":327104646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":328134313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,1,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":773144,"kb_used_data":350016,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93598696,"statfs":{"total":96636764160,"available":95845064704,"internally_reserved":0,"allocated":358416384,"data_stored":358140732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":433254059},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629592,"kb_used_data":282756,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93742248,"statfs":{"total":96636764160,"available":95992061952,"internally_reserved":0,"allocated":289542144,"data_stored":289228721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,8,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357666816,"data_stored":357630686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289013760,"data_stored":288944654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326549504,"data_stored":326509302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:13.941 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:14.518 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:14.518 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:14.548 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-15T21:27:12.806709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":385594121,"num_objects":94151,"num_object_clones":0,"num_object_copies":188302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94151,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":196308,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8313,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160816,"ondisk_log_size":160816,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2042916,"kb_used_data":952484,"kb_used_omap":12,"kb_used_meta":1090355,"kb_avail":281072604,"statfs":{"total":289910292480,"available":287818346496,"internally_reserved":0,"allocated":975343616,"data_stored":974474099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1116523523},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,7,18,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9580544,"num_objects":-2339,"num_object_clones":0,"num_object_copies":-4678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2339,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004119"},"pg_stats":[{"pgid":"2.f","version":"45'15020","reported_seq":15097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.085516+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:08.085516+0000","last_peered":"2024-09-15T21:27:08.085516+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:08.085516+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25116672,"num_objects":6132,"num_object_clones":0,"num_object_copies":12264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2000}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"46'14726","reported_seq":14834,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.799620+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:08.799620+0000","last_peered":"2024-09-15T21:27:08.799620+0000","last_clean":"2024-09-15T21:27:08.799620+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:08.799620+0000","last_undegraded":"2024-09-15T21:27:08.799620+0000","last_fullsized":"2024-09-15T21:27:08.799620+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24363974,"num_objects":5950,"num_object_clones":0,"num_object_copies":11900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5950,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13007,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"46'15166","reported_seq":15422,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:09.119101+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:09.119101+0000","last_peered":"2024-09-15T21:27:09.119101+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:09.119101+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24326168,"num_objects":5940,"num_object_clones":0,"num_object_copies":11880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4205,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5940,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13266,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1735}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"46'14533","reported_seq":14607,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.881346+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:08.881346+0000","last_peered":"2024-09-15T21:27:08.881346+0000","last_clean":"2024-09-15T21:27:08.881346+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:08.881346+0000","last_undegraded":"2024-09-15T21:27:08.881346+0000","last_fullsized":"2024-09-15T21:27:08.881346+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23678976,"num_objects":5781,"num_object_clones":0,"num_object_copies":11562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14533,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'14649","reported_seq":14783,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.038281+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:08.038281+0000","last_peered":"2024-09-15T21:27:08.038281+0000","last_clean":"2024-09-15T21:27:08.038281+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:08.038281+0000","last_undegraded":"2024-09-15T21:27:08.038281+0000","last_fullsized":"2024-09-15T21:27:08.038281+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23986659,"num_objects":5857,"num_object_clones":0,"num_object_copies":11714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11104,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'14732","reported_seq":14868,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.120020+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:08.120020+0000","last_peered":"2024-09-15T21:27:08.120020+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:08.120020+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24179171,"num_objects":5904,"num_object_clones":0,"num_object_copies":11808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5904,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11408,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1828},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'14824","reported_seq":14961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.099431+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:08.099431+0000","last_peered":"2024-09-15T21:27:08.099431+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:08.099431+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23946182,"num_objects":5848,"num_object_clones":0,"num_object_copies":11696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11135,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1687},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'14994","reported_seq":15124,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.077109+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:08.077109+0000","last_peered":"2024-09-15T21:27:08.077109+0000","last_clean":"2024-09-15T21:27:08.077109+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:08.077109+0000","last_undegraded":"2024-09-15T21:27:08.077109+0000","last_fullsized":"2024-09-15T21:27:08.077109+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24059904,"num_objects":5874,"num_object_clones":0,"num_object_copies":11748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11766,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.319792+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:08.319792+0000","last_peered":"2024-09-15T21:27:08.319792+0000","last_clean":"2024-09-15T21:27:08.319792+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:08.319792+0000","last_undegraded":"2024-09-15T21:27:08.319792+0000","last_fullsized":"2024-09-15T21:27:08.319792+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"46'14694","reported_seq":14824,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.912711+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:08.912711+0000","last_peered":"2024-09-15T21:27:08.912711+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:08.912711+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23835107,"num_objects":5820,"num_object_clones":0,"num_object_copies":11640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5820,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12626,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1733}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.330448+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:08.330448+0000","last_peered":"2024-09-15T21:27:08.330448+0000","last_clean":"2024-09-15T21:27:08.330448+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:08.330448+0000","last_undegraded":"2024-09-15T21:27:08.330448+0000","last_fullsized":"2024-09-15T21:27:08.330448+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'14589","reported_seq":14721,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.013412+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:08.013412+0000","last_peered":"2024-09-15T21:27:08.013412+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:08.013412+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24137728,"num_objects":5893,"num_object_clones":0,"num_object_copies":11786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11262,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1873}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":102,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-15T21:24:11.181969+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:24:11.181969+0000","last_peered":"2024-09-15T21:24:11.181969+0000","last_clean":"2024-09-15T21:24:11.181969+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:24:11.181969+0000","last_undegraded":"2024-09-15T21:24:11.181969+0000","last_fullsized":"2024-09-15T21:24:11.181969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"46'14744","reported_seq":14866,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.875897+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:08.875897+0000","last_peered":"2024-09-15T21:27:08.875897+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:08.875897+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24162787,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12603,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1810}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":82,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.328955+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:08.328955+0000","last_peered":"2024-09-15T21:27:08.328955+0000","last_clean":"2024-09-15T21:27:08.328955+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:08.328955+0000","last_undegraded":"2024-09-15T21:27:08.328955+0000","last_fullsized":"2024-09-15T21:27:08.328955+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"46'14765","reported_seq":23227,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.837707+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:08.837707+0000","last_peered":"2024-09-15T21:27:08.837707+0000","last_clean":"2024-09-15T21:27:08.837707+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:08.837707+0000","last_undegraded":"2024-09-15T21:27:08.837707+0000","last_fullsized":"2024-09-15T21:27:08.837707+0000","mapping_epoch":44,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23736320,"num_objects":5795,"num_object_clones":0,"num_object_copies":11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12458,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'14786","reported_seq":14919,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.081419+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:08.081419+0000","last_peered":"2024-09-15T21:27:08.081419+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:08.081419+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24166400,"num_objects":5900,"num_object_clones":0,"num_object_copies":11800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11730,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1806}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.327424+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:08.327424+0000","last_peered":"2024-09-15T21:27:08.327424+0000","last_clean":"2024-09-15T21:27:08.327424+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:08.327424+0000","last_undegraded":"2024-09-15T21:27:08.327424+0000","last_fullsized":"2024-09-15T21:27:08.327424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'14775","reported_seq":14909,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.036587+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:08.036587+0000","last_peered":"2024-09-15T21:27:08.036587+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:08.036587+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23818240,"num_objects":5815,"num_object_clones":0,"num_object_copies":11630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11684,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":73,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.332729+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:08.332729+0000","last_peered":"2024-09-15T21:27:08.332729+0000","last_clean":"2024-09-15T21:27:08.332729+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:08.332729+0000","last_undegraded":"2024-09-15T21:27:08.332729+0000","last_fullsized":"2024-09-15T21:27:08.332729+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'14603","reported_seq":22909,"reported_epoch":46,"state":"peering","last_fresh":"2024-09-15T21:27:08.211442+0000","last_change":"2024-09-15T21:27:09.168381+0000","last_active":"2024-09-15T21:27:09.168381+0000","last_peered":"2024-09-15T21:27:08.211442+0000","last_clean":"2024-09-15T21:21:12.325188+0000","last_became_active":"2024-09-15T21:21:14.804192+0000","last_became_peered":"2024-09-15T21:21:14.804192+0000","last_unstale":"2024-09-15T21:27:08.211442+0000","last_undegraded":"2024-09-15T21:27:08.211442+0000","last_fullsized":"2024-09-15T21:21:14.356072+0000","mapping_epoch":46,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23695360,"num_objects":5785,"num_object_clones":0,"num_object_copies":11570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11087,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.327139+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:08.327139+0000","last_peered":"2024-09-15T21:27:08.327139+0000","last_clean":"2024-09-15T21:27:08.327139+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:08.327139+0000","last_undegraded":"2024-09-15T21:27:08.327139+0000","last_fullsized":"2024-09-15T21:27:08.327139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-15T21:27:08.336040+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:08.336040+0000","last_peered":"2024-09-15T21:27:08.336040+0000","last_clean":"2024-09-15T21:27:08.336040+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:08.336040+0000","last_undegraded":"2024-09-15T21:27:08.336040+0000","last_fullsized":"2024-09-15T21:27:08.336040+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'14913","reported_seq":15053,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:08.096609+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:08.096609+0000","last_peered":"2024-09-15T21:27:08.096609+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:08.096609+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24384454,"num_objects":5955,"num_object_clones":0,"num_object_copies":11910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5955,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11617,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1808},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":385594102,"num_objects":94149,"num_object_clones":0,"num_object_copies":188298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94149,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":196306,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8312,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":973230080,"data_stored":973084642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160813,"ondisk_log_size":160813,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640180,"kb_used_data":319712,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93731660,"statfs":{"total":96636764160,"available":95981219840,"internally_reserved":0,"allocated":327385088,"data_stored":327104646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":328134313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,1,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":773144,"kb_used_data":350016,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93598696,"statfs":{"total":96636764160,"available":95845064704,"internally_reserved":0,"allocated":358416384,"data_stored":358140732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":433254059},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629592,"kb_used_data":282756,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93742248,"statfs":{"total":96636764160,"available":95992061952,"internally_reserved":0,"allocated":289542144,"data_stored":289228721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355135151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,8,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":357666816,"data_stored":357630686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289013760,"data_stored":288944654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":326549504,"data_stored":326509302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:14.549 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:14.549 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "45'15020", 'reported_seq': 15097, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.085516+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:27:08.085516+0000', 'last_peered': '2024-09-15T21:27:08.085516+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:27:08.085516+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25116672, 'num_objects': 6132, 'num_object_clones': 0, 'num_object_copies': 12264, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6132, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15020, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2000}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:14.549 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:27:14.549 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "46'15166", 'reported_seq': 15422, 'reported_epoch': 46, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:09.119101+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:27:09.119101+0000', 'last_peered': '2024-09-15T21:27:09.119101+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:27:09.119101+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24326168, 'num_objects': 5940, 'num_object_clones': 0, 'num_object_copies': 11880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4205, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5940, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13266, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 63, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1735}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:14.549 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:14.549 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "45'14732", 'reported_seq': 14868, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.120020+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:27:08.120020+0000', 'last_peered': '2024-09-15T21:27:08.120020+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:27:08.120020+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24179171, 'num_objects': 5904, 'num_object_clones': 0, 'num_object_copies': 11808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5904, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11408, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1828}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:14.549 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:14.550 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "45'14824", 'reported_seq': 14961, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.099431+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:27:08.099431+0000', 'last_peered': '2024-09-15T21:27:08.099431+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:27:08.099431+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23946182, 'num_objects': 5848, 'num_object_clones': 0, 'num_object_copies': 11696, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5848, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11135, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1687}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:14.550 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:27:14.550 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "46'14694", 'reported_seq': 14824, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.912711+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:27:08.912711+0000', 'last_peered': '2024-09-15T21:27:08.912711+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:27:08.912711+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23835107, 'num_objects': 5820, 'num_object_clones': 0, 'num_object_copies': 11640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5820, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12626, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1733}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:14.550 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:14.550 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "45'14589", 'reported_seq': 14721, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.013412+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:27:08.013412+0000', 'last_peered': '2024-09-15T21:27:08.013412+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:27:08.013412+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24137728, 'num_objects': 5893, 'num_object_clones': 0, 'num_object_copies': 11786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11262, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1873}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:14.550 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:27:14.550 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "46'14744", 'reported_seq': 14866, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.875897+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:27:08.875897+0000', 'last_peered': '2024-09-15T21:27:08.875897+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:27:08.875897+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24162787, 'num_objects': 5900, 'num_object_clones': 0, 'num_object_copies': 11800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5900, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12603, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1810}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:14.550 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:14.550 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "45'14786", 'reported_seq': 14919, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.081419+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:27:08.081419+0000', 'last_peered': '2024-09-15T21:27:08.081419+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:27:08.081419+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24166400, 'num_objects': 5900, 'num_object_clones': 0, 'num_object_copies': 11800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11730, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1806}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:14.550 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:27:14.551 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "45'14775", 'reported_seq': 14909, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.036587+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:27:08.036587+0000', 'last_peered': '2024-09-15T21:27:08.036587+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:27:08.036587+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23818240, 'num_objects': 5815, 'num_object_clones': 0, 'num_object_copies': 11630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5815, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11684, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1676}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:14.551 INFO:tasks.lost_unfound.ceph_manager:PG 2.5 is not active+clean 2024-09-15T21:27:14.551 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.5', 'version': "45'14603", 'reported_seq': 22909, 'reported_epoch': 46, 'state': 'peering', 'last_fresh': '2024-09-15T21:27:08.211442+0000', 'last_change': '2024-09-15T21:27:09.168381+0000', 'last_active': '2024-09-15T21:27:09.168381+0000', 'last_peered': '2024-09-15T21:27:08.211442+0000', 'last_clean': '2024-09-15T21:21:12.325188+0000', 'last_became_active': '2024-09-15T21:21:14.804192+0000', 'last_became_peered': '2024-09-15T21:21:14.804192+0000', 'last_unstale': '2024-09-15T21:27:08.211442+0000', 'last_undegraded': '2024-09-15T21:27:08.211442+0000', 'last_fullsized': '2024-09-15T21:21:14.356072+0000', 'mapping_epoch': 46, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3547", 'last_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:02.966850+0000', 'objects_scrubbed': 3512, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1213, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23695360, 'num_objects': 5785, 'num_object_clones': 0, 'num_object_copies': 11570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5785, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11087, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4084, 'num_bytes_recovered': 15577088, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-15T21:27:14.551 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:27:14.551 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "45'14913", 'reported_seq': 15053, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:08.096609+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:27:08.096609+0000', 'last_peered': '2024-09-15T21:27:08.096609+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:27:08.096609+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24384454, 'num_objects': 5955, 'num_object_clones': 0, 'num_object_copies': 11910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5955, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11617, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1808}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:14.551 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:15.243 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:15.244 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41162/188302 objects degraded (21.860%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":486,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":94151,"data_bytes":385594121,"bytes_used":2091945984,"bytes_avail":287818346496,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":41162,"degraded_total":188302,"degraded_ratio":0.21859566016292972,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":186,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":292},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:15.270 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:15.270 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:15.829 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:15.830 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:15.853 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-09-15T21:27:14.807167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":376632073,"num_objects":91963,"num_object_clones":0,"num_object_copies":183926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91963,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":198496,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8416,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160704,"ondisk_log_size":160704,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2009496,"kb_used_data":935704,"kb_used_omap":12,"kb_used_meta":1073715,"kb_avail":281106024,"statfs":{"total":289910292480,"available":287852568576,"internally_reserved":0,"allocated":958160896,"data_stored":957272136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1099484163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,4,5,9,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18542592,"num_objects":-4527,"num_object_clones":0,"num_object_copies":-9054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4527,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001743"},"pg_stats":[{"pgid":"2.f","version":"47'15156","reported_seq":15237,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:13.213502+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:13.213502+0000","last_peered":"2024-09-15T21:27:13.213502+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:13.213502+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24559616,"num_objects":5996,"num_object_clones":0,"num_object_copies":11992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15156,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1864}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14860","reported_seq":14970,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.060852+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:14.060852+0000","last_peered":"2024-09-15T21:27:14.060852+0000","last_clean":"2024-09-15T21:27:14.060852+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:14.060852+0000","last_undegraded":"2024-09-15T21:27:14.060852+0000","last_fullsized":"2024-09-15T21:27:14.060852+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23815110,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13141,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15309","reported_seq":15773,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:14.124103+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:14.124103+0000","last_peered":"2024-09-15T21:27:14.124103+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:14.124103+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23740440,"num_objects":5797,"num_object_clones":0,"num_object_copies":11594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5797,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13409,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14662","reported_seq":14738,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.110679+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:14.110679+0000","last_peered":"2024-09-15T21:27:14.110679+0000","last_clean":"2024-09-15T21:27:14.110679+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:14.110679+0000","last_undegraded":"2024-09-15T21:27:14.110679+0000","last_fullsized":"2024-09-15T21:27:14.110679+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23150592,"num_objects":5652,"num_object_clones":0,"num_object_copies":11304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14662,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14804","reported_seq":14942,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:13.195123+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:13.195123+0000","last_peered":"2024-09-15T21:27:13.195123+0000","last_clean":"2024-09-15T21:27:13.195123+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:13.195123+0000","last_undegraded":"2024-09-15T21:27:13.195123+0000","last_fullsized":"2024-09-15T21:27:13.195123+0000","mapping_epoch":34,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23351779,"num_objects":5702,"num_object_clones":0,"num_object_copies":11404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11259,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14842","reported_seq":14982,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:13.139587+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:13.139587+0000","last_peered":"2024-09-15T21:27:13.139587+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:13.139587+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23728611,"num_objects":5794,"num_object_clones":0,"num_object_copies":11588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5794,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11518,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1718},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14949","reported_seq":15090,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:13.155265+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:13.155265+0000","last_peered":"2024-09-15T21:27:13.155265+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:13.155265+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23434182,"num_objects":5723,"num_object_clones":0,"num_object_copies":11446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11260,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1562},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15140","reported_seq":15274,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:13.166112+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:13.166112+0000","last_peered":"2024-09-15T21:27:13.166112+0000","last_clean":"2024-09-15T21:27:13.166112+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:13.166112+0000","last_undegraded":"2024-09-15T21:27:13.166112+0000","last_fullsized":"2024-09-15T21:27:13.166112+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23461888,"num_objects":5728,"num_object_clones":0,"num_object_copies":11456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11912,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14833","reported_seq":14965,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:14.092894+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:14.092894+0000","last_peered":"2024-09-15T21:27:14.092894+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:14.092894+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23265763,"num_objects":5681,"num_object_clones":0,"num_object_copies":11362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12765,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1594}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14737","reported_seq":14872,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:13.147337+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:13.147337+0000","last_peered":"2024-09-15T21:27:13.147337+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:13.147337+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23531520,"num_objects":5745,"num_object_clones":0,"num_object_copies":11490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11410,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1725}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14885","reported_seq":15009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:14.057596+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:14.057596+0000","last_peered":"2024-09-15T21:27:14.057596+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:14.057596+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23585251,"num_objects":5759,"num_object_clones":0,"num_object_copies":11518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5759,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12744,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1669}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317433+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317433+0000","last_peered":"2024-09-15T21:27:09.317433+0000","last_clean":"2024-09-15T21:27:09.317433+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317433+0000","last_undegraded":"2024-09-15T21:27:09.317433+0000","last_fullsized":"2024-09-15T21:27:09.317433+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14890","reported_seq":23354,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.109666+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:14.109666+0000","last_peered":"2024-09-15T21:27:14.109666+0000","last_clean":"2024-09-15T21:27:14.109666+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:14.109666+0000","last_undegraded":"2024-09-15T21:27:14.109666+0000","last_fullsized":"2024-09-15T21:27:14.109666+0000","mapping_epoch":44,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23224320,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12583,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14908","reported_seq":15047,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:13.023179+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:13.023179+0000","last_peered":"2024-09-15T21:27:13.023179+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:13.023179+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23666688,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11852,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1684}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317318+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317318+0000","last_peered":"2024-09-15T21:27:09.317318+0000","last_clean":"2024-09-15T21:27:09.317318+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317318+0000","last_undegraded":"2024-09-15T21:27:09.317318+0000","last_fullsized":"2024-09-15T21:27:09.317318+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14920","reported_seq":15058,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:13.167467+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:13.167467+0000","last_peered":"2024-09-15T21:27:13.167467+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:13.167467+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23224320,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11829,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1531}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14753","reported_seq":23066,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.180255+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:14.180255+0000","last_peered":"2024-09-15T21:27:14.180255+0000","last_clean":"2024-09-15T21:27:14.180255+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:14.180255+0000","last_undegraded":"2024-09-15T21:27:14.180255+0000","last_fullsized":"2024-09-15T21:27:14.180255+0000","mapping_epoch":46,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23080960,"num_objects":5635,"num_object_clones":0,"num_object_copies":11270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11237,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317284+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317284+0000","last_peered":"2024-09-15T21:27:09.317284+0000","last_clean":"2024-09-15T21:27:09.317284+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317284+0000","last_undegraded":"2024-09-15T21:27:09.317284+0000","last_fullsized":"2024-09-15T21:27:09.317284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15053","reported_seq":15197,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:13.080856+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:13.080856+0000","last_peered":"2024-09-15T21:27:13.080856+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:13.080856+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23811014,"num_objects":5815,"num_object_clones":0,"num_object_copies":11630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5815,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11757,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1668},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":376632054,"num_objects":91961,"num_object_clones":0,"num_object_copies":183922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91961,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":198494,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8415,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":955191296,"data_stored":955045858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160701,"ondisk_log_size":160701,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640092,"kb_used_data":315460,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93731748,"statfs":{"total":96636764160,"available":95981309952,"internally_reserved":0,"allocated":323031040,"data_stored":322745782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":332394153},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":754508,"kb_used_data":343860,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93617332,"statfs":{"total":96636764160,"available":95864147968,"internally_reserved":0,"allocated":352112640,"data_stored":351827377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659209,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614896,"kb_used_data":276384,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93756944,"statfs":{"total":96636764160,"available":96007110656,"internally_reserved":0,"allocated":283017216,"data_stored":282698977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":351145984,"data_stored":351109854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":282083328,"data_stored":282014222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321961984,"data_stored":321921782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:18.855 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:19.410 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:19.411 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:19.432 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-15T21:27:18.808256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":371315465,"num_objects":90665,"num_object_clones":0,"num_object_copies":181330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90665,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":199794,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8416,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160902,"ondisk_log_size":160902,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2006348,"kb_used_data":928404,"kb_used_omap":12,"kb_used_meta":1077875,"kb_avail":281109172,"statfs":{"total":289910292480,"available":287855792128,"internally_reserved":0,"allocated":950685696,"data_stored":949796936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1103744003},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,2,13,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17784832,"num_objects":-4342,"num_object_clones":0,"num_object_copies":-8684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4342,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002406"},"pg_stats":[{"pgid":"2.f","version":"47'15311","reported_seq":15392,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.168530+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:18.168530+0000","last_peered":"2024-09-15T21:27:18.168530+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:18.168530+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23924736,"num_objects":5841,"num_object_clones":0,"num_object_copies":11682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15311,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1709}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14860","reported_seq":14970,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.060852+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:14.060852+0000","last_peered":"2024-09-15T21:27:14.060852+0000","last_clean":"2024-09-15T21:27:14.060852+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:14.060852+0000","last_undegraded":"2024-09-15T21:27:14.060852+0000","last_fullsized":"2024-09-15T21:27:14.060852+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23815110,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13141,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15309","reported_seq":15773,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:14.124103+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:14.124103+0000","last_peered":"2024-09-15T21:27:14.124103+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:14.124103+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23740440,"num_objects":5797,"num_object_clones":0,"num_object_copies":11594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5797,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13409,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14662","reported_seq":14738,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.110679+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:14.110679+0000","last_peered":"2024-09-15T21:27:14.110679+0000","last_clean":"2024-09-15T21:27:14.110679+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:14.110679+0000","last_undegraded":"2024-09-15T21:27:14.110679+0000","last_fullsized":"2024-09-15T21:27:14.110679+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23150592,"num_objects":5652,"num_object_clones":0,"num_object_copies":11304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14662,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14937","reported_seq":15075,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:18.191212+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:18.191212+0000","last_peered":"2024-09-15T21:27:18.191212+0000","last_clean":"2024-09-15T21:27:18.191212+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:18.191212+0000","last_undegraded":"2024-09-15T21:27:18.191212+0000","last_fullsized":"2024-09-15T21:27:18.191212+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22807011,"num_objects":5569,"num_object_clones":0,"num_object_copies":11138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11392,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14969","reported_seq":15110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.225878+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:18.225878+0000","last_peered":"2024-09-15T21:27:18.225878+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:18.225878+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23208419,"num_objects":5667,"num_object_clones":0,"num_object_copies":11334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5667,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11645,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1591},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15100","reported_seq":15239,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.215106+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:18.215106+0000","last_peered":"2024-09-15T21:27:18.215106+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:18.215106+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22815686,"num_objects":5572,"num_object_clones":0,"num_object_copies":11144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11411,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1411},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15273","reported_seq":15407,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:18.238386+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:18.238386+0000","last_peered":"2024-09-15T21:27:18.238386+0000","last_clean":"2024-09-15T21:27:18.238386+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:18.238386+0000","last_undegraded":"2024-09-15T21:27:18.238386+0000","last_fullsized":"2024-09-15T21:27:18.238386+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22917120,"num_objects":5595,"num_object_clones":0,"num_object_copies":11190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12045,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14833","reported_seq":14965,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:14.092894+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:14.092894+0000","last_peered":"2024-09-15T21:27:14.092894+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:14.092894+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23265763,"num_objects":5681,"num_object_clones":0,"num_object_copies":11362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12765,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1594}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14878","reported_seq":15014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.196478+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:18.196478+0000","last_peered":"2024-09-15T21:27:18.196478+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:18.196478+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22953984,"num_objects":5604,"num_object_clones":0,"num_object_copies":11208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11551,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1584}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14885","reported_seq":15009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:14.057596+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:14.057596+0000","last_peered":"2024-09-15T21:27:14.057596+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:14.057596+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23585251,"num_objects":5759,"num_object_clones":0,"num_object_copies":11518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5759,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12744,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1669}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317433+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317433+0000","last_peered":"2024-09-15T21:27:09.317433+0000","last_clean":"2024-09-15T21:27:09.317433+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317433+0000","last_undegraded":"2024-09-15T21:27:09.317433+0000","last_fullsized":"2024-09-15T21:27:09.317433+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14890","reported_seq":23354,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.109666+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:14.109666+0000","last_peered":"2024-09-15T21:27:14.109666+0000","last_clean":"2024-09-15T21:27:14.109666+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:14.109666+0000","last_undegraded":"2024-09-15T21:27:14.109666+0000","last_fullsized":"2024-09-15T21:27:14.109666+0000","mapping_epoch":44,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23224320,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12583,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15069","reported_seq":15208,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.137645+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:18.137645+0000","last_peered":"2024-09-15T21:27:18.137645+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:18.137645+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23007232,"num_objects":5617,"num_object_clones":0,"num_object_copies":11234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12013,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317318+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317318+0000","last_peered":"2024-09-15T21:27:09.317318+0000","last_clean":"2024-09-15T21:27:09.317318+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317318+0000","last_undegraded":"2024-09-15T21:27:09.317318+0000","last_fullsized":"2024-09-15T21:27:09.317318+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15069","reported_seq":15207,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.163846+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:18.163846+0000","last_peered":"2024-09-15T21:27:18.163846+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:18.163846+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22614016,"num_objects":5521,"num_object_clones":0,"num_object_copies":11042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11978,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14753","reported_seq":23066,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.180255+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:14.180255+0000","last_peered":"2024-09-15T21:27:14.180255+0000","last_clean":"2024-09-15T21:27:14.180255+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:14.180255+0000","last_undegraded":"2024-09-15T21:27:14.180255+0000","last_fullsized":"2024-09-15T21:27:14.180255+0000","mapping_epoch":46,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23080960,"num_objects":5635,"num_object_clones":0,"num_object_copies":11270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11237,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317284+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317284+0000","last_peered":"2024-09-15T21:27:09.317284+0000","last_clean":"2024-09-15T21:27:09.317284+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317284+0000","last_undegraded":"2024-09-15T21:27:09.317284+0000","last_fullsized":"2024-09-15T21:27:09.317284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15201","reported_seq":15345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.223854+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:18.223854+0000","last_peered":"2024-09-15T21:27:18.223854+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:18.223854+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23204806,"num_objects":5667,"num_object_clones":0,"num_object_copies":11334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5667,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11905,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1520},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":371315446,"num_objects":90663,"num_object_clones":0,"num_object_copies":181326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90663,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":199792,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8415,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":947965952,"data_stored":947820514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160899,"ondisk_log_size":160899,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640092,"kb_used_data":315460,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93731748,"statfs":{"total":96636764160,"available":95981309952,"internally_reserved":0,"allocated":323031040,"data_stored":322745782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":332394153},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751360,"kb_used_data":336560,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93620480,"statfs":{"total":96636764160,"available":95867371520,"internally_reserved":0,"allocated":344637440,"data_stored":344352177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659209,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614896,"kb_used_data":276384,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93756944,"statfs":{"total":96636764160,"available":96007110656,"internally_reserved":0,"allocated":283017216,"data_stored":282698977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343920640,"data_stored":343884510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":282083328,"data_stored":282014222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321961984,"data_stored":321921782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:19.434 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:19.978 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:19.978 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:20.008 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-15T21:27:18.808256+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":371315465,"num_objects":90665,"num_object_clones":0,"num_object_copies":181330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90665,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":199794,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8416,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160902,"ondisk_log_size":160902,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2006348,"kb_used_data":928404,"kb_used_omap":12,"kb_used_meta":1077875,"kb_avail":281109172,"statfs":{"total":289910292480,"available":287855792128,"internally_reserved":0,"allocated":950685696,"data_stored":949796936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1103744003},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,2,13,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17784832,"num_objects":-4342,"num_object_clones":0,"num_object_copies":-8684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-166,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4342,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002406"},"pg_stats":[{"pgid":"2.f","version":"47'15311","reported_seq":15392,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.168530+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:18.168530+0000","last_peered":"2024-09-15T21:27:18.168530+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:18.168530+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23924736,"num_objects":5841,"num_object_clones":0,"num_object_copies":11682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15311,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1709}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14860","reported_seq":14970,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.060852+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:14.060852+0000","last_peered":"2024-09-15T21:27:14.060852+0000","last_clean":"2024-09-15T21:27:14.060852+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:14.060852+0000","last_undegraded":"2024-09-15T21:27:14.060852+0000","last_fullsized":"2024-09-15T21:27:14.060852+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23815110,"num_objects":5816,"num_object_clones":0,"num_object_copies":11632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5816,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13141,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15309","reported_seq":15773,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:14.124103+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:14.124103+0000","last_peered":"2024-09-15T21:27:14.124103+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:14.124103+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23740440,"num_objects":5797,"num_object_clones":0,"num_object_copies":11594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5797,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13409,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1695}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14662","reported_seq":14738,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.110679+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:14.110679+0000","last_peered":"2024-09-15T21:27:14.110679+0000","last_clean":"2024-09-15T21:27:14.110679+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:14.110679+0000","last_undegraded":"2024-09-15T21:27:14.110679+0000","last_fullsized":"2024-09-15T21:27:14.110679+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23150592,"num_objects":5652,"num_object_clones":0,"num_object_copies":11304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14662,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14937","reported_seq":15075,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:18.191212+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:18.191212+0000","last_peered":"2024-09-15T21:27:18.191212+0000","last_clean":"2024-09-15T21:27:18.191212+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:18.191212+0000","last_undegraded":"2024-09-15T21:27:18.191212+0000","last_fullsized":"2024-09-15T21:27:18.191212+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22807011,"num_objects":5569,"num_object_clones":0,"num_object_copies":11138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11392,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14969","reported_seq":15110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.225878+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:18.225878+0000","last_peered":"2024-09-15T21:27:18.225878+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:18.225878+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23208419,"num_objects":5667,"num_object_clones":0,"num_object_copies":11334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5667,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11645,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1591},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15100","reported_seq":15239,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.215106+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:18.215106+0000","last_peered":"2024-09-15T21:27:18.215106+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:18.215106+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22815686,"num_objects":5572,"num_object_clones":0,"num_object_copies":11144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11411,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1411},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15273","reported_seq":15407,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:18.238386+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:18.238386+0000","last_peered":"2024-09-15T21:27:18.238386+0000","last_clean":"2024-09-15T21:27:18.238386+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:18.238386+0000","last_undegraded":"2024-09-15T21:27:18.238386+0000","last_fullsized":"2024-09-15T21:27:18.238386+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22917120,"num_objects":5595,"num_object_clones":0,"num_object_copies":11190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12045,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14833","reported_seq":14965,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:14.092894+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:14.092894+0000","last_peered":"2024-09-15T21:27:14.092894+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:14.092894+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23265763,"num_objects":5681,"num_object_clones":0,"num_object_copies":11362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5681,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12765,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1594}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14878","reported_seq":15014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.196478+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:18.196478+0000","last_peered":"2024-09-15T21:27:18.196478+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:18.196478+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22953984,"num_objects":5604,"num_object_clones":0,"num_object_copies":11208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11551,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1584}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14885","reported_seq":15009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:14.057596+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:14.057596+0000","last_peered":"2024-09-15T21:27:14.057596+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:14.057596+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23585251,"num_objects":5759,"num_object_clones":0,"num_object_copies":11518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5759,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12744,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1669}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317433+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317433+0000","last_peered":"2024-09-15T21:27:09.317433+0000","last_clean":"2024-09-15T21:27:09.317433+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317433+0000","last_undegraded":"2024-09-15T21:27:09.317433+0000","last_fullsized":"2024-09-15T21:27:09.317433+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14890","reported_seq":23354,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.109666+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:14.109666+0000","last_peered":"2024-09-15T21:27:14.109666+0000","last_clean":"2024-09-15T21:27:14.109666+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:14.109666+0000","last_undegraded":"2024-09-15T21:27:14.109666+0000","last_fullsized":"2024-09-15T21:27:14.109666+0000","mapping_epoch":44,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23224320,"num_objects":5670,"num_object_clones":0,"num_object_copies":11340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12583,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15069","reported_seq":15208,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.137645+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:18.137645+0000","last_peered":"2024-09-15T21:27:18.137645+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:18.137645+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23007232,"num_objects":5617,"num_object_clones":0,"num_object_copies":11234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12013,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317318+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317318+0000","last_peered":"2024-09-15T21:27:09.317318+0000","last_clean":"2024-09-15T21:27:09.317318+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317318+0000","last_undegraded":"2024-09-15T21:27:09.317318+0000","last_fullsized":"2024-09-15T21:27:09.317318+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15069","reported_seq":15207,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.163846+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:18.163846+0000","last_peered":"2024-09-15T21:27:18.163846+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:18.163846+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22614016,"num_objects":5521,"num_object_clones":0,"num_object_copies":11042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11978,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14753","reported_seq":23066,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.180255+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:14.180255+0000","last_peered":"2024-09-15T21:27:14.180255+0000","last_clean":"2024-09-15T21:27:14.180255+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:14.180255+0000","last_undegraded":"2024-09-15T21:27:14.180255+0000","last_fullsized":"2024-09-15T21:27:14.180255+0000","mapping_epoch":46,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23080960,"num_objects":5635,"num_object_clones":0,"num_object_copies":11270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5635,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11237,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317284+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317284+0000","last_peered":"2024-09-15T21:27:09.317284+0000","last_clean":"2024-09-15T21:27:09.317284+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317284+0000","last_undegraded":"2024-09-15T21:27:09.317284+0000","last_fullsized":"2024-09-15T21:27:09.317284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15201","reported_seq":15345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.223854+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:18.223854+0000","last_peered":"2024-09-15T21:27:18.223854+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:18.223854+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23204806,"num_objects":5667,"num_object_clones":0,"num_object_copies":11334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5667,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11905,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1520},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":371315446,"num_objects":90663,"num_object_clones":0,"num_object_copies":181326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41059,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90663,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":199792,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8415,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":947965952,"data_stored":947820514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160899,"ondisk_log_size":160899,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640092,"kb_used_data":315460,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93731748,"statfs":{"total":96636764160,"available":95981309952,"internally_reserved":0,"allocated":323031040,"data_stored":322745782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":332394153},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751360,"kb_used_data":336560,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93620480,"statfs":{"total":96636764160,"available":95867371520,"internally_reserved":0,"allocated":344637440,"data_stored":344352177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659209,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":614896,"kb_used_data":276384,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93756944,"statfs":{"total":96636764160,"available":96007110656,"internally_reserved":0,"allocated":283017216,"data_stored":282698977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346615471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343920640,"data_stored":343884510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":282083328,"data_stored":282014222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":321961984,"data_stored":321921782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:20.010 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:20.010 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'15311", 'reported_seq': 15392, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:18.168530+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:27:18.168530+0000', 'last_peered': '2024-09-15T21:27:18.168530+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:27:18.168530+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23924736, 'num_objects': 5841, 'num_object_clones': 0, 'num_object_copies': 11682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5841, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15311, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1709}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:20.010 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:27:20.010 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'15309", 'reported_seq': 15773, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:14.124103+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:27:14.124103+0000', 'last_peered': '2024-09-15T21:27:14.124103+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:27:14.124103+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23740440, 'num_objects': 5797, 'num_object_clones': 0, 'num_object_copies': 11594, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4102, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5797, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13409, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 166, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1695}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:20.010 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:20.010 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'14969", 'reported_seq': 15110, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:18.225878+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:27:18.225878+0000', 'last_peered': '2024-09-15T21:27:18.225878+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:27:18.225878+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23208419, 'num_objects': 5667, 'num_object_clones': 0, 'num_object_copies': 11334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5667, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11645, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1591}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:20.010 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:20.010 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'15100", 'reported_seq': 15239, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:18.215106+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:27:18.215106+0000', 'last_peered': '2024-09-15T21:27:18.215106+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:27:18.215106+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22815686, 'num_objects': 5572, 'num_object_clones': 0, 'num_object_copies': 11144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5572, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11411, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1411}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:20.011 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:27:20.011 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'14833", 'reported_seq': 14965, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:14.092894+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:27:14.092894+0000', 'last_peered': '2024-09-15T21:27:14.092894+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:27:14.092894+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23265763, 'num_objects': 5681, 'num_object_clones': 0, 'num_object_copies': 11362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5681, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12765, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1594}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:20.011 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:20.011 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'14878", 'reported_seq': 15014, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:18.196478+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:27:18.196478+0000', 'last_peered': '2024-09-15T21:27:18.196478+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:27:18.196478+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22953984, 'num_objects': 5604, 'num_object_clones': 0, 'num_object_copies': 11208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5604, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11551, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1584}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:20.011 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:27:20.011 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'14885", 'reported_seq': 15009, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:14.057596+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:27:14.057596+0000', 'last_peered': '2024-09-15T21:27:14.057596+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:27:14.057596+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23585251, 'num_objects': 5759, 'num_object_clones': 0, 'num_object_copies': 11518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5759, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12744, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1669}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:20.011 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:20.011 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'15069", 'reported_seq': 15208, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:18.137645+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:27:18.137645+0000', 'last_peered': '2024-09-15T21:27:18.137645+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:27:18.137645+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23007232, 'num_objects': 5617, 'num_object_clones': 0, 'num_object_copies': 11234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5617, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12013, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1523}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:20.011 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:27:20.011 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'15069", 'reported_seq': 15207, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:18.163846+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:27:18.163846+0000', 'last_peered': '2024-09-15T21:27:18.163846+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:27:18.163846+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22614016, 'num_objects': 5521, 'num_object_clones': 0, 'num_object_copies': 11042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5521, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11978, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1382}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:20.012 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:27:20.012 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'15201", 'reported_seq': 15345, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:18.223854+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:27:18.223854+0000', 'last_peered': '2024-09-15T21:27:18.223854+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:27:18.223854+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23204806, 'num_objects': 5667, 'num_object_clones': 0, 'num_object_copies': 11334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5667, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11905, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1520}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:20.014 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:20.706 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:20.707 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 41059/181330 objects degraded (22.643%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":492,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":90665,"data_bytes":371315465,"bytes_used":2054500352,"bytes_avail":287855792128,"bytes_total":289910292480,"degraded_objects":41059,"degraded_total":181330,"degraded_ratio":0.2264324711851321,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":166,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":434},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:20.735 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:20.735 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:21.312 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:21.312 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:21.343 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-09-15T21:27:20.808882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":367055625,"num_objects":89625,"num_object_clones":0,"num_object_copies":179250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89625,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":200834,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8514,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160842,"ondisk_log_size":160842,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2020916,"kb_used_data":916608,"kb_used_omap":12,"kb_used_meta":1104243,"kb_avail":281094604,"statfs":{"total":289910292480,"available":287840874496,"internally_reserved":0,"allocated":938606592,"data_stored":937717832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1130744835},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,3,13,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18538496,"num_objects":-4526,"num_object_clones":0,"num_object_copies":-9052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4526,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.492069"},"pg_stats":[{"pgid":"2.f","version":"47'15311","reported_seq":15392,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.168530+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:18.168530+0000","last_peered":"2024-09-15T21:27:18.168530+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:18.168530+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23924736,"num_objects":5841,"num_object_clones":0,"num_object_copies":11682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15311,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1709}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15023","reported_seq":15133,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:18.968644+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:18.968644+0000","last_peered":"2024-09-15T21:27:18.968644+0000","last_clean":"2024-09-15T21:27:18.968644+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:18.968644+0000","last_undegraded":"2024-09-15T21:27:18.968644+0000","last_fullsized":"2024-09-15T21:27:18.968644+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23147462,"num_objects":5653,"num_object_clones":0,"num_object_copies":11306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5653,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13304,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15445","reported_seq":16106,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:19.114018+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:19.114018+0000","last_peered":"2024-09-15T21:27:19.114018+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:19.114018+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23183384,"num_objects":5661,"num_object_clones":0,"num_object_copies":11322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4004,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5661,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13545,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":264,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1657}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14828","reported_seq":14904,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:19.051099+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:19.051099+0000","last_peered":"2024-09-15T21:27:19.051099+0000","last_clean":"2024-09-15T21:27:19.051099+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:19.051099+0000","last_undegraded":"2024-09-15T21:27:19.051099+0000","last_fullsized":"2024-09-15T21:27:19.051099+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22470656,"num_objects":5486,"num_object_clones":0,"num_object_copies":10972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14828,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14937","reported_seq":15075,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:18.191212+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:18.191212+0000","last_peered":"2024-09-15T21:27:18.191212+0000","last_clean":"2024-09-15T21:27:18.191212+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:18.191212+0000","last_undegraded":"2024-09-15T21:27:18.191212+0000","last_fullsized":"2024-09-15T21:27:18.191212+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22807011,"num_objects":5569,"num_object_clones":0,"num_object_copies":11138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11392,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14969","reported_seq":15110,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.225878+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:18.225878+0000","last_peered":"2024-09-15T21:27:18.225878+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:18.225878+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23208419,"num_objects":5667,"num_object_clones":0,"num_object_copies":11334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5667,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11645,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1591},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15100","reported_seq":15239,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.215106+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:18.215106+0000","last_peered":"2024-09-15T21:27:18.215106+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:18.215106+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22815686,"num_objects":5572,"num_object_clones":0,"num_object_copies":11144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11411,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1411},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15273","reported_seq":15407,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:18.238386+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:18.238386+0000","last_peered":"2024-09-15T21:27:18.238386+0000","last_clean":"2024-09-15T21:27:18.238386+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:18.238386+0000","last_undegraded":"2024-09-15T21:27:18.238386+0000","last_fullsized":"2024-09-15T21:27:18.238386+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22917120,"num_objects":5595,"num_object_clones":0,"num_object_copies":11190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5595,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12045,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14963","reported_seq":15095,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:19.113927+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:19.113927+0000","last_peered":"2024-09-15T21:27:19.113927+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:19.113927+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22733283,"num_objects":5551,"num_object_clones":0,"num_object_copies":11102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5551,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12895,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1464}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14878","reported_seq":15014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.196478+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:18.196478+0000","last_peered":"2024-09-15T21:27:18.196478+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:18.196478+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22953984,"num_objects":5604,"num_object_clones":0,"num_object_copies":11208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11551,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1584}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15042","reported_seq":15166,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:19.078013+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:19.078013+0000","last_peered":"2024-09-15T21:27:19.078013+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:19.078013+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22942179,"num_objects":5602,"num_object_clones":0,"num_object_copies":11204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5602,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12901,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1512}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317433+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317433+0000","last_peered":"2024-09-15T21:27:09.317433+0000","last_clean":"2024-09-15T21:27:09.317433+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317433+0000","last_undegraded":"2024-09-15T21:27:09.317433+0000","last_fullsized":"2024-09-15T21:27:09.317433+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15043","reported_seq":23507,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:19.144066+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:19.144066+0000","last_peered":"2024-09-15T21:27:19.144066+0000","last_clean":"2024-09-15T21:27:19.144066+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:19.144066+0000","last_undegraded":"2024-09-15T21:27:19.144066+0000","last_fullsized":"2024-09-15T21:27:19.144066+0000","mapping_epoch":44,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22597632,"num_objects":5517,"num_object_clones":0,"num_object_copies":11034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12736,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15069","reported_seq":15208,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.137645+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:18.137645+0000","last_peered":"2024-09-15T21:27:18.137645+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:18.137645+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23007232,"num_objects":5617,"num_object_clones":0,"num_object_copies":11234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12013,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1523}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317318+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317318+0000","last_peered":"2024-09-15T21:27:09.317318+0000","last_clean":"2024-09-15T21:27:09.317318+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317318+0000","last_undegraded":"2024-09-15T21:27:09.317318+0000","last_fullsized":"2024-09-15T21:27:09.317318+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15069","reported_seq":15207,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.163846+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:18.163846+0000","last_peered":"2024-09-15T21:27:18.163846+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:18.163846+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22614016,"num_objects":5521,"num_object_clones":0,"num_object_copies":11042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11978,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1382}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14888","reported_seq":23203,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:19.061993+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:19.061993+0000","last_peered":"2024-09-15T21:27:19.061993+0000","last_clean":"2024-09-15T21:27:19.061993+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:19.061993+0000","last_undegraded":"2024-09-15T21:27:19.061993+0000","last_fullsized":"2024-09-15T21:27:19.061993+0000","mapping_epoch":46,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22528000,"num_objects":5500,"num_object_clones":0,"num_object_copies":11000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11372,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317284+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317284+0000","last_peered":"2024-09-15T21:27:09.317284+0000","last_clean":"2024-09-15T21:27:09.317284+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317284+0000","last_undegraded":"2024-09-15T21:27:09.317284+0000","last_fullsized":"2024-09-15T21:27:09.317284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15201","reported_seq":15345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:18.223854+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:18.223854+0000","last_peered":"2024-09-15T21:27:18.223854+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:18.223854+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23204806,"num_objects":5667,"num_object_clones":0,"num_object_copies":11334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5667,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11905,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1520},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":367055606,"num_objects":89623,"num_object_clones":0,"num_object_copies":179246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89623,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":200832,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8513,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":935993344,"data_stored":935847906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":639516,"kb_used_data":310728,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93732324,"statfs":{"total":96636764160,"available":95981899776,"internally_reserved":0,"allocated":318185472,"data_stored":317900214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":336653993},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":751360,"kb_used_data":336560,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93620480,"statfs":{"total":96636764160,"available":95867371520,"internally_reserved":0,"allocated":344637440,"data_stored":344352177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":630040,"kb_used_data":269320,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93741800,"statfs":{"total":96636764160,"available":95991603200,"internally_reserved":0,"allocated":275783680,"data_stored":275465441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":343920640,"data_stored":343884510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":274878464,"data_stored":274809358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":317194240,"data_stored":317154038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:24.346 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:24.911 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:24.933 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-15T21:27:24.810509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":357544713,"num_objects":87303,"num_object_clones":0,"num_object_copies":174606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40867,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87303,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":203156,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8608,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160864,"ondisk_log_size":160864,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1998380,"kb_used_data":898208,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281117140,"statfs":{"total":289910292480,"available":287863951360,"internally_reserved":0,"allocated":919764992,"data_stored":918876232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1126484995},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,5,5,11,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28049408,"num_objects":-6848,"num_object_clones":0,"num_object_copies":-13696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6848,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003800"},"pg_stats":[{"pgid":"2.f","version":"47'15464","reported_seq":15545,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.214660+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:23.214660+0000","last_peered":"2024-09-15T21:27:23.214660+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:23.214660+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23298048,"num_objects":5688,"num_object_clones":0,"num_object_copies":11376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15464,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1556}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15167","reported_seq":15277,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.012220+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:24.012220+0000","last_peered":"2024-09-15T21:27:24.012220+0000","last_clean":"2024-09-15T21:27:24.012220+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:24.012220+0000","last_undegraded":"2024-09-15T21:27:24.012220+0000","last_fullsized":"2024-09-15T21:27:24.012220+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22557638,"num_objects":5509,"num_object_clones":0,"num_object_copies":11018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5509,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13448,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15578","reported_seq":16427,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:24.116043+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:24.116043+0000","last_peered":"2024-09-15T21:27:24.116043+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:24.116043+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22638616,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13678,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1618}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14980","reported_seq":15056,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.151244+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:24.151244+0000","last_peered":"2024-09-15T21:27:24.151244+0000","last_clean":"2024-09-15T21:27:24.151244+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:24.151244+0000","last_undegraded":"2024-09-15T21:27:24.151244+0000","last_fullsized":"2024-09-15T21:27:24.151244+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848064,"num_objects":5334,"num_object_clones":0,"num_object_copies":10668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14980,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15062","reported_seq":15200,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:23.246550+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:23.246550+0000","last_peered":"2024-09-15T21:27:23.246550+0000","last_clean":"2024-09-15T21:27:23.246550+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:23.246550+0000","last_undegraded":"2024-09-15T21:27:23.246550+0000","last_fullsized":"2024-09-15T21:27:23.246550+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22295011,"num_objects":5444,"num_object_clones":0,"num_object_copies":10888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11517,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15111","reported_seq":15252,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.200450+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:23.200450+0000","last_peered":"2024-09-15T21:27:23.200450+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:23.200450+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626787,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11787,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1449},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15236","reported_seq":15377,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.082619+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:23.082619+0000","last_peered":"2024-09-15T21:27:23.082619+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:23.082619+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22258630,"num_objects":5436,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11547,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1275},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15413","reported_seq":15547,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:23.189982+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:23.189982+0000","last_peered":"2024-09-15T21:27:23.189982+0000","last_clean":"2024-09-15T21:27:23.189982+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:23.189982+0000","last_undegraded":"2024-09-15T21:27:23.189982+0000","last_fullsized":"2024-09-15T21:27:23.189982+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22343680,"num_objects":5455,"num_object_clones":0,"num_object_copies":10910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12185,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15105","reported_seq":15236,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:24.144969+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:24.144969+0000","last_peered":"2024-09-15T21:27:24.144969+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:24.144969+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22151651,"num_objects":5409,"num_object_clones":0,"num_object_copies":10818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13037,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1322}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15052","reported_seq":15188,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.264587+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:23.264587+0000","last_peered":"2024-09-15T21:27:23.264587+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:23.264587+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22241280,"num_objects":5430,"num_object_clones":0,"num_object_copies":10860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11725,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1410}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15198","reported_seq":15321,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:24.144051+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:24.144051+0000","last_peered":"2024-09-15T21:27:24.144051+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:24.144051+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22303203,"num_objects":5446,"num_object_clones":0,"num_object_copies":10892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5446,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13057,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1356}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317433+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317433+0000","last_peered":"2024-09-15T21:27:09.317433+0000","last_clean":"2024-09-15T21:27:09.317433+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317433+0000","last_undegraded":"2024-09-15T21:27:09.317433+0000","last_fullsized":"2024-09-15T21:27:09.317433+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15182","reported_seq":23646,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.100346+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:24.100346+0000","last_peered":"2024-09-15T21:27:24.100346+0000","last_clean":"2024-09-15T21:27:24.100346+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:24.100346+0000","last_undegraded":"2024-09-15T21:27:24.100346+0000","last_fullsized":"2024-09-15T21:27:24.100346+0000","mapping_epoch":44,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22028288,"num_objects":5378,"num_object_clones":0,"num_object_copies":10756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12875,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15216","reported_seq":15355,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.218914+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:23.218914+0000","last_peered":"2024-09-15T21:27:23.218914+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:23.218914+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22405120,"num_objects":5470,"num_object_clones":0,"num_object_copies":10940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12160,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317318+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317318+0000","last_peered":"2024-09-15T21:27:09.317318+0000","last_clean":"2024-09-15T21:27:09.317318+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317318+0000","last_undegraded":"2024-09-15T21:27:09.317318+0000","last_fullsized":"2024-09-15T21:27:09.317318+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15199","reported_seq":15337,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.219501+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:23.219501+0000","last_peered":"2024-09-15T21:27:23.219501+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:23.219501+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22081536,"num_objects":5391,"num_object_clones":0,"num_object_copies":10782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12108,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15041","reported_seq":23354,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.319750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:24.319750+0000","last_peered":"2024-09-15T21:27:24.319750+0000","last_clean":"2024-09-15T21:27:24.319750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:24.319750+0000","last_undegraded":"2024-09-15T21:27:24.319750+0000","last_fullsized":"2024-09-15T21:27:24.319750+0000","mapping_epoch":46,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11525,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317284+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317284+0000","last_peered":"2024-09-15T21:27:09.317284+0000","last_clean":"2024-09-15T21:27:09.317284+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317284+0000","last_undegraded":"2024-09-15T21:27:09.317284+0000","last_fullsized":"2024-09-15T21:27:09.317284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15357","reported_seq":15501,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.217300+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:23.217300+0000","last_peered":"2024-09-15T21:27:23.217300+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:23.217300+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22565830,"num_objects":5511,"num_object_clones":0,"num_object_copies":11022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12061,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1364},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":357544694,"num_objects":87301,"num_object_clones":0,"num_object_copies":174602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40867,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87301,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":203154,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8607,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":916783104,"data_stored":916637666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160861,"ondisk_log_size":160861,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622560,"kb_used_data":306252,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93749280,"statfs":{"total":96636764160,"available":95999262720,"internally_reserved":0,"allocated":313602048,"data_stored":313316790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":748584,"kb_used_data":329612,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93623256,"statfs":{"total":96636764160,"available":95870214144,"internally_reserved":0,"allocated":337522688,"data_stored":337237425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,3,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627236,"kb_used_data":262344,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93744604,"statfs":{"total":96636764160,"available":95994474496,"internally_reserved":0,"allocated":268640256,"data_stored":268322017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":336785408,"data_stored":336749278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":267530240,"data_stored":267461134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312467456,"data_stored":312427254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:24.935 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:25.493 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:25.493 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:25.521 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-15T21:27:24.810509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":357544713,"num_objects":87303,"num_object_clones":0,"num_object_copies":174606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40867,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87303,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":203156,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8608,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160864,"ondisk_log_size":160864,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1998380,"kb_used_data":898208,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281117140,"statfs":{"total":289910292480,"available":287863951360,"internally_reserved":0,"allocated":919764992,"data_stored":918876232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1126484995},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,5,5,11,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28049408,"num_objects":-6848,"num_object_clones":0,"num_object_copies":-13696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-295,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6848,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003800"},"pg_stats":[{"pgid":"2.f","version":"47'15464","reported_seq":15545,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.214660+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:23.214660+0000","last_peered":"2024-09-15T21:27:23.214660+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:23.214660+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23298048,"num_objects":5688,"num_object_clones":0,"num_object_copies":11376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15464,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1556}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15167","reported_seq":15277,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.012220+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:24.012220+0000","last_peered":"2024-09-15T21:27:24.012220+0000","last_clean":"2024-09-15T21:27:24.012220+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:24.012220+0000","last_undegraded":"2024-09-15T21:27:24.012220+0000","last_fullsized":"2024-09-15T21:27:24.012220+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22557638,"num_objects":5509,"num_object_clones":0,"num_object_copies":11018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5509,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13448,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15578","reported_seq":16427,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:24.116043+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:24.116043+0000","last_peered":"2024-09-15T21:27:24.116043+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:24.116043+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22638616,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13678,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1618}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14980","reported_seq":15056,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.151244+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:24.151244+0000","last_peered":"2024-09-15T21:27:24.151244+0000","last_clean":"2024-09-15T21:27:24.151244+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:24.151244+0000","last_undegraded":"2024-09-15T21:27:24.151244+0000","last_fullsized":"2024-09-15T21:27:24.151244+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848064,"num_objects":5334,"num_object_clones":0,"num_object_copies":10668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14980,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15062","reported_seq":15200,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:23.246550+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:23.246550+0000","last_peered":"2024-09-15T21:27:23.246550+0000","last_clean":"2024-09-15T21:27:23.246550+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:23.246550+0000","last_undegraded":"2024-09-15T21:27:23.246550+0000","last_fullsized":"2024-09-15T21:27:23.246550+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22295011,"num_objects":5444,"num_object_clones":0,"num_object_copies":10888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11517,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15111","reported_seq":15252,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.200450+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:23.200450+0000","last_peered":"2024-09-15T21:27:23.200450+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:23.200450+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626787,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11787,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1449},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15236","reported_seq":15377,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.082619+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:23.082619+0000","last_peered":"2024-09-15T21:27:23.082619+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:23.082619+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22258630,"num_objects":5436,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11547,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1275},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15413","reported_seq":15547,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:23.189982+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:23.189982+0000","last_peered":"2024-09-15T21:27:23.189982+0000","last_clean":"2024-09-15T21:27:23.189982+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:23.189982+0000","last_undegraded":"2024-09-15T21:27:23.189982+0000","last_fullsized":"2024-09-15T21:27:23.189982+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22343680,"num_objects":5455,"num_object_clones":0,"num_object_copies":10910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12185,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15105","reported_seq":15236,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:24.144969+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:24.144969+0000","last_peered":"2024-09-15T21:27:24.144969+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:24.144969+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22151651,"num_objects":5409,"num_object_clones":0,"num_object_copies":10818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13037,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1322}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15052","reported_seq":15188,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.264587+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:23.264587+0000","last_peered":"2024-09-15T21:27:23.264587+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:23.264587+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22241280,"num_objects":5430,"num_object_clones":0,"num_object_copies":10860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11725,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1410}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15198","reported_seq":15321,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:24.144051+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:24.144051+0000","last_peered":"2024-09-15T21:27:24.144051+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:24.144051+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22303203,"num_objects":5446,"num_object_clones":0,"num_object_copies":10892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5446,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13057,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1356}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317433+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317433+0000","last_peered":"2024-09-15T21:27:09.317433+0000","last_clean":"2024-09-15T21:27:09.317433+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317433+0000","last_undegraded":"2024-09-15T21:27:09.317433+0000","last_fullsized":"2024-09-15T21:27:09.317433+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15182","reported_seq":23646,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.100346+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:24.100346+0000","last_peered":"2024-09-15T21:27:24.100346+0000","last_clean":"2024-09-15T21:27:24.100346+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:24.100346+0000","last_undegraded":"2024-09-15T21:27:24.100346+0000","last_fullsized":"2024-09-15T21:27:24.100346+0000","mapping_epoch":44,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22028288,"num_objects":5378,"num_object_clones":0,"num_object_copies":10756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12875,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15216","reported_seq":15355,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.218914+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:23.218914+0000","last_peered":"2024-09-15T21:27:23.218914+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:23.218914+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22405120,"num_objects":5470,"num_object_clones":0,"num_object_copies":10940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12160,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317318+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317318+0000","last_peered":"2024-09-15T21:27:09.317318+0000","last_clean":"2024-09-15T21:27:09.317318+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317318+0000","last_undegraded":"2024-09-15T21:27:09.317318+0000","last_fullsized":"2024-09-15T21:27:09.317318+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15199","reported_seq":15337,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.219501+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:23.219501+0000","last_peered":"2024-09-15T21:27:23.219501+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:23.219501+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22081536,"num_objects":5391,"num_object_clones":0,"num_object_copies":10782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12108,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15041","reported_seq":23354,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.319750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:24.319750+0000","last_peered":"2024-09-15T21:27:24.319750+0000","last_clean":"2024-09-15T21:27:24.319750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:24.319750+0000","last_undegraded":"2024-09-15T21:27:24.319750+0000","last_fullsized":"2024-09-15T21:27:24.319750+0000","mapping_epoch":46,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11525,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317284+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317284+0000","last_peered":"2024-09-15T21:27:09.317284+0000","last_clean":"2024-09-15T21:27:09.317284+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317284+0000","last_undegraded":"2024-09-15T21:27:09.317284+0000","last_fullsized":"2024-09-15T21:27:09.317284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15357","reported_seq":15501,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.217300+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:23.217300+0000","last_peered":"2024-09-15T21:27:23.217300+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:23.217300+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22565830,"num_objects":5511,"num_object_clones":0,"num_object_copies":11022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12061,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1364},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":357544694,"num_objects":87301,"num_object_clones":0,"num_object_copies":174602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40867,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87301,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":203154,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8607,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":916783104,"data_stored":916637666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160861,"ondisk_log_size":160861,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622560,"kb_used_data":306252,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93749280,"statfs":{"total":96636764160,"available":95999262720,"internally_reserved":0,"allocated":313602048,"data_stored":313316790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":748584,"kb_used_data":329612,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93623256,"statfs":{"total":96636764160,"available":95870214144,"internally_reserved":0,"allocated":337522688,"data_stored":337237425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,3,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627236,"kb_used_data":262344,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93744604,"statfs":{"total":96636764160,"available":95994474496,"internally_reserved":0,"allocated":268640256,"data_stored":268322017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":336785408,"data_stored":336749278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":267530240,"data_stored":267461134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312467456,"data_stored":312427254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:25.523 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:25.523 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'15464", 'reported_seq': 15545, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:23.214660+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:27:23.214660+0000', 'last_peered': '2024-09-15T21:27:23.214660+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:27:23.214660+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23298048, 'num_objects': 5688, 'num_object_clones': 0, 'num_object_copies': 11376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5688, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15464, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1556}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:25.523 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:27:25.523 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'15578", 'reported_seq': 16427, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:24.116043+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:27:24.116043+0000', 'last_peered': '2024-09-15T21:27:24.116043+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:27:24.116043+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22638616, 'num_objects': 5528, 'num_object_clones': 0, 'num_object_copies': 11056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3910, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5528, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13678, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 358, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1618}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:25.523 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:25.523 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'15111", 'reported_seq': 15252, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:23.200450+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:27:23.200450+0000', 'last_peered': '2024-09-15T21:27:23.200450+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:27:23.200450+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22626787, 'num_objects': 5525, 'num_object_clones': 0, 'num_object_copies': 11050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5525, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11787, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1449}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:25.523 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:25.523 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'15236", 'reported_seq': 15377, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:23.082619+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:27:23.082619+0000', 'last_peered': '2024-09-15T21:27:23.082619+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:27:23.082619+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22258630, 'num_objects': 5436, 'num_object_clones': 0, 'num_object_copies': 10872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5436, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11547, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1275}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:25.524 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:27:25.524 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'15105", 'reported_seq': 15236, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:24.144969+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:27:24.144969+0000', 'last_peered': '2024-09-15T21:27:24.144969+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:27:24.144969+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22151651, 'num_objects': 5409, 'num_object_clones': 0, 'num_object_copies': 10818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5409, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13037, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1322}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:25.524 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:25.524 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15052", 'reported_seq': 15188, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:23.264587+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:27:23.264587+0000', 'last_peered': '2024-09-15T21:27:23.264587+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:27:23.264587+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22241280, 'num_objects': 5430, 'num_object_clones': 0, 'num_object_copies': 10860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5430, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11725, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1410}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:25.524 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:27:25.524 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'15198", 'reported_seq': 15321, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:24.144051+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:27:24.144051+0000', 'last_peered': '2024-09-15T21:27:24.144051+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:27:24.144051+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22303203, 'num_objects': 5446, 'num_object_clones': 0, 'num_object_copies': 10892, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5446, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13057, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1356}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:25.524 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:25.524 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'15216", 'reported_seq': 15355, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:23.218914+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:27:23.218914+0000', 'last_peered': '2024-09-15T21:27:23.218914+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:27:23.218914+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22405120, 'num_objects': 5470, 'num_object_clones': 0, 'num_object_copies': 10940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5470, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12160, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1376}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:25.524 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:27:25.524 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'15199", 'reported_seq': 15337, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:23.219501+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:27:23.219501+0000', 'last_peered': '2024-09-15T21:27:23.219501+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:27:23.219501+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22081536, 'num_objects': 5391, 'num_object_clones': 0, 'num_object_copies': 10782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5391, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12108, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1252}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:25.524 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:27:25.525 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'15357", 'reported_seq': 15501, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:23.217300+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:27:23.217300+0000', 'last_peered': '2024-09-15T21:27:23.217300+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:27:23.217300+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22565830, 'num_objects': 5511, 'num_object_clones': 0, 'num_object_copies': 11022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5511, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12061, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1364}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:25.525 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:26.253 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:26.253 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40867/174606 objects degraded (23.405%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":497,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":87303,"data_bytes":357544713,"bytes_used":2046341120,"bytes_avail":287863951360,"bytes_total":289910292480,"degraded_objects":40867,"degraded_total":174606,"degraded_ratio":0.2340526671477498,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":295,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":570},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:26.277 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:26.277 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:26.870 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:26.872 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:26.894 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-15T21:27:26.811255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":357544713,"num_objects":87303,"num_object_clones":0,"num_object_copies":174606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40867,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87303,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":203156,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8608,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160864,"ondisk_log_size":160864,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1998380,"kb_used_data":898208,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281117140,"statfs":{"total":289910292480,"available":287863951360,"internally_reserved":0,"allocated":919764992,"data_stored":918876232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1126484995},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,5,5,11,10,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19087360,"num_objects":-4660,"num_object_clones":0,"num_object_copies":-9320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4660,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004087"},"pg_stats":[{"pgid":"2.f","version":"47'15464","reported_seq":15545,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.214660+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:23.214660+0000","last_peered":"2024-09-15T21:27:23.214660+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:23.214660+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23298048,"num_objects":5688,"num_object_clones":0,"num_object_copies":11376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15464,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1556}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15167","reported_seq":15277,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.012220+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:24.012220+0000","last_peered":"2024-09-15T21:27:24.012220+0000","last_clean":"2024-09-15T21:27:24.012220+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:24.012220+0000","last_undegraded":"2024-09-15T21:27:24.012220+0000","last_fullsized":"2024-09-15T21:27:24.012220+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22557638,"num_objects":5509,"num_object_clones":0,"num_object_copies":11018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5509,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13448,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15578","reported_seq":16427,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:24.116043+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:24.116043+0000","last_peered":"2024-09-15T21:27:24.116043+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:24.116043+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22638616,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13678,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1618}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14980","reported_seq":15056,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.151244+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:24.151244+0000","last_peered":"2024-09-15T21:27:24.151244+0000","last_clean":"2024-09-15T21:27:24.151244+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:24.151244+0000","last_undegraded":"2024-09-15T21:27:24.151244+0000","last_fullsized":"2024-09-15T21:27:24.151244+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848064,"num_objects":5334,"num_object_clones":0,"num_object_copies":10668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14980,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15062","reported_seq":15200,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:23.246550+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:23.246550+0000","last_peered":"2024-09-15T21:27:23.246550+0000","last_clean":"2024-09-15T21:27:23.246550+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:23.246550+0000","last_undegraded":"2024-09-15T21:27:23.246550+0000","last_fullsized":"2024-09-15T21:27:23.246550+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22295011,"num_objects":5444,"num_object_clones":0,"num_object_copies":10888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11517,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15111","reported_seq":15252,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.200450+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:23.200450+0000","last_peered":"2024-09-15T21:27:23.200450+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:23.200450+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626787,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11787,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1449},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15236","reported_seq":15377,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.082619+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:23.082619+0000","last_peered":"2024-09-15T21:27:23.082619+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:23.082619+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22258630,"num_objects":5436,"num_object_clones":0,"num_object_copies":10872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11547,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1275},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15413","reported_seq":15547,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:23.189982+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:23.189982+0000","last_peered":"2024-09-15T21:27:23.189982+0000","last_clean":"2024-09-15T21:27:23.189982+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:23.189982+0000","last_undegraded":"2024-09-15T21:27:23.189982+0000","last_fullsized":"2024-09-15T21:27:23.189982+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22343680,"num_objects":5455,"num_object_clones":0,"num_object_copies":10910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12185,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15105","reported_seq":15236,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:24.144969+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:24.144969+0000","last_peered":"2024-09-15T21:27:24.144969+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:24.144969+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22151651,"num_objects":5409,"num_object_clones":0,"num_object_copies":10818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13037,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1322}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15052","reported_seq":15188,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.264587+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:23.264587+0000","last_peered":"2024-09-15T21:27:23.264587+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:23.264587+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22241280,"num_objects":5430,"num_object_clones":0,"num_object_copies":10860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11725,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1410}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15198","reported_seq":15321,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:24.144051+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:24.144051+0000","last_peered":"2024-09-15T21:27:24.144051+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:24.144051+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22303203,"num_objects":5446,"num_object_clones":0,"num_object_copies":10892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5446,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13057,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1356}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317433+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317433+0000","last_peered":"2024-09-15T21:27:09.317433+0000","last_clean":"2024-09-15T21:27:09.317433+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317433+0000","last_undegraded":"2024-09-15T21:27:09.317433+0000","last_fullsized":"2024-09-15T21:27:09.317433+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15182","reported_seq":23646,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.100346+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:24.100346+0000","last_peered":"2024-09-15T21:27:24.100346+0000","last_clean":"2024-09-15T21:27:24.100346+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:24.100346+0000","last_undegraded":"2024-09-15T21:27:24.100346+0000","last_fullsized":"2024-09-15T21:27:24.100346+0000","mapping_epoch":44,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22028288,"num_objects":5378,"num_object_clones":0,"num_object_copies":10756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12875,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15216","reported_seq":15355,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.218914+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:23.218914+0000","last_peered":"2024-09-15T21:27:23.218914+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:23.218914+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22405120,"num_objects":5470,"num_object_clones":0,"num_object_copies":10940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12160,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317318+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317318+0000","last_peered":"2024-09-15T21:27:09.317318+0000","last_clean":"2024-09-15T21:27:09.317318+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317318+0000","last_undegraded":"2024-09-15T21:27:09.317318+0000","last_fullsized":"2024-09-15T21:27:09.317318+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15199","reported_seq":15337,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.219501+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:23.219501+0000","last_peered":"2024-09-15T21:27:23.219501+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:23.219501+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22081536,"num_objects":5391,"num_object_clones":0,"num_object_copies":10782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12108,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1252}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15041","reported_seq":23354,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.319750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:24.319750+0000","last_peered":"2024-09-15T21:27:24.319750+0000","last_clean":"2024-09-15T21:27:24.319750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:24.319750+0000","last_undegraded":"2024-09-15T21:27:24.319750+0000","last_fullsized":"2024-09-15T21:27:24.319750+0000","mapping_epoch":46,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11525,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317284+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317284+0000","last_peered":"2024-09-15T21:27:09.317284+0000","last_clean":"2024-09-15T21:27:09.317284+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317284+0000","last_undegraded":"2024-09-15T21:27:09.317284+0000","last_fullsized":"2024-09-15T21:27:09.317284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15357","reported_seq":15501,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:23.217300+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:23.217300+0000","last_peered":"2024-09-15T21:27:23.217300+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:23.217300+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22565830,"num_objects":5511,"num_object_clones":0,"num_object_copies":11022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12061,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1364},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":357544694,"num_objects":87301,"num_object_clones":0,"num_object_copies":174602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40867,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87301,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":203154,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8607,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":916783104,"data_stored":916637666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160861,"ondisk_log_size":160861,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622560,"kb_used_data":306252,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93749280,"statfs":{"total":96636764160,"available":95999262720,"internally_reserved":0,"allocated":313602048,"data_stored":313316790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":748584,"kb_used_data":329612,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93623256,"statfs":{"total":96636764160,"available":95870214144,"internally_reserved":0,"allocated":337522688,"data_stored":337237425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,3,3,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627236,"kb_used_data":262344,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93744604,"statfs":{"total":96636764160,"available":95994474496,"internally_reserved":0,"allocated":268640256,"data_stored":268322017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":336785408,"data_stored":336749278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":267530240,"data_stored":267461134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312467456,"data_stored":312427254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:29.896 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:30.462 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:30.465 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:30.484 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":290,"stamp":"2024-09-15T21:27:28.811679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":352162569,"num_objects":85989,"num_object_clones":0,"num_object_copies":171978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40867,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85989,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":204470,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8608,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160878,"ondisk_log_size":160878,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1983004,"kb_used_data":891156,"kb_used_omap":12,"kb_used_meta":1091763,"kb_avail":281132516,"statfs":{"total":289910292480,"available":287879696384,"internally_reserved":0,"allocated":912543744,"data_stored":911654984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1117965315},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,6,5,11,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24469504,"num_objects":-5974,"num_object_clones":0,"num_object_copies":-11948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5974,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003851"},"pg_stats":[{"pgid":"2.f","version":"47'15603","reported_seq":15684,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.200420+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:28.200420+0000","last_peered":"2024-09-15T21:27:28.200420+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:28.200420+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22728704,"num_objects":5549,"num_object_clones":0,"num_object_copies":11098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15603,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1417}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15167","reported_seq":15277,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.012220+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:24.012220+0000","last_peered":"2024-09-15T21:27:24.012220+0000","last_clean":"2024-09-15T21:27:24.012220+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:24.012220+0000","last_undegraded":"2024-09-15T21:27:24.012220+0000","last_fullsized":"2024-09-15T21:27:24.012220+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22557638,"num_objects":5509,"num_object_clones":0,"num_object_copies":11018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5509,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13448,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15578","reported_seq":16427,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:24.116043+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:24.116043+0000","last_peered":"2024-09-15T21:27:24.116043+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:24.116043+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22638616,"num_objects":5528,"num_object_clones":0,"num_object_copies":11056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13678,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1618}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14980","reported_seq":15056,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.151244+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:24.151244+0000","last_peered":"2024-09-15T21:27:24.151244+0000","last_clean":"2024-09-15T21:27:24.151244+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:24.151244+0000","last_undegraded":"2024-09-15T21:27:24.151244+0000","last_fullsized":"2024-09-15T21:27:24.151244+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21848064,"num_objects":5334,"num_object_clones":0,"num_object_copies":10668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14980,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15216","reported_seq":15354,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.267798+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:28.267798+0000","last_peered":"2024-09-15T21:27:28.267798+0000","last_clean":"2024-09-15T21:27:28.267798+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:28.267798+0000","last_undegraded":"2024-09-15T21:27:28.267798+0000","last_fullsized":"2024-09-15T21:27:28.267798+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21664227,"num_objects":5290,"num_object_clones":0,"num_object_copies":10580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11671,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15271","reported_seq":15407,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.250956+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:28.250956+0000","last_peered":"2024-09-15T21:27:28.250956+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:28.250956+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21971427,"num_objects":5365,"num_object_clones":0,"num_object_copies":10730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11947,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1289},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15373","reported_seq":15514,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.204471+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:28.204471+0000","last_peered":"2024-09-15T21:27:28.204471+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:28.204471+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21697478,"num_objects":5299,"num_object_clones":0,"num_object_copies":10598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11684,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1138},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15550","reported_seq":15684,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.230687+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:28.230687+0000","last_peered":"2024-09-15T21:27:28.230687+0000","last_clean":"2024-09-15T21:27:28.230687+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:28.230687+0000","last_undegraded":"2024-09-15T21:27:28.230687+0000","last_fullsized":"2024-09-15T21:27:28.230687+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21782528,"num_objects":5318,"num_object_clones":0,"num_object_copies":10636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12322,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15105","reported_seq":15236,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:24.144969+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:24.144969+0000","last_peered":"2024-09-15T21:27:24.144969+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:24.144969+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22151651,"num_objects":5409,"num_object_clones":0,"num_object_copies":10818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13037,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1322}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15184","reported_seq":15320,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.223211+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:28.223211+0000","last_peered":"2024-09-15T21:27:28.223211+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:28.223211+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21700608,"num_objects":5298,"num_object_clones":0,"num_object_copies":10596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11857,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1278}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15198","reported_seq":15321,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:24.144051+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:24.144051+0000","last_peered":"2024-09-15T21:27:24.144051+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:24.144051+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22303203,"num_objects":5446,"num_object_clones":0,"num_object_copies":10892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5446,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13057,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1356}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317433+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317433+0000","last_peered":"2024-09-15T21:27:09.317433+0000","last_clean":"2024-09-15T21:27:09.317433+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317433+0000","last_undegraded":"2024-09-15T21:27:09.317433+0000","last_fullsized":"2024-09-15T21:27:09.317433+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15182","reported_seq":23646,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.100346+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:24.100346+0000","last_peered":"2024-09-15T21:27:24.100346+0000","last_clean":"2024-09-15T21:27:24.100346+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:24.100346+0000","last_undegraded":"2024-09-15T21:27:24.100346+0000","last_fullsized":"2024-09-15T21:27:24.100346+0000","mapping_epoch":44,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22028288,"num_objects":5378,"num_object_clones":0,"num_object_copies":10756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12875,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15365","reported_seq":15504,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.230796+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:28.230796+0000","last_peered":"2024-09-15T21:27:28.230796+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:28.230796+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21794816,"num_objects":5321,"num_object_clones":0,"num_object_copies":10642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12309,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1227}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317318+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317318+0000","last_peered":"2024-09-15T21:27:09.317318+0000","last_clean":"2024-09-15T21:27:09.317318+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317318+0000","last_undegraded":"2024-09-15T21:27:09.317318+0000","last_fullsized":"2024-09-15T21:27:09.317318+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15354","reported_seq":15492,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.255403+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:28.255403+0000","last_peered":"2024-09-15T21:27:28.255403+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:28.255403+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21446656,"num_objects":5236,"num_object_clones":0,"num_object_copies":10472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12263,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1097}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15041","reported_seq":23354,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:24.319750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:24.319750+0000","last_peered":"2024-09-15T21:27:24.319750+0000","last_clean":"2024-09-15T21:27:24.319750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:24.319750+0000","last_undegraded":"2024-09-15T21:27:24.319750+0000","last_fullsized":"2024-09-15T21:27:24.319750+0000","mapping_epoch":46,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21901312,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11525,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317284+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317284+0000","last_peered":"2024-09-15T21:27:09.317284+0000","last_clean":"2024-09-15T21:27:09.317284+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317284+0000","last_undegraded":"2024-09-15T21:27:09.317284+0000","last_fullsized":"2024-09-15T21:27:09.317284+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15508","reported_seq":15652,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.196171+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:28.196171+0000","last_peered":"2024-09-15T21:27:28.196171+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:28.196171+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21947334,"num_objects":5360,"num_object_clones":0,"num_object_copies":10720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5360,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12212,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1213},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":352162550,"num_objects":85987,"num_object_clones":0,"num_object_copies":171974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40867,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85987,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":204468,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8607,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":909619200,"data_stored":909473762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622560,"kb_used_data":306252,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93749280,"statfs":{"total":96636764160,"available":95999262720,"internally_reserved":0,"allocated":313602048,"data_stored":313316790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":733208,"kb_used_data":322560,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93638632,"statfs":{"total":96636764160,"available":95885959168,"internally_reserved":0,"allocated":330301440,"data_stored":330016177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":627236,"kb_used_data":262344,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93744604,"statfs":{"total":96636764160,"available":95994474496,"internally_reserved":0,"allocated":268640256,"data_stored":268322017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":329621504,"data_stored":329585374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":267530240,"data_stored":267461134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312467456,"data_stored":312427254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:30.486 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:31.022 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:31.023 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:31.051 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-15T21:27:30.812288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":347906825,"num_objects":84950,"num_object_clones":0,"num_object_copies":169900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84950,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":205509,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8692,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160717,"ondisk_log_size":160717,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1967440,"kb_used_data":879764,"kb_used_omap":12,"kb_used_meta":1087603,"kb_avail":281148080,"statfs":{"total":289910292480,"available":287895633920,"internally_reserved":0,"allocated":900878336,"data_stored":899989576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1113705475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,8,7,12,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23408640,"num_objects":-5715,"num_object_clones":0,"num_object_copies":-11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-276,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5715,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004031"},"pg_stats":[{"pgid":"2.f","version":"47'15603","reported_seq":15684,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.200420+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:28.200420+0000","last_peered":"2024-09-15T21:27:28.200420+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:28.200420+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22728704,"num_objects":5549,"num_object_clones":0,"num_object_copies":11098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15603,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1417}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15324","reported_seq":15435,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.185654+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:29.185654+0000","last_peered":"2024-09-15T21:27:29.185654+0000","last_clean":"2024-09-15T21:27:29.185654+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:29.185654+0000","last_undegraded":"2024-09-15T21:27:29.185654+0000","last_fullsized":"2024-09-15T21:27:29.185654+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21914566,"num_objects":5352,"num_object_clones":0,"num_object_copies":10704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5352,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13605,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15738","reported_seq":16755,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:29.159142+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:29.159142+0000","last_peered":"2024-09-15T21:27:29.159142+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:29.159142+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21983256,"num_objects":5368,"num_object_clones":0,"num_object_copies":10736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3826,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5368,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13838,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1542}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15119","reported_seq":15195,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.149545+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:29.149545+0000","last_peered":"2024-09-15T21:27:29.149545+0000","last_clean":"2024-09-15T21:27:29.149545+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:29.149545+0000","last_undegraded":"2024-09-15T21:27:29.149545+0000","last_fullsized":"2024-09-15T21:27:29.149545+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21278720,"num_objects":5195,"num_object_clones":0,"num_object_copies":10390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15119,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15216","reported_seq":15354,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.267798+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:28.267798+0000","last_peered":"2024-09-15T21:27:28.267798+0000","last_clean":"2024-09-15T21:27:28.267798+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:28.267798+0000","last_undegraded":"2024-09-15T21:27:28.267798+0000","last_fullsized":"2024-09-15T21:27:28.267798+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21664227,"num_objects":5290,"num_object_clones":0,"num_object_copies":10580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11671,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15271","reported_seq":15407,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.250956+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:28.250956+0000","last_peered":"2024-09-15T21:27:28.250956+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:28.250956+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21971427,"num_objects":5365,"num_object_clones":0,"num_object_copies":10730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11947,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1289},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15373","reported_seq":15514,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.204471+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:28.204471+0000","last_peered":"2024-09-15T21:27:28.204471+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:28.204471+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21697478,"num_objects":5299,"num_object_clones":0,"num_object_copies":10598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11684,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1138},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15550","reported_seq":15684,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.230687+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:28.230687+0000","last_peered":"2024-09-15T21:27:28.230687+0000","last_clean":"2024-09-15T21:27:28.230687+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:28.230687+0000","last_undegraded":"2024-09-15T21:27:28.230687+0000","last_fullsized":"2024-09-15T21:27:28.230687+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21782528,"num_objects":5318,"num_object_clones":0,"num_object_copies":10636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12322,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15249","reported_seq":15381,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:29.183349+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:29.183349+0000","last_peered":"2024-09-15T21:27:29.183349+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:29.183349+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561827,"num_objects":5265,"num_object_clones":0,"num_object_copies":10530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5265,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13181,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1178}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15184","reported_seq":15320,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.223211+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:28.223211+0000","last_peered":"2024-09-15T21:27:28.223211+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:28.223211+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21700608,"num_objects":5298,"num_object_clones":0,"num_object_copies":10596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11857,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1278}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15336","reported_seq":15459,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:29.127810+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:29.127810+0000","last_peered":"2024-09-15T21:27:29.127810+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:29.127810+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21737955,"num_objects":5308,"num_object_clones":0,"num_object_copies":10616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13195,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1218}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15335","reported_seq":23798,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.158375+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:29.158375+0000","last_peered":"2024-09-15T21:27:29.158375+0000","last_clean":"2024-09-15T21:27:29.158375+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:29.158375+0000","last_undegraded":"2024-09-15T21:27:29.158375+0000","last_fullsized":"2024-09-15T21:27:29.158375+0000","mapping_epoch":44,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21401600,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13028,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15365","reported_seq":15504,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.230796+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:28.230796+0000","last_peered":"2024-09-15T21:27:28.230796+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:28.230796+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21794816,"num_objects":5321,"num_object_clones":0,"num_object_copies":10642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12309,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1227}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15354","reported_seq":15492,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.255403+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:28.255403+0000","last_peered":"2024-09-15T21:27:28.255403+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:28.255403+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21446656,"num_objects":5236,"num_object_clones":0,"num_object_copies":10472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12263,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1097}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15189","reported_seq":23504,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.321078+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:29.321078+0000","last_peered":"2024-09-15T21:27:29.321078+0000","last_clean":"2024-09-15T21:27:29.321078+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:29.321078+0000","last_undegraded":"2024-09-15T21:27:29.321078+0000","last_fullsized":"2024-09-15T21:27:29.321078+0000","mapping_epoch":46,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21295104,"num_objects":5199,"num_object_clones":0,"num_object_copies":10398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11673,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15508","reported_seq":15652,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.196171+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:28.196171+0000","last_peered":"2024-09-15T21:27:28.196171+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:28.196171+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21947334,"num_objects":5360,"num_object_clones":0,"num_object_copies":10720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5360,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12212,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1213},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":347906806,"num_objects":84948,"num_object_clones":0,"num_object_copies":169896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84948,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":205507,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8691,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":897593344,"data_stored":897447906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622144,"kb_used_data":301680,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93749696,"statfs":{"total":96636764160,"available":95999688704,"internally_reserved":0,"allocated":308920320,"data_stored":308635062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":328134313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,6,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":733208,"kb_used_data":322560,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93638632,"statfs":{"total":96636764160,"available":95885959168,"internally_reserved":0,"allocated":330301440,"data_stored":330016177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612088,"kb_used_data":255524,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93759752,"statfs":{"total":96636764160,"available":96009986048,"internally_reserved":0,"allocated":261656576,"data_stored":261338337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":329621504,"data_stored":329585374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260349952,"data_stored":260280846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307621888,"data_stored":307581686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:31.052 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:31.052 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'15603", 'reported_seq': 15684, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.200420+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:27:28.200420+0000', 'last_peered': '2024-09-15T21:27:28.200420+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:27:28.200420+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22728704, 'num_objects': 5549, 'num_object_clones': 0, 'num_object_copies': 11098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5549, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15603, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1417}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:31.053 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:27:31.053 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'15738", 'reported_seq': 16755, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:29.159142+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:27:29.159142+0000', 'last_peered': '2024-09-15T21:27:29.159142+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:27:29.159142+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21983256, 'num_objects': 5368, 'num_object_clones': 0, 'num_object_copies': 10736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3826, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5368, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13838, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 442, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1542}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:31.053 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:31.053 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'15271", 'reported_seq': 15407, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.250956+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:27:28.250956+0000', 'last_peered': '2024-09-15T21:27:28.250956+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:27:28.250956+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21971427, 'num_objects': 5365, 'num_object_clones': 0, 'num_object_copies': 10730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5365, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11947, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1289}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:31.053 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:31.053 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'15373", 'reported_seq': 15514, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.204471+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:27:28.204471+0000', 'last_peered': '2024-09-15T21:27:28.204471+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:27:28.204471+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21697478, 'num_objects': 5299, 'num_object_clones': 0, 'num_object_copies': 10598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5299, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11684, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1138}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:31.053 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:27:31.053 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'15249", 'reported_seq': 15381, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:29.183349+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:27:29.183349+0000', 'last_peered': '2024-09-15T21:27:29.183349+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:27:29.183349+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21561827, 'num_objects': 5265, 'num_object_clones': 0, 'num_object_copies': 10530, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5265, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13181, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1178}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:31.053 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:31.054 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15184", 'reported_seq': 15320, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.223211+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:27:28.223211+0000', 'last_peered': '2024-09-15T21:27:28.223211+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:27:28.223211+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21700608, 'num_objects': 5298, 'num_object_clones': 0, 'num_object_copies': 10596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5298, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11857, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1278}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:31.054 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:27:31.054 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'15336", 'reported_seq': 15459, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:29.127810+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:27:29.127810+0000', 'last_peered': '2024-09-15T21:27:29.127810+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:27:29.127810+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21737955, 'num_objects': 5308, 'num_object_clones': 0, 'num_object_copies': 10616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5308, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13195, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1218}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:31.054 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:31.054 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'15365", 'reported_seq': 15504, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.230796+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:27:28.230796+0000', 'last_peered': '2024-09-15T21:27:28.230796+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:27:28.230796+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21794816, 'num_objects': 5321, 'num_object_clones': 0, 'num_object_copies': 10642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5321, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12309, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1227}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:31.054 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:27:31.054 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'15354", 'reported_seq': 15492, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.255403+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:27:28.255403+0000', 'last_peered': '2024-09-15T21:27:28.255403+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:27:28.255403+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21446656, 'num_objects': 5236, 'num_object_clones': 0, 'num_object_copies': 10472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5236, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12263, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1097}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:31.054 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:27:31.054 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'15508", 'reported_seq': 15652, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:28.196171+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:27:28.196171+0000', 'last_peered': '2024-09-15T21:27:28.196171+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:27:28.196171+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21947334, 'num_objects': 5360, 'num_object_clones': 0, 'num_object_copies': 10720, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5360, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12212, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1213}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:31.055 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:31.770 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:31.770 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40783/169900 objects degraded (24.004%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":503,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":84950,"data_bytes":347906825,"bytes_used":2014658560,"bytes_avail":287895633920,"bytes_total":289910292480,"degraded_objects":40783,"degraded_total":169900,"degraded_ratio":0.24004120070629786,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":276,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":476},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:31.788 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:31.788 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:32.400 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:32.402 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:32.423 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-15T21:27:30.812288+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":347906825,"num_objects":84950,"num_object_clones":0,"num_object_copies":169900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84950,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":205509,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8692,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160717,"ondisk_log_size":160717,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1967440,"kb_used_data":879764,"kb_used_omap":12,"kb_used_meta":1087603,"kb_avail":281148080,"statfs":{"total":289910292480,"available":287895633920,"internally_reserved":0,"allocated":900878336,"data_stored":899989576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1113705475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,8,7,12,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23408640,"num_objects":-5715,"num_object_clones":0,"num_object_copies":-11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-276,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5715,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004031"},"pg_stats":[{"pgid":"2.f","version":"47'15603","reported_seq":15684,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.200420+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:28.200420+0000","last_peered":"2024-09-15T21:27:28.200420+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:28.200420+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22728704,"num_objects":5549,"num_object_clones":0,"num_object_copies":11098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15603,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1417}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15324","reported_seq":15435,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.185654+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:29.185654+0000","last_peered":"2024-09-15T21:27:29.185654+0000","last_clean":"2024-09-15T21:27:29.185654+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:29.185654+0000","last_undegraded":"2024-09-15T21:27:29.185654+0000","last_fullsized":"2024-09-15T21:27:29.185654+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21914566,"num_objects":5352,"num_object_clones":0,"num_object_copies":10704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5352,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13605,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15738","reported_seq":16755,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:29.159142+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:29.159142+0000","last_peered":"2024-09-15T21:27:29.159142+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:29.159142+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21983256,"num_objects":5368,"num_object_clones":0,"num_object_copies":10736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3826,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5368,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13838,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":442,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1542}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15119","reported_seq":15195,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.149545+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:29.149545+0000","last_peered":"2024-09-15T21:27:29.149545+0000","last_clean":"2024-09-15T21:27:29.149545+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:29.149545+0000","last_undegraded":"2024-09-15T21:27:29.149545+0000","last_fullsized":"2024-09-15T21:27:29.149545+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21278720,"num_objects":5195,"num_object_clones":0,"num_object_copies":10390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15119,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15216","reported_seq":15354,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.267798+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:28.267798+0000","last_peered":"2024-09-15T21:27:28.267798+0000","last_clean":"2024-09-15T21:27:28.267798+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:28.267798+0000","last_undegraded":"2024-09-15T21:27:28.267798+0000","last_fullsized":"2024-09-15T21:27:28.267798+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21664227,"num_objects":5290,"num_object_clones":0,"num_object_copies":10580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11671,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15271","reported_seq":15407,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.250956+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:28.250956+0000","last_peered":"2024-09-15T21:27:28.250956+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:28.250956+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21971427,"num_objects":5365,"num_object_clones":0,"num_object_copies":10730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11947,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1289},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15373","reported_seq":15514,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.204471+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:28.204471+0000","last_peered":"2024-09-15T21:27:28.204471+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:28.204471+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21697478,"num_objects":5299,"num_object_clones":0,"num_object_copies":10598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11684,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1138},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15550","reported_seq":15684,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:28.230687+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:28.230687+0000","last_peered":"2024-09-15T21:27:28.230687+0000","last_clean":"2024-09-15T21:27:28.230687+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:28.230687+0000","last_undegraded":"2024-09-15T21:27:28.230687+0000","last_fullsized":"2024-09-15T21:27:28.230687+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21782528,"num_objects":5318,"num_object_clones":0,"num_object_copies":10636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12322,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15249","reported_seq":15381,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:29.183349+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:29.183349+0000","last_peered":"2024-09-15T21:27:29.183349+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:29.183349+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561827,"num_objects":5265,"num_object_clones":0,"num_object_copies":10530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5265,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13181,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1178}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15184","reported_seq":15320,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.223211+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:28.223211+0000","last_peered":"2024-09-15T21:27:28.223211+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:28.223211+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21700608,"num_objects":5298,"num_object_clones":0,"num_object_copies":10596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11857,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1278}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15336","reported_seq":15459,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:29.127810+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:29.127810+0000","last_peered":"2024-09-15T21:27:29.127810+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:29.127810+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21737955,"num_objects":5308,"num_object_clones":0,"num_object_copies":10616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5308,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13195,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1218}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15335","reported_seq":23798,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.158375+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:29.158375+0000","last_peered":"2024-09-15T21:27:29.158375+0000","last_clean":"2024-09-15T21:27:29.158375+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:29.158375+0000","last_undegraded":"2024-09-15T21:27:29.158375+0000","last_fullsized":"2024-09-15T21:27:29.158375+0000","mapping_epoch":44,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21401600,"num_objects":5225,"num_object_clones":0,"num_object_copies":10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13028,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15365","reported_seq":15504,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.230796+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:28.230796+0000","last_peered":"2024-09-15T21:27:28.230796+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:28.230796+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21794816,"num_objects":5321,"num_object_clones":0,"num_object_copies":10642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12309,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1227}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15354","reported_seq":15492,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.255403+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:28.255403+0000","last_peered":"2024-09-15T21:27:28.255403+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:28.255403+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21446656,"num_objects":5236,"num_object_clones":0,"num_object_copies":10472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5236,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12263,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1097}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15189","reported_seq":23504,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.321078+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:29.321078+0000","last_peered":"2024-09-15T21:27:29.321078+0000","last_clean":"2024-09-15T21:27:29.321078+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:29.321078+0000","last_undegraded":"2024-09-15T21:27:29.321078+0000","last_fullsized":"2024-09-15T21:27:29.321078+0000","mapping_epoch":46,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21295104,"num_objects":5199,"num_object_clones":0,"num_object_copies":10398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11673,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15508","reported_seq":15652,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:28.196171+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:28.196171+0000","last_peered":"2024-09-15T21:27:28.196171+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:28.196171+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21947334,"num_objects":5360,"num_object_clones":0,"num_object_copies":10720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5360,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12212,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1213},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":347906806,"num_objects":84948,"num_object_clones":0,"num_object_copies":169896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84948,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":205507,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8691,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":897593344,"data_stored":897447906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160714,"ondisk_log_size":160714,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622144,"kb_used_data":301680,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93749696,"statfs":{"total":96636764160,"available":95999688704,"internally_reserved":0,"allocated":308920320,"data_stored":308635062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":328134313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,6,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":733208,"kb_used_data":322560,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93638632,"statfs":{"total":96636764160,"available":95885959168,"internally_reserved":0,"allocated":330301440,"data_stored":330016177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612088,"kb_used_data":255524,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93759752,"statfs":{"total":96636764160,"available":96009986048,"internally_reserved":0,"allocated":261656576,"data_stored":261338337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":329621504,"data_stored":329585374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260349952,"data_stored":260280846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":307621888,"data_stored":307581686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:35.425 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:35.965 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:35.967 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:35.991 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-15T21:27:34.813834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":338494217,"num_objects":82652,"num_object_clones":0,"num_object_copies":165304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82652,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":207807,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8786,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1961312,"kb_used_data":861132,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281154208,"statfs":{"total":289910292480,"available":287901908992,"internally_reserved":0,"allocated":881799168,"data_stored":880910408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1126484995},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,3,4,2,12,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28561408,"num_objects":-6973,"num_object_clones":0,"num_object_copies":-13946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-272,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6973,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004144"},"pg_stats":[{"pgid":"2.f","version":"47'15722","reported_seq":15803,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.316960+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:33.316960+0000","last_peered":"2024-09-15T21:27:33.316960+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:33.316960+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22241280,"num_objects":5430,"num_object_clones":0,"num_object_copies":10860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15722,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15468","reported_seq":15579,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:34.173971+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:34.173971+0000","last_peered":"2024-09-15T21:27:34.173971+0000","last_clean":"2024-09-15T21:27:34.173971+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:34.173971+0000","last_undegraded":"2024-09-15T21:27:34.173971+0000","last_fullsized":"2024-09-15T21:27:34.173971+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21324742,"num_objects":5208,"num_object_clones":0,"num_object_copies":10416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5208,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13749,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15882","reported_seq":17087,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:34.190595+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:34.190595+0000","last_peered":"2024-09-15T21:27:34.190595+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:34.190595+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21393432,"num_objects":5224,"num_object_clones":0,"num_object_copies":10448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5224,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13982,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1492}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15255","reported_seq":15331,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:34.161485+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:34.161485+0000","last_peered":"2024-09-15T21:27:34.161485+0000","last_clean":"2024-09-15T21:27:34.161485+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:34.161485+0000","last_undegraded":"2024-09-15T21:27:34.161485+0000","last_fullsized":"2024-09-15T21:27:34.161485+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20721664,"num_objects":5059,"num_object_clones":0,"num_object_copies":10118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15255,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15367","reported_seq":15505,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:33.309841+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:33.309841+0000","last_peered":"2024-09-15T21:27:33.309841+0000","last_clean":"2024-09-15T21:27:33.309841+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:33.309841+0000","last_undegraded":"2024-09-15T21:27:33.309841+0000","last_fullsized":"2024-09-15T21:27:33.309841+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21045731,"num_objects":5139,"num_object_clones":0,"num_object_copies":10278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11822,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15421","reported_seq":15560,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.287098+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:33.287098+0000","last_peered":"2024-09-15T21:27:33.287098+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:33.287098+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21357027,"num_objects":5215,"num_object_clones":0,"num_object_copies":10430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12097,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1139},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15527","reported_seq":15668,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.287322+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:33.287322+0000","last_peered":"2024-09-15T21:27:33.287322+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:33.287322+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21066694,"num_objects":5145,"num_object_clones":0,"num_object_copies":10290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11838,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":984},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15701","reported_seq":15835,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:33.115850+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:33.115850+0000","last_peered":"2024-09-15T21:27:33.115850+0000","last_clean":"2024-09-15T21:27:33.115850+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:33.115850+0000","last_undegraded":"2024-09-15T21:27:33.115850+0000","last_fullsized":"2024-09-15T21:27:33.115850+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21164032,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12473,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15393","reported_seq":15525,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:34.157822+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:34.157822+0000","last_peered":"2024-09-15T21:27:34.157822+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:34.157822+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972003,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13325,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1034}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15333","reported_seq":15469,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.235713+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:33.235713+0000","last_peered":"2024-09-15T21:27:33.235713+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:33.235713+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21090304,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12006,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1129}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15484","reported_seq":15608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.944553+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:33.944553+0000","last_peered":"2024-09-15T21:27:33.944553+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:33.944553+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21131747,"num_objects":5160,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13343,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1070}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15479","reported_seq":23943,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:34.061836+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:34.061836+0000","last_peered":"2024-09-15T21:27:34.061836+0000","last_clean":"2024-09-15T21:27:34.061836+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:34.061836+0000","last_undegraded":"2024-09-15T21:27:34.061836+0000","last_fullsized":"2024-09-15T21:27:34.061836+0000","mapping_epoch":44,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20811776,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13172,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15506","reported_seq":15645,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.263871+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:33.263871+0000","last_peered":"2024-09-15T21:27:33.263871+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:33.263871+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21217280,"num_objects":5180,"num_object_clones":0,"num_object_copies":10360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12450,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15495","reported_seq":15633,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.344900+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:33.344900+0000","last_peered":"2024-09-15T21:27:33.344900+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:33.344900+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20869120,"num_objects":5095,"num_object_clones":0,"num_object_copies":10190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12404,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":956}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15331","reported_seq":23646,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:34.393923+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:34.393923+0000","last_peered":"2024-09-15T21:27:34.393923+0000","last_clean":"2024-09-15T21:27:34.393923+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:34.393923+0000","last_undegraded":"2024-09-15T21:27:34.393923+0000","last_fullsized":"2024-09-15T21:27:34.393923+0000","mapping_epoch":46,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20713472,"num_objects":5057,"num_object_clones":0,"num_object_copies":10114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11815,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15648","reported_seq":15792,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.291573+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:33.291573+0000","last_peered":"2024-09-15T21:27:33.291573+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:33.291573+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21373894,"num_objects":5220,"num_object_clones":0,"num_object_copies":10440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5220,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12352,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1073},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":338494198,"num_objects":82650,"num_object_clones":0,"num_object_copies":165300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82650,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":207805,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8785,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":878612480,"data_stored":878467042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621644,"kb_used_data":297012,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93750196,"statfs":{"total":96636764160,"available":96000200704,"internally_reserved":0,"allocated":304140288,"data_stored":303855030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":332394153},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730508,"kb_used_data":315684,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93641332,"statfs":{"total":96636764160,"available":95888723968,"internally_reserved":0,"allocated":323260416,"data_stored":322975153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,0,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659213,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609160,"kb_used_data":248436,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93762680,"statfs":{"total":96636764160,"available":96012984320,"internally_reserved":0,"allocated":254398464,"data_stored":254080225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322539520,"data_stored":322503390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253186048,"data_stored":253116942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302886912,"data_stored":302846710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:35.993 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:36.609 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:36.609 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:36.643 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-15T21:27:34.813834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":338494217,"num_objects":82652,"num_object_clones":0,"num_object_copies":165304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82652,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":207807,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8786,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1961312,"kb_used_data":861132,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281154208,"statfs":{"total":289910292480,"available":287901908992,"internally_reserved":0,"allocated":881799168,"data_stored":880910408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1126484995},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,3,4,2,12,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28561408,"num_objects":-6973,"num_object_clones":0,"num_object_copies":-13946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-272,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6973,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004144"},"pg_stats":[{"pgid":"2.f","version":"47'15722","reported_seq":15803,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.316960+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:33.316960+0000","last_peered":"2024-09-15T21:27:33.316960+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:33.316960+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22241280,"num_objects":5430,"num_object_clones":0,"num_object_copies":10860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15722,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15468","reported_seq":15579,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:34.173971+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:34.173971+0000","last_peered":"2024-09-15T21:27:34.173971+0000","last_clean":"2024-09-15T21:27:34.173971+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:34.173971+0000","last_undegraded":"2024-09-15T21:27:34.173971+0000","last_fullsized":"2024-09-15T21:27:34.173971+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21324742,"num_objects":5208,"num_object_clones":0,"num_object_copies":10416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5208,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13749,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15882","reported_seq":17087,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:34.190595+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:34.190595+0000","last_peered":"2024-09-15T21:27:34.190595+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:34.190595+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21393432,"num_objects":5224,"num_object_clones":0,"num_object_copies":10448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5224,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13982,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1492}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15255","reported_seq":15331,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:34.161485+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:34.161485+0000","last_peered":"2024-09-15T21:27:34.161485+0000","last_clean":"2024-09-15T21:27:34.161485+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:34.161485+0000","last_undegraded":"2024-09-15T21:27:34.161485+0000","last_fullsized":"2024-09-15T21:27:34.161485+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20721664,"num_objects":5059,"num_object_clones":0,"num_object_copies":10118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15255,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15367","reported_seq":15505,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:33.309841+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:33.309841+0000","last_peered":"2024-09-15T21:27:33.309841+0000","last_clean":"2024-09-15T21:27:33.309841+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:33.309841+0000","last_undegraded":"2024-09-15T21:27:33.309841+0000","last_fullsized":"2024-09-15T21:27:33.309841+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21045731,"num_objects":5139,"num_object_clones":0,"num_object_copies":10278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11822,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15421","reported_seq":15560,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.287098+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:33.287098+0000","last_peered":"2024-09-15T21:27:33.287098+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:33.287098+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21357027,"num_objects":5215,"num_object_clones":0,"num_object_copies":10430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12097,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1139},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15527","reported_seq":15668,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.287322+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:33.287322+0000","last_peered":"2024-09-15T21:27:33.287322+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:33.287322+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21066694,"num_objects":5145,"num_object_clones":0,"num_object_copies":10290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11838,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":984},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15701","reported_seq":15835,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:33.115850+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:33.115850+0000","last_peered":"2024-09-15T21:27:33.115850+0000","last_clean":"2024-09-15T21:27:33.115850+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:33.115850+0000","last_undegraded":"2024-09-15T21:27:33.115850+0000","last_fullsized":"2024-09-15T21:27:33.115850+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21164032,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12473,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15393","reported_seq":15525,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:34.157822+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:34.157822+0000","last_peered":"2024-09-15T21:27:34.157822+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:34.157822+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972003,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13325,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1034}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15333","reported_seq":15469,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.235713+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:33.235713+0000","last_peered":"2024-09-15T21:27:33.235713+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:33.235713+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21090304,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12006,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1129}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15484","reported_seq":15608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.944553+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:33.944553+0000","last_peered":"2024-09-15T21:27:33.944553+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:33.944553+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21131747,"num_objects":5160,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13343,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1070}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15479","reported_seq":23943,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:34.061836+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:34.061836+0000","last_peered":"2024-09-15T21:27:34.061836+0000","last_clean":"2024-09-15T21:27:34.061836+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:34.061836+0000","last_undegraded":"2024-09-15T21:27:34.061836+0000","last_fullsized":"2024-09-15T21:27:34.061836+0000","mapping_epoch":44,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20811776,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13172,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15506","reported_seq":15645,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.263871+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:33.263871+0000","last_peered":"2024-09-15T21:27:33.263871+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:33.263871+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21217280,"num_objects":5180,"num_object_clones":0,"num_object_copies":10360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12450,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15495","reported_seq":15633,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.344900+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:33.344900+0000","last_peered":"2024-09-15T21:27:33.344900+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:33.344900+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20869120,"num_objects":5095,"num_object_clones":0,"num_object_copies":10190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12404,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":956}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15331","reported_seq":23646,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:34.393923+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:34.393923+0000","last_peered":"2024-09-15T21:27:34.393923+0000","last_clean":"2024-09-15T21:27:34.393923+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:34.393923+0000","last_undegraded":"2024-09-15T21:27:34.393923+0000","last_fullsized":"2024-09-15T21:27:34.393923+0000","mapping_epoch":46,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20713472,"num_objects":5057,"num_object_clones":0,"num_object_copies":10114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11815,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15648","reported_seq":15792,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.291573+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:33.291573+0000","last_peered":"2024-09-15T21:27:33.291573+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:33.291573+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21373894,"num_objects":5220,"num_object_clones":0,"num_object_copies":10440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5220,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12352,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1073},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":338494198,"num_objects":82650,"num_object_clones":0,"num_object_copies":165300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82650,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":207805,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8785,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":878612480,"data_stored":878467042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621644,"kb_used_data":297012,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93750196,"statfs":{"total":96636764160,"available":96000200704,"internally_reserved":0,"allocated":304140288,"data_stored":303855030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":332394153},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730508,"kb_used_data":315684,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93641332,"statfs":{"total":96636764160,"available":95888723968,"internally_reserved":0,"allocated":323260416,"data_stored":322975153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,0,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659213,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609160,"kb_used_data":248436,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93762680,"statfs":{"total":96636764160,"available":96012984320,"internally_reserved":0,"allocated":254398464,"data_stored":254080225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322539520,"data_stored":322503390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253186048,"data_stored":253116942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302886912,"data_stored":302846710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:36.644 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:36.644 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'15722", 'reported_seq': 15803, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:33.316960+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:27:33.316960+0000', 'last_peered': '2024-09-15T21:27:33.316960+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:27:33.316960+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22241280, 'num_objects': 5430, 'num_object_clones': 0, 'num_object_copies': 10860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5430, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15722, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1298}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:36.645 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:27:36.645 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'15882", 'reported_seq': 17087, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:34.190595+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:27:34.190595+0000', 'last_peered': '2024-09-15T21:27:34.190595+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:27:34.190595+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21393432, 'num_objects': 5224, 'num_object_clones': 0, 'num_object_copies': 10448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3732, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5224, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13982, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 536, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1492}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:36.645 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:36.645 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'15421", 'reported_seq': 15560, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:33.287098+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:27:33.287098+0000', 'last_peered': '2024-09-15T21:27:33.287098+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:27:33.287098+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21357027, 'num_objects': 5215, 'num_object_clones': 0, 'num_object_copies': 10430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5215, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12097, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1139}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:36.645 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:36.645 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'15527", 'reported_seq': 15668, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:33.287322+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:27:33.287322+0000', 'last_peered': '2024-09-15T21:27:33.287322+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:27:33.287322+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21066694, 'num_objects': 5145, 'num_object_clones': 0, 'num_object_copies': 10290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5145, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11838, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 984}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:36.645 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:27:36.645 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'15393", 'reported_seq': 15525, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:34.157822+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:27:34.157822+0000', 'last_peered': '2024-09-15T21:27:34.157822+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:27:34.157822+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20972003, 'num_objects': 5121, 'num_object_clones': 0, 'num_object_copies': 10242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5121, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13325, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1034}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:36.645 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:36.645 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15333", 'reported_seq': 15469, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:33.235713+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:27:33.235713+0000', 'last_peered': '2024-09-15T21:27:33.235713+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:27:33.235713+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21090304, 'num_objects': 5149, 'num_object_clones': 0, 'num_object_copies': 10298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5149, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12006, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1129}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:36.646 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:27:36.646 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'15484", 'reported_seq': 15608, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:33.944553+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:27:33.944553+0000', 'last_peered': '2024-09-15T21:27:33.944553+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:27:33.944553+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21131747, 'num_objects': 5160, 'num_object_clones': 0, 'num_object_copies': 10320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5160, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13343, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 1070}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:36.646 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:36.646 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'15506", 'reported_seq': 15645, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:33.263871+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:27:33.263871+0000', 'last_peered': '2024-09-15T21:27:33.263871+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:27:33.263871+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21217280, 'num_objects': 5180, 'num_object_clones': 0, 'num_object_copies': 10360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5180, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12450, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1086}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:36.646 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:27:36.646 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'15495", 'reported_seq': 15633, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:33.344900+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:27:33.344900+0000', 'last_peered': '2024-09-15T21:27:33.344900+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:27:33.344900+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20869120, 'num_objects': 5095, 'num_object_clones': 0, 'num_object_copies': 10190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5095, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12404, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 956}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:36.646 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:27:36.646 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'15648", 'reported_seq': 15792, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:33.291573+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:27:33.291573+0000', 'last_peered': '2024-09-15T21:27:33.291573+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:27:33.291573+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21373894, 'num_objects': 5220, 'num_object_clones': 0, 'num_object_copies': 10440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5220, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12352, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1073}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:36.647 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:37.360 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40689/165304 objects degraded (24.615%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":509,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":82652,"data_bytes":338494217,"bytes_used":2008383488,"bytes_avail":287901908992,"bytes_total":289910292480,"degraded_objects":40689,"degraded_total":165304,"degraded_ratio":0.24614649373275901,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":272,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":580},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:37.389 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:37.390 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:37.969 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:37.997 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-15T21:27:36.814592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":338494217,"num_objects":82652,"num_object_clones":0,"num_object_copies":165304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82652,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":207807,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8786,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1961312,"kb_used_data":861132,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281154208,"statfs":{"total":289910292480,"available":287901908992,"internally_reserved":0,"allocated":881799168,"data_stored":880910408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1126484995},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,3,4,2,12,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19050496,"num_objects":-4651,"num_object_clones":0,"num_object_copies":-9302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-178,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4651,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004082"},"pg_stats":[{"pgid":"2.f","version":"47'15722","reported_seq":15803,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.316960+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:33.316960+0000","last_peered":"2024-09-15T21:27:33.316960+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:33.316960+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22241280,"num_objects":5430,"num_object_clones":0,"num_object_copies":10860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15722,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1298}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15468","reported_seq":15579,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:34.173971+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:34.173971+0000","last_peered":"2024-09-15T21:27:34.173971+0000","last_clean":"2024-09-15T21:27:34.173971+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:34.173971+0000","last_undegraded":"2024-09-15T21:27:34.173971+0000","last_fullsized":"2024-09-15T21:27:34.173971+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21324742,"num_objects":5208,"num_object_clones":0,"num_object_copies":10416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5208,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13749,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15882","reported_seq":17087,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:34.190595+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:34.190595+0000","last_peered":"2024-09-15T21:27:34.190595+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:34.190595+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21393432,"num_objects":5224,"num_object_clones":0,"num_object_copies":10448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5224,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13982,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":536,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1492}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15255","reported_seq":15331,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:34.161485+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:34.161485+0000","last_peered":"2024-09-15T21:27:34.161485+0000","last_clean":"2024-09-15T21:27:34.161485+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:34.161485+0000","last_undegraded":"2024-09-15T21:27:34.161485+0000","last_fullsized":"2024-09-15T21:27:34.161485+0000","mapping_epoch":35,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20721664,"num_objects":5059,"num_object_clones":0,"num_object_copies":10118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15255,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15367","reported_seq":15505,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:33.309841+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:33.309841+0000","last_peered":"2024-09-15T21:27:33.309841+0000","last_clean":"2024-09-15T21:27:33.309841+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:33.309841+0000","last_undegraded":"2024-09-15T21:27:33.309841+0000","last_fullsized":"2024-09-15T21:27:33.309841+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21045731,"num_objects":5139,"num_object_clones":0,"num_object_copies":10278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11822,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15421","reported_seq":15560,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.287098+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:33.287098+0000","last_peered":"2024-09-15T21:27:33.287098+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:33.287098+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21357027,"num_objects":5215,"num_object_clones":0,"num_object_copies":10430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5215,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12097,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1139},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15527","reported_seq":15668,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.287322+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:33.287322+0000","last_peered":"2024-09-15T21:27:33.287322+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:33.287322+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21066694,"num_objects":5145,"num_object_clones":0,"num_object_copies":10290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11838,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":984},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15701","reported_seq":15835,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:33.115850+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:33.115850+0000","last_peered":"2024-09-15T21:27:33.115850+0000","last_clean":"2024-09-15T21:27:33.115850+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:33.115850+0000","last_undegraded":"2024-09-15T21:27:33.115850+0000","last_fullsized":"2024-09-15T21:27:33.115850+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21164032,"num_objects":5167,"num_object_clones":0,"num_object_copies":10334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12473,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15393","reported_seq":15525,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:34.157822+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:34.157822+0000","last_peered":"2024-09-15T21:27:34.157822+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:34.157822+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20972003,"num_objects":5121,"num_object_clones":0,"num_object_copies":10242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13325,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1034}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15333","reported_seq":15469,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.235713+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:33.235713+0000","last_peered":"2024-09-15T21:27:33.235713+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:33.235713+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21090304,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12006,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1129}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15484","reported_seq":15608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.944553+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:33.944553+0000","last_peered":"2024-09-15T21:27:33.944553+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:33.944553+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21131747,"num_objects":5160,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13343,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":1070}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15479","reported_seq":23943,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:34.061836+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:34.061836+0000","last_peered":"2024-09-15T21:27:34.061836+0000","last_clean":"2024-09-15T21:27:34.061836+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:34.061836+0000","last_undegraded":"2024-09-15T21:27:34.061836+0000","last_fullsized":"2024-09-15T21:27:34.061836+0000","mapping_epoch":44,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20811776,"num_objects":5081,"num_object_clones":0,"num_object_copies":10162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13172,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15506","reported_seq":15645,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.263871+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:33.263871+0000","last_peered":"2024-09-15T21:27:33.263871+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:33.263871+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21217280,"num_objects":5180,"num_object_clones":0,"num_object_copies":10360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12450,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15495","reported_seq":15633,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.344900+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:33.344900+0000","last_peered":"2024-09-15T21:27:33.344900+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:33.344900+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20869120,"num_objects":5095,"num_object_clones":0,"num_object_copies":10190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12404,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":956}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15331","reported_seq":23646,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:34.393923+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:34.393923+0000","last_peered":"2024-09-15T21:27:34.393923+0000","last_clean":"2024-09-15T21:27:34.393923+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:34.393923+0000","last_undegraded":"2024-09-15T21:27:34.393923+0000","last_fullsized":"2024-09-15T21:27:34.393923+0000","mapping_epoch":46,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20713472,"num_objects":5057,"num_object_clones":0,"num_object_copies":10114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11815,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15648","reported_seq":15792,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:33.291573+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:33.291573+0000","last_peered":"2024-09-15T21:27:33.291573+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:33.291573+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21373894,"num_objects":5220,"num_object_clones":0,"num_object_copies":10440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5220,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12352,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1073},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":338494198,"num_objects":82650,"num_object_clones":0,"num_object_copies":165300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40689,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82650,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":207805,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8785,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":878612480,"data_stored":878467042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621644,"kb_used_data":297012,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93750196,"statfs":{"total":96636764160,"available":96000200704,"internally_reserved":0,"allocated":304140288,"data_stored":303855030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":332394153},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,0,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":730508,"kb_used_data":315684,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93641332,"statfs":{"total":96636764160,"available":95888723968,"internally_reserved":0,"allocated":323260416,"data_stored":322975153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,0,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659213,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609160,"kb_used_data":248436,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93762680,"statfs":{"total":96636764160,"available":96012984320,"internally_reserved":0,"allocated":254398464,"data_stored":254080225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369356463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":322539520,"data_stored":322503390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253186048,"data_stored":253116942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":302886912,"data_stored":302846710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:40.999 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:41.559 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:41.560 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:41.583 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-15T21:27:40.816170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":329388809,"num_objects":80429,"num_object_clones":0,"num_object_copies":160858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80429,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":210030,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8877,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1943744,"kb_used_data":843584,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281171776,"statfs":{"total":289910292480,"available":287919898624,"internally_reserved":0,"allocated":863830016,"data_stored":862941256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1126484995},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,3,8,12,6,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22773760,"num_objects":-5560,"num_object_clones":0,"num_object_copies":-11120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5560,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004491"},"pg_stats":[{"pgid":"2.f","version":"47'15869","reported_seq":15949,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.358247+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:38.358247+0000","last_peered":"2024-09-15T21:27:38.358247+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:38.358247+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21639168,"num_objects":5283,"num_object_clones":0,"num_object_copies":10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15869,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1151}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15612","reported_seq":15722,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:39.078790+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:39.078790+0000","last_peered":"2024-09-15T21:27:39.078790+0000","last_clean":"2024-09-15T21:27:39.078790+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:39.078790+0000","last_undegraded":"2024-09-15T21:27:39.078790+0000","last_fullsized":"2024-09-15T21:27:39.078790+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20734918,"num_objects":5064,"num_object_clones":0,"num_object_copies":10128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5064,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13893,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16023","reported_seq":17410,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:39.163105+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:39.163105+0000","last_peered":"2024-09-15T21:27:39.163105+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:39.163105+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20815896,"num_objects":5083,"num_object_clones":0,"num_object_copies":10166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3641,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5083,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14123,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1442}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15418","reported_seq":15494,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:38.976959+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:38.976959+0000","last_peered":"2024-09-15T21:27:38.976959+0000","last_clean":"2024-09-15T21:27:38.976959+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:38.976959+0000","last_undegraded":"2024-09-15T21:27:38.976959+0000","last_fullsized":"2024-09-15T21:27:38.976959+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20054016,"num_objects":4896,"num_object_clones":0,"num_object_copies":9792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15418,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15500","reported_seq":15638,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:38.362242+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:38.362242+0000","last_peered":"2024-09-15T21:27:38.362242+0000","last_clean":"2024-09-15T21:27:38.362242+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:38.362242+0000","last_undegraded":"2024-09-15T21:27:38.362242+0000","last_fullsized":"2024-09-15T21:27:38.362242+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20500963,"num_objects":5006,"num_object_clones":0,"num_object_copies":10012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11955,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15569","reported_seq":15711,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.323246+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:38.323246+0000","last_peered":"2024-09-15T21:27:38.323246+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:38.323246+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20750819,"num_objects":5067,"num_object_clones":0,"num_object_copies":10134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5067,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12245,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":991},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15655","reported_seq":15796,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.399091+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:38.399091+0000","last_peered":"2024-09-15T21:27:38.399091+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:38.399091+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20542406,"num_objects":5017,"num_object_clones":0,"num_object_copies":10034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11966,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":856},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15845","reported_seq":15979,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:38.339542+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:38.339542+0000","last_peered":"2024-09-15T21:27:38.339542+0000","last_clean":"2024-09-15T21:27:38.339542+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:38.339542+0000","last_undegraded":"2024-09-15T21:27:38.339542+0000","last_fullsized":"2024-09-15T21:27:38.339542+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20574208,"num_objects":5023,"num_object_clones":0,"num_object_copies":10046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12617,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15523","reported_seq":15655,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:39.183998+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:39.183998+0000","last_peered":"2024-09-15T21:27:39.183998+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:39.183998+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20439523,"num_objects":4991,"num_object_clones":0,"num_object_copies":9982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13455,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":904}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15480","reported_seq":15613,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.393754+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:38.393754+0000","last_peered":"2024-09-15T21:27:38.393754+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:38.393754+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20488192,"num_objects":5002,"num_object_clones":0,"num_object_copies":10004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12153,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":982}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15622","reported_seq":15746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:39.044326+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:39.044326+0000","last_peered":"2024-09-15T21:27:39.044326+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:39.044326+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20566499,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13481,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":932}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15616","reported_seq":24080,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:39.177019+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:39.177019+0000","last_peered":"2024-09-15T21:27:39.177019+0000","last_clean":"2024-09-15T21:27:39.177019+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:39.177019+0000","last_undegraded":"2024-09-15T21:27:39.177019+0000","last_fullsized":"2024-09-15T21:27:39.177019+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":4944,"num_object_clones":0,"num_object_copies":9888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13309,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15634","reported_seq":15766,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.405562+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:38.405562+0000","last_peered":"2024-09-15T21:27:38.405562+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:38.405562+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20692992,"num_objects":5052,"num_object_clones":0,"num_object_copies":10104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12578,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":958}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15611","reported_seq":15748,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.372909+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:38.372909+0000","last_peered":"2024-09-15T21:27:38.372909+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:38.372909+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20393984,"num_objects":4979,"num_object_clones":0,"num_object_copies":9958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12520,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15461","reported_seq":23776,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:39.261086+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:39.261086+0000","last_peered":"2024-09-15T21:27:39.261086+0000","last_clean":"2024-09-15T21:27:39.261086+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:39.261086+0000","last_undegraded":"2024-09-15T21:27:39.261086+0000","last_fullsized":"2024-09-15T21:27:39.261086+0000","mapping_epoch":46,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20180992,"num_objects":4927,"num_object_clones":0,"num_object_copies":9854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11945,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15797","reported_seq":15941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.377476+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:38.377476+0000","last_peered":"2024-09-15T21:27:38.377476+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:38.377476+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20763590,"num_objects":5071,"num_object_clones":0,"num_object_copies":10142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12501,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":924},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":329388790,"num_objects":80427,"num_object_clones":0,"num_object_copies":160854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80427,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":210028,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8876,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":860323840,"data_stored":860178402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621356,"kb_used_data":292568,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93750484,"statfs":{"total":96636764160,"available":96000495616,"internally_reserved":0,"allocated":299589632,"data_stored":299304374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":336653993},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,2,2,2,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728004,"kb_used_data":309036,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93643836,"statfs":{"total":96636764160,"available":95891288064,"internally_reserved":0,"allocated":316452864,"data_stored":316167601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594384,"kb_used_data":241980,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93777456,"statfs":{"total":96636764160,"available":96028114944,"internally_reserved":0,"allocated":247787520,"data_stored":247469281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315564032,"data_stored":315527902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":246468608,"data_stored":246399502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298291200,"data_stored":298250998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:41.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:42.124 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:42.127 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:42.149 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":296,"stamp":"2024-09-15T21:27:40.816170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":329388809,"num_objects":80429,"num_object_clones":0,"num_object_copies":160858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80429,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":210030,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8877,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1943744,"kb_used_data":843584,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281171776,"statfs":{"total":289910292480,"available":287919898624,"internally_reserved":0,"allocated":863830016,"data_stored":862941256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1126484995},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,3,8,12,6,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22773760,"num_objects":-5560,"num_object_clones":0,"num_object_copies":-11120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5560,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004491"},"pg_stats":[{"pgid":"2.f","version":"47'15869","reported_seq":15949,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.358247+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:38.358247+0000","last_peered":"2024-09-15T21:27:38.358247+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:38.358247+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21639168,"num_objects":5283,"num_object_clones":0,"num_object_copies":10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15869,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1151}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15612","reported_seq":15722,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:39.078790+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:39.078790+0000","last_peered":"2024-09-15T21:27:39.078790+0000","last_clean":"2024-09-15T21:27:39.078790+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:39.078790+0000","last_undegraded":"2024-09-15T21:27:39.078790+0000","last_fullsized":"2024-09-15T21:27:39.078790+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20734918,"num_objects":5064,"num_object_clones":0,"num_object_copies":10128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5064,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13893,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16023","reported_seq":17410,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:39.163105+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:39.163105+0000","last_peered":"2024-09-15T21:27:39.163105+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:39.163105+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20815896,"num_objects":5083,"num_object_clones":0,"num_object_copies":10166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3641,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5083,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14123,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1442}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15418","reported_seq":15494,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:38.976959+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:38.976959+0000","last_peered":"2024-09-15T21:27:38.976959+0000","last_clean":"2024-09-15T21:27:38.976959+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:38.976959+0000","last_undegraded":"2024-09-15T21:27:38.976959+0000","last_fullsized":"2024-09-15T21:27:38.976959+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20054016,"num_objects":4896,"num_object_clones":0,"num_object_copies":9792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15418,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15500","reported_seq":15638,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:38.362242+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:38.362242+0000","last_peered":"2024-09-15T21:27:38.362242+0000","last_clean":"2024-09-15T21:27:38.362242+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:38.362242+0000","last_undegraded":"2024-09-15T21:27:38.362242+0000","last_fullsized":"2024-09-15T21:27:38.362242+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20500963,"num_objects":5006,"num_object_clones":0,"num_object_copies":10012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11955,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15569","reported_seq":15711,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.323246+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:38.323246+0000","last_peered":"2024-09-15T21:27:38.323246+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:38.323246+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20750819,"num_objects":5067,"num_object_clones":0,"num_object_copies":10134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5067,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12245,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":991},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15655","reported_seq":15796,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.399091+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:38.399091+0000","last_peered":"2024-09-15T21:27:38.399091+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:38.399091+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20542406,"num_objects":5017,"num_object_clones":0,"num_object_copies":10034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11966,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":856},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15845","reported_seq":15979,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:38.339542+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:38.339542+0000","last_peered":"2024-09-15T21:27:38.339542+0000","last_clean":"2024-09-15T21:27:38.339542+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:38.339542+0000","last_undegraded":"2024-09-15T21:27:38.339542+0000","last_fullsized":"2024-09-15T21:27:38.339542+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20574208,"num_objects":5023,"num_object_clones":0,"num_object_copies":10046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12617,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15523","reported_seq":15655,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:39.183998+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:39.183998+0000","last_peered":"2024-09-15T21:27:39.183998+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:39.183998+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20439523,"num_objects":4991,"num_object_clones":0,"num_object_copies":9982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13455,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":904}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15480","reported_seq":15613,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.393754+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:38.393754+0000","last_peered":"2024-09-15T21:27:38.393754+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:38.393754+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20488192,"num_objects":5002,"num_object_clones":0,"num_object_copies":10004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12153,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":982}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15622","reported_seq":15746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:39.044326+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:39.044326+0000","last_peered":"2024-09-15T21:27:39.044326+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:39.044326+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20566499,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13481,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":932}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15616","reported_seq":24080,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:39.177019+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:39.177019+0000","last_peered":"2024-09-15T21:27:39.177019+0000","last_clean":"2024-09-15T21:27:39.177019+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:39.177019+0000","last_undegraded":"2024-09-15T21:27:39.177019+0000","last_fullsized":"2024-09-15T21:27:39.177019+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":4944,"num_object_clones":0,"num_object_copies":9888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13309,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15634","reported_seq":15766,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.405562+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:38.405562+0000","last_peered":"2024-09-15T21:27:38.405562+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:38.405562+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20692992,"num_objects":5052,"num_object_clones":0,"num_object_copies":10104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12578,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":958}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15611","reported_seq":15748,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.372909+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:38.372909+0000","last_peered":"2024-09-15T21:27:38.372909+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:38.372909+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20393984,"num_objects":4979,"num_object_clones":0,"num_object_copies":9958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12520,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15461","reported_seq":23776,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:39.261086+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:39.261086+0000","last_peered":"2024-09-15T21:27:39.261086+0000","last_clean":"2024-09-15T21:27:39.261086+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:39.261086+0000","last_undegraded":"2024-09-15T21:27:39.261086+0000","last_fullsized":"2024-09-15T21:27:39.261086+0000","mapping_epoch":46,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20180992,"num_objects":4927,"num_object_clones":0,"num_object_copies":9854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11945,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15797","reported_seq":15941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.377476+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:38.377476+0000","last_peered":"2024-09-15T21:27:38.377476+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:38.377476+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20763590,"num_objects":5071,"num_object_clones":0,"num_object_copies":10142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12501,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":924},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":329388790,"num_objects":80427,"num_object_clones":0,"num_object_copies":160854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80427,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":210028,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8876,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":860323840,"data_stored":860178402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621356,"kb_used_data":292568,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93750484,"statfs":{"total":96636764160,"available":96000495616,"internally_reserved":0,"allocated":299589632,"data_stored":299304374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":336653993},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,2,2,2,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728004,"kb_used_data":309036,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93643836,"statfs":{"total":96636764160,"available":95891288064,"internally_reserved":0,"allocated":316452864,"data_stored":316167601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594384,"kb_used_data":241980,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93777456,"statfs":{"total":96636764160,"available":96028114944,"internally_reserved":0,"allocated":247787520,"data_stored":247469281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315564032,"data_stored":315527902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":246468608,"data_stored":246399502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298291200,"data_stored":298250998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:42.151 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:42.151 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'15869", 'reported_seq': 15949, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:38.358247+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:27:38.358247+0000', 'last_peered': '2024-09-15T21:27:38.358247+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:27:38.358247+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21639168, 'num_objects': 5283, 'num_object_clones': 0, 'num_object_copies': 10566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5283, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15869, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1151}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:42.151 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:27:42.151 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'16023", 'reported_seq': 17410, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:39.163105+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:27:39.163105+0000', 'last_peered': '2024-09-15T21:27:39.163105+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:27:39.163105+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20815896, 'num_objects': 5083, 'num_object_clones': 0, 'num_object_copies': 10166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3641, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5083, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14123, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 627, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1442}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:42.151 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:42.151 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'15569", 'reported_seq': 15711, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:38.323246+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:27:38.323246+0000', 'last_peered': '2024-09-15T21:27:38.323246+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:27:38.323246+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20750819, 'num_objects': 5067, 'num_object_clones': 0, 'num_object_copies': 10134, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5067, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12245, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 991}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:42.151 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:42.151 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'15655", 'reported_seq': 15796, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:38.399091+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:27:38.399091+0000', 'last_peered': '2024-09-15T21:27:38.399091+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:27:38.399091+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20542406, 'num_objects': 5017, 'num_object_clones': 0, 'num_object_copies': 10034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5017, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11966, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 856}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:42.151 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:27:42.151 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'15523", 'reported_seq': 15655, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:39.183998+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:27:39.183998+0000', 'last_peered': '2024-09-15T21:27:39.183998+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:27:39.183998+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20439523, 'num_objects': 4991, 'num_object_clones': 0, 'num_object_copies': 9982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4991, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13455, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 904}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:42.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:42.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15480", 'reported_seq': 15613, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:38.393754+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:27:38.393754+0000', 'last_peered': '2024-09-15T21:27:38.393754+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:27:38.393754+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20488192, 'num_objects': 5002, 'num_object_clones': 0, 'num_object_copies': 10004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5002, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12153, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 982}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:42.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:27:42.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'15622", 'reported_seq': 15746, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:39.044326+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:27:39.044326+0000', 'last_peered': '2024-09-15T21:27:39.044326+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:27:39.044326+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20566499, 'num_objects': 5022, 'num_object_clones': 0, 'num_object_copies': 10044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5022, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13481, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 932}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:42.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:42.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'15634", 'reported_seq': 15766, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:38.405562+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:27:38.405562+0000', 'last_peered': '2024-09-15T21:27:38.405562+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:27:38.405562+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20692992, 'num_objects': 5052, 'num_object_clones': 0, 'num_object_copies': 10104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5052, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12578, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 958}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:42.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:27:42.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'15611", 'reported_seq': 15748, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:38.372909+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:27:38.372909+0000', 'last_peered': '2024-09-15T21:27:38.372909+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:27:38.372909+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20393984, 'num_objects': 4979, 'num_object_clones': 0, 'num_object_copies': 9958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4979, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12520, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 840}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:42.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:27:42.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'15797", 'reported_seq': 15941, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:38.377476+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:27:38.377476+0000', 'last_peered': '2024-09-15T21:27:38.377476+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:27:38.377476+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20763590, 'num_objects': 5071, 'num_object_clones': 0, 'num_object_copies': 10142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5071, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12501, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 924}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:42.153 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:42.851 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:42.851 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40598/160858 objects degraded (25.238%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":514,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":80429,"data_bytes":329388809,"bytes_used":1990393856,"bytes_avail":287919898624,"bytes_total":289910292480,"degraded_objects":40598,"degraded_total":160858,"degraded_ratio":0.25238409031568215,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":269,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":463},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:42.879 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:42.879 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:43.452 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:43.453 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:43.481 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-15T21:27:42.817041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":329388809,"num_objects":80429,"num_object_clones":0,"num_object_copies":160858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80429,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":210030,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8877,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1943744,"kb_used_data":843584,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281171776,"statfs":{"total":289910292480,"available":287919898624,"internally_reserved":0,"allocated":863830016,"data_stored":862941256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1126484995},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,3,8,12,6,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18518016,"num_objects":-4521,"num_object_clones":0,"num_object_copies":-9042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4521,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004752"},"pg_stats":[{"pgid":"2.f","version":"47'15869","reported_seq":15949,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.358247+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:38.358247+0000","last_peered":"2024-09-15T21:27:38.358247+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:38.358247+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21639168,"num_objects":5283,"num_object_clones":0,"num_object_copies":10566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15869,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1151}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15612","reported_seq":15722,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:39.078790+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:39.078790+0000","last_peered":"2024-09-15T21:27:39.078790+0000","last_clean":"2024-09-15T21:27:39.078790+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:39.078790+0000","last_undegraded":"2024-09-15T21:27:39.078790+0000","last_fullsized":"2024-09-15T21:27:39.078790+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20734918,"num_objects":5064,"num_object_clones":0,"num_object_copies":10128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5064,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13893,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16023","reported_seq":17410,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:39.163105+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:39.163105+0000","last_peered":"2024-09-15T21:27:39.163105+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:39.163105+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20815896,"num_objects":5083,"num_object_clones":0,"num_object_copies":10166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3641,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5083,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14123,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":627,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1442}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15418","reported_seq":15494,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:38.976959+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:38.976959+0000","last_peered":"2024-09-15T21:27:38.976959+0000","last_clean":"2024-09-15T21:27:38.976959+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:38.976959+0000","last_undegraded":"2024-09-15T21:27:38.976959+0000","last_fullsized":"2024-09-15T21:27:38.976959+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20054016,"num_objects":4896,"num_object_clones":0,"num_object_copies":9792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15418,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15500","reported_seq":15638,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:38.362242+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:38.362242+0000","last_peered":"2024-09-15T21:27:38.362242+0000","last_clean":"2024-09-15T21:27:38.362242+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:38.362242+0000","last_undegraded":"2024-09-15T21:27:38.362242+0000","last_fullsized":"2024-09-15T21:27:38.362242+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20500963,"num_objects":5006,"num_object_clones":0,"num_object_copies":10012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11955,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15569","reported_seq":15711,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.323246+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:38.323246+0000","last_peered":"2024-09-15T21:27:38.323246+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:38.323246+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20750819,"num_objects":5067,"num_object_clones":0,"num_object_copies":10134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5067,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12245,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":991},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15655","reported_seq":15796,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.399091+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:38.399091+0000","last_peered":"2024-09-15T21:27:38.399091+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:38.399091+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20542406,"num_objects":5017,"num_object_clones":0,"num_object_copies":10034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11966,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":856},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15845","reported_seq":15979,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:38.339542+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:38.339542+0000","last_peered":"2024-09-15T21:27:38.339542+0000","last_clean":"2024-09-15T21:27:38.339542+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:38.339542+0000","last_undegraded":"2024-09-15T21:27:38.339542+0000","last_fullsized":"2024-09-15T21:27:38.339542+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20574208,"num_objects":5023,"num_object_clones":0,"num_object_copies":10046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12617,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15523","reported_seq":15655,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:39.183998+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:39.183998+0000","last_peered":"2024-09-15T21:27:39.183998+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:39.183998+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20439523,"num_objects":4991,"num_object_clones":0,"num_object_copies":9982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4991,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13455,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":904}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15480","reported_seq":15613,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.393754+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:38.393754+0000","last_peered":"2024-09-15T21:27:38.393754+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:38.393754+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20488192,"num_objects":5002,"num_object_clones":0,"num_object_copies":10004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12153,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":982}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15622","reported_seq":15746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:39.044326+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:39.044326+0000","last_peered":"2024-09-15T21:27:39.044326+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:39.044326+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20566499,"num_objects":5022,"num_object_clones":0,"num_object_copies":10044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13481,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":932}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15616","reported_seq":24080,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:39.177019+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:39.177019+0000","last_peered":"2024-09-15T21:27:39.177019+0000","last_clean":"2024-09-15T21:27:39.177019+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:39.177019+0000","last_undegraded":"2024-09-15T21:27:39.177019+0000","last_fullsized":"2024-09-15T21:27:39.177019+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":4944,"num_object_clones":0,"num_object_copies":9888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13309,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15634","reported_seq":15766,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.405562+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:38.405562+0000","last_peered":"2024-09-15T21:27:38.405562+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:38.405562+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20692992,"num_objects":5052,"num_object_clones":0,"num_object_copies":10104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12578,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":958}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15611","reported_seq":15748,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.372909+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:38.372909+0000","last_peered":"2024-09-15T21:27:38.372909+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:38.372909+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20393984,"num_objects":4979,"num_object_clones":0,"num_object_copies":9958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12520,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15461","reported_seq":23776,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:39.261086+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:39.261086+0000","last_peered":"2024-09-15T21:27:39.261086+0000","last_clean":"2024-09-15T21:27:39.261086+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:39.261086+0000","last_undegraded":"2024-09-15T21:27:39.261086+0000","last_fullsized":"2024-09-15T21:27:39.261086+0000","mapping_epoch":46,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20180992,"num_objects":4927,"num_object_clones":0,"num_object_copies":9854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11945,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317359+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317359+0000","last_peered":"2024-09-15T21:27:09.317359+0000","last_clean":"2024-09-15T21:27:09.317359+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317359+0000","last_undegraded":"2024-09-15T21:27:09.317359+0000","last_fullsized":"2024-09-15T21:27:09.317359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15797","reported_seq":15941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:38.377476+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:38.377476+0000","last_peered":"2024-09-15T21:27:38.377476+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:38.377476+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20763590,"num_objects":5071,"num_object_clones":0,"num_object_copies":10142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12501,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":924},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":329388790,"num_objects":80427,"num_object_clones":0,"num_object_copies":160854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40598,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80427,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":210028,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8876,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":860323840,"data_stored":860178402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":621356,"kb_used_data":292568,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93750484,"statfs":{"total":96636764160,"available":96000495616,"internally_reserved":0,"allocated":299589632,"data_stored":299304374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":336653993},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,2,2,2,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":728004,"kb_used_data":309036,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93643836,"statfs":{"total":96636764160,"available":95891288064,"internally_reserved":0,"allocated":316452864,"data_stored":316167601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594384,"kb_used_data":241980,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93777456,"statfs":{"total":96636764160,"available":96028114944,"internally_reserved":0,"allocated":247787520,"data_stored":247469281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,4,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":315564032,"data_stored":315527902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":246468608,"data_stored":246399502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":298291200,"data_stored":298250998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:46.484 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:47.052 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:47.053 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:47.082 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-15T21:27:46.818458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":319718153,"num_objects":78068,"num_object_clones":0,"num_object_copies":156136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78068,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":212391,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8967,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1907496,"kb_used_data":823976,"kb_used_omap":12,"kb_used_meta":1083443,"kb_avail":281208024,"statfs":{"total":289910292480,"available":287957016576,"internally_reserved":0,"allocated":843751424,"data_stored":842862664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1109445635},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,0,7,10,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18776064,"num_objects":-4584,"num_object_clones":0,"num_object_copies":-9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4584,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004623"},"pg_stats":[{"pgid":"2.f","version":"47'16019","reported_seq":16097,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.307524+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:43.307524+0000","last_peered":"2024-09-15T21:27:43.307524+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:43.307524+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21024768,"num_objects":5133,"num_object_clones":0,"num_object_copies":10266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16019,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1001}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15770","reported_seq":15880,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.202791+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:44.202791+0000","last_peered":"2024-09-15T21:27:44.202791+0000","last_clean":"2024-09-15T21:27:44.202791+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:44.202791+0000","last_undegraded":"2024-09-15T21:27:44.202791+0000","last_fullsized":"2024-09-15T21:27:44.202791+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20087750,"num_objects":4906,"num_object_clones":0,"num_object_copies":9812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4906,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14051,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16175","reported_seq":17740,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:44.206732+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:44.206732+0000","last_peered":"2024-09-15T21:27:44.206732+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:44.206732+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20193304,"num_objects":4931,"num_object_clones":0,"num_object_copies":9862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4931,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14275,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1380}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15564","reported_seq":15640,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.085803+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:44.085803+0000","last_peered":"2024-09-15T21:27:44.085803+0000","last_clean":"2024-09-15T21:27:44.085803+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:44.085803+0000","last_undegraded":"2024-09-15T21:27:44.085803+0000","last_fullsized":"2024-09-15T21:27:44.085803+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19456000,"num_objects":4750,"num_object_clones":0,"num_object_copies":9500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15564,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15638","reported_seq":15776,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:43.316565+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:43.316565+0000","last_peered":"2024-09-15T21:27:43.316565+0000","last_clean":"2024-09-15T21:27:43.316565+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:43.316565+0000","last_undegraded":"2024-09-15T21:27:43.316565+0000","last_fullsized":"2024-09-15T21:27:43.316565+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19935715,"num_objects":4868,"num_object_clones":0,"num_object_copies":9736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12093,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15709","reported_seq":15850,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.379334+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:43.379334+0000","last_peered":"2024-09-15T21:27:43.379334+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:43.379334+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20177379,"num_objects":4927,"num_object_clones":0,"num_object_copies":9854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4927,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12385,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":851},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15810","reported_seq":15952,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.290373+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:43.290373+0000","last_peered":"2024-09-15T21:27:43.290373+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:43.290373+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19907526,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12121,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":701},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16010","reported_seq":16144,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:43.348949+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:43.348949+0000","last_peered":"2024-09-15T21:27:43.348949+0000","last_clean":"2024-09-15T21:27:43.348949+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:43.348949+0000","last_undegraded":"2024-09-15T21:27:43.348949+0000","last_fullsized":"2024-09-15T21:27:43.348949+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19898368,"num_objects":4858,"num_object_clones":0,"num_object_copies":9716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12782,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15677","reported_seq":15809,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:44.151887+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:44.151887+0000","last_peered":"2024-09-15T21:27:44.151887+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:44.151887+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19808739,"num_objects":4837,"num_object_clones":0,"num_object_copies":9674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13609,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":750}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15619","reported_seq":15755,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.184585+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:43.184585+0000","last_peered":"2024-09-15T21:27:43.184585+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:43.184585+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19918848,"num_objects":4863,"num_object_clones":0,"num_object_copies":9726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12292,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":843}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15755","reported_seq":15879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:44.167983+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:44.167983+0000","last_peered":"2024-09-15T21:27:44.167983+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:44.167983+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021731,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13614,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15745","reported_seq":24209,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.141427+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:44.141427+0000","last_peered":"2024-09-15T21:27:44.141427+0000","last_clean":"2024-09-15T21:27:44.141427+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:44.141427+0000","last_undegraded":"2024-09-15T21:27:44.141427+0000","last_fullsized":"2024-09-15T21:27:44.141427+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19722240,"num_objects":4815,"num_object_clones":0,"num_object_copies":9630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13438,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15782","reported_seq":15921,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.388701+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:43.388701+0000","last_peered":"2024-09-15T21:27:43.388701+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:43.388701+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20086784,"num_objects":4904,"num_object_clones":0,"num_object_copies":9808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12726,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15773","reported_seq":15911,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.393704+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:43.393704+0000","last_peered":"2024-09-15T21:27:43.393704+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:43.393704+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19730432,"num_objects":4817,"num_object_clones":0,"num_object_copies":9634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12682,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":678}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15613","reported_seq":23928,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.412817+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:44.412817+0000","last_peered":"2024-09-15T21:27:44.412817+0000","last_clean":"2024-09-15T21:27:44.412817+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:44.412817+0000","last_undegraded":"2024-09-15T21:27:44.412817+0000","last_fullsized":"2024-09-15T21:27:44.412817+0000","mapping_epoch":46,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19558400,"num_objects":4775,"num_object_clones":0,"num_object_copies":9550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12097,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15937","reported_seq":16081,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.399000+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:43.399000+0000","last_peered":"2024-09-15T21:27:43.399000+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:43.399000+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20190150,"num_objects":4931,"num_object_clones":0,"num_object_copies":9862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4931,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12641,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":784},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":319718134,"num_objects":78066,"num_object_clones":0,"num_object_copies":156132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78066,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":212389,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8966,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":841125888,"data_stored":840980450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160696,"ondisk_log_size":160696,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603348,"kb_used_data":287036,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93768492,"statfs":{"total":96636764160,"available":96018935808,"internally_reserved":0,"allocated":293924864,"data_stored":293639606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712468,"kb_used_data":301820,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93659372,"statfs":{"total":96636764160,"available":95907196928,"internally_reserved":0,"allocated":309063680,"data_stored":308778417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,3,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591680,"kb_used_data":235120,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93780160,"statfs":{"total":96636764160,"available":96030883840,"internally_reserved":0,"allocated":240762880,"data_stored":240444641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":308346880,"data_stored":308310750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":239259648,"data_stored":239190542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293519360,"data_stored":293479158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:47.083 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:47.676 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:47.676 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:47.702 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-15T21:27:46.818458+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":319718153,"num_objects":78068,"num_object_clones":0,"num_object_copies":156136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78068,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":212391,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8967,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1907496,"kb_used_data":823976,"kb_used_omap":12,"kb_used_meta":1083443,"kb_avail":281208024,"statfs":{"total":289910292480,"available":287957016576,"internally_reserved":0,"allocated":843751424,"data_stored":842862664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1109445635},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,0,7,10,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18776064,"num_objects":-4584,"num_object_clones":0,"num_object_copies":-9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4584,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004623"},"pg_stats":[{"pgid":"2.f","version":"47'16019","reported_seq":16097,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.307524+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:43.307524+0000","last_peered":"2024-09-15T21:27:43.307524+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:43.307524+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21024768,"num_objects":5133,"num_object_clones":0,"num_object_copies":10266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5133,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16019,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1001}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15770","reported_seq":15880,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.202791+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:44.202791+0000","last_peered":"2024-09-15T21:27:44.202791+0000","last_clean":"2024-09-15T21:27:44.202791+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:44.202791+0000","last_undegraded":"2024-09-15T21:27:44.202791+0000","last_fullsized":"2024-09-15T21:27:44.202791+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20087750,"num_objects":4906,"num_object_clones":0,"num_object_copies":9812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4906,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14051,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16175","reported_seq":17740,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:44.206732+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:44.206732+0000","last_peered":"2024-09-15T21:27:44.206732+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:44.206732+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20193304,"num_objects":4931,"num_object_clones":0,"num_object_copies":9862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4931,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14275,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1380}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15564","reported_seq":15640,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.085803+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:44.085803+0000","last_peered":"2024-09-15T21:27:44.085803+0000","last_clean":"2024-09-15T21:27:44.085803+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:44.085803+0000","last_undegraded":"2024-09-15T21:27:44.085803+0000","last_fullsized":"2024-09-15T21:27:44.085803+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19456000,"num_objects":4750,"num_object_clones":0,"num_object_copies":9500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15564,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15638","reported_seq":15776,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:43.316565+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:43.316565+0000","last_peered":"2024-09-15T21:27:43.316565+0000","last_clean":"2024-09-15T21:27:43.316565+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:43.316565+0000","last_undegraded":"2024-09-15T21:27:43.316565+0000","last_fullsized":"2024-09-15T21:27:43.316565+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19935715,"num_objects":4868,"num_object_clones":0,"num_object_copies":9736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12093,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15709","reported_seq":15850,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.379334+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:43.379334+0000","last_peered":"2024-09-15T21:27:43.379334+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:43.379334+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20177379,"num_objects":4927,"num_object_clones":0,"num_object_copies":9854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4927,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12385,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":851},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15810","reported_seq":15952,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.290373+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:43.290373+0000","last_peered":"2024-09-15T21:27:43.290373+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:43.290373+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19907526,"num_objects":4862,"num_object_clones":0,"num_object_copies":9724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12121,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":701},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16010","reported_seq":16144,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:43.348949+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:43.348949+0000","last_peered":"2024-09-15T21:27:43.348949+0000","last_clean":"2024-09-15T21:27:43.348949+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:43.348949+0000","last_undegraded":"2024-09-15T21:27:43.348949+0000","last_fullsized":"2024-09-15T21:27:43.348949+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19898368,"num_objects":4858,"num_object_clones":0,"num_object_copies":9716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12782,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15677","reported_seq":15809,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:44.151887+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:44.151887+0000","last_peered":"2024-09-15T21:27:44.151887+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:44.151887+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19808739,"num_objects":4837,"num_object_clones":0,"num_object_copies":9674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13609,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":750}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15619","reported_seq":15755,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.184585+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:43.184585+0000","last_peered":"2024-09-15T21:27:43.184585+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:43.184585+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19918848,"num_objects":4863,"num_object_clones":0,"num_object_copies":9726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12292,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":843}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15755","reported_seq":15879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:44.167983+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:44.167983+0000","last_peered":"2024-09-15T21:27:44.167983+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:44.167983+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021731,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13614,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15745","reported_seq":24209,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.141427+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:44.141427+0000","last_peered":"2024-09-15T21:27:44.141427+0000","last_clean":"2024-09-15T21:27:44.141427+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:44.141427+0000","last_undegraded":"2024-09-15T21:27:44.141427+0000","last_fullsized":"2024-09-15T21:27:44.141427+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19722240,"num_objects":4815,"num_object_clones":0,"num_object_copies":9630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13438,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15782","reported_seq":15921,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.388701+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:43.388701+0000","last_peered":"2024-09-15T21:27:43.388701+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:43.388701+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20086784,"num_objects":4904,"num_object_clones":0,"num_object_copies":9808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12726,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":810}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15773","reported_seq":15911,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.393704+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:43.393704+0000","last_peered":"2024-09-15T21:27:43.393704+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:43.393704+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19730432,"num_objects":4817,"num_object_clones":0,"num_object_copies":9634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12682,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":678}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15613","reported_seq":23928,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.412817+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:44.412817+0000","last_peered":"2024-09-15T21:27:44.412817+0000","last_clean":"2024-09-15T21:27:44.412817+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:44.412817+0000","last_undegraded":"2024-09-15T21:27:44.412817+0000","last_fullsized":"2024-09-15T21:27:44.412817+0000","mapping_epoch":46,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19558400,"num_objects":4775,"num_object_clones":0,"num_object_copies":9550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12097,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15937","reported_seq":16081,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:43.399000+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:43.399000+0000","last_peered":"2024-09-15T21:27:43.399000+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:43.399000+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20190150,"num_objects":4931,"num_object_clones":0,"num_object_copies":9862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4931,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12641,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":784},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":319718134,"num_objects":78066,"num_object_clones":0,"num_object_copies":156132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78066,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":212389,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8966,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":841125888,"data_stored":840980450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160696,"ondisk_log_size":160696,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603348,"kb_used_data":287036,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93768492,"statfs":{"total":96636764160,"available":96018935808,"internally_reserved":0,"allocated":293924864,"data_stored":293639606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":712468,"kb_used_data":301820,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93659372,"statfs":{"total":96636764160,"available":95907196928,"internally_reserved":0,"allocated":309063680,"data_stored":308778417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,0,3,7,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591680,"kb_used_data":235120,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93780160,"statfs":{"total":96636764160,"available":96030883840,"internally_reserved":0,"allocated":240762880,"data_stored":240444641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":308346880,"data_stored":308310750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":239259648,"data_stored":239190542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293519360,"data_stored":293479158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:47.703 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:47.703 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16019", 'reported_seq': 16097, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:43.307524+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:27:43.307524+0000', 'last_peered': '2024-09-15T21:27:43.307524+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:27:43.307524+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21024768, 'num_objects': 5133, 'num_object_clones': 0, 'num_object_copies': 10266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5133, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16019, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1001}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:47.703 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:27:47.703 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'16175", 'reported_seq': 17740, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:44.206732+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:27:44.206732+0000', 'last_peered': '2024-09-15T21:27:44.206732+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:27:44.206732+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20193304, 'num_objects': 4931, 'num_object_clones': 0, 'num_object_copies': 9862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3551, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4931, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14275, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 717, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1380}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:47.703 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:47.703 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'15709", 'reported_seq': 15850, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:43.379334+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:27:43.379334+0000', 'last_peered': '2024-09-15T21:27:43.379334+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:27:43.379334+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20177379, 'num_objects': 4927, 'num_object_clones': 0, 'num_object_copies': 9854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4927, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12385, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 851}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:47.704 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:47.704 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'15810", 'reported_seq': 15952, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:43.290373+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:27:43.290373+0000', 'last_peered': '2024-09-15T21:27:43.290373+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:27:43.290373+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19907526, 'num_objects': 4862, 'num_object_clones': 0, 'num_object_copies': 9724, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4862, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12121, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 701}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:47.704 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:27:47.704 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'15677", 'reported_seq': 15809, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:44.151887+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:27:44.151887+0000', 'last_peered': '2024-09-15T21:27:44.151887+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:27:44.151887+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19808739, 'num_objects': 4837, 'num_object_clones': 0, 'num_object_copies': 9674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4837, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13609, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 750}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:47.704 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:47.704 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15619", 'reported_seq': 15755, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:43.184585+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:27:43.184585+0000', 'last_peered': '2024-09-15T21:27:43.184585+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:27:43.184585+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19918848, 'num_objects': 4863, 'num_object_clones': 0, 'num_object_copies': 9726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4863, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12292, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 843}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:47.704 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:27:47.704 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'15755", 'reported_seq': 15879, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:44.167983+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:27:44.167983+0000', 'last_peered': '2024-09-15T21:27:44.167983+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:27:44.167983+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20021731, 'num_objects': 4889, 'num_object_clones': 0, 'num_object_copies': 9778, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4889, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13614, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 799}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:47.704 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:47.704 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'15782", 'reported_seq': 15921, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:43.388701+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:27:43.388701+0000', 'last_peered': '2024-09-15T21:27:43.388701+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:27:43.388701+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20086784, 'num_objects': 4904, 'num_object_clones': 0, 'num_object_copies': 9808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4904, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12726, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 810}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:47.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:27:47.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'15773", 'reported_seq': 15911, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:43.393704+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:27:43.393704+0000', 'last_peered': '2024-09-15T21:27:43.393704+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:27:43.393704+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19730432, 'num_objects': 4817, 'num_object_clones': 0, 'num_object_copies': 9634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4817, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12682, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 678}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:47.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:27:47.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'15937", 'reported_seq': 16081, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:43.399000+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:27:43.399000+0000', 'last_peered': '2024-09-15T21:27:43.399000+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:27:43.399000+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20190150, 'num_objects': 4931, 'num_object_clones': 0, 'num_object_copies': 9862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4931, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12641, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 784}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:47.705 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:48.405 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:48.405 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40508/156136 objects degraded (25.944%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":520,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":78068,"data_bytes":319718153,"bytes_used":1953275904,"bytes_avail":287957016576,"bytes_total":289910292480,"degraded_objects":40508,"degraded_total":156136,"degraded_ratio":0.25944048777988421,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":381},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:48.430 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:48.430 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:49.020 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:49.021 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:49.047 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":300,"stamp":"2024-09-15T21:27:48.819058+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":314221321,"num_objects":76726,"num_object_clones":0,"num_object_copies":153452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76726,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":213733,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8967,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160941,"ondisk_log_size":160941,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1904696,"kb_used_data":816996,"kb_used_omap":12,"kb_used_meta":1087603,"kb_avail":281210824,"statfs":{"total":289910292480,"available":287959883776,"internally_reserved":0,"allocated":836603904,"data_stored":835715144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1113705475},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,1,5,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24272896,"num_objects":-5926,"num_object_clones":0,"num_object_copies":-11852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5926,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004465"},"pg_stats":[{"pgid":"2.f","version":"47'16182","reported_seq":16263,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.391074+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:48.391074+0000","last_peered":"2024-09-15T21:27:48.391074+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:48.391074+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20357120,"num_objects":4970,"num_object_clones":0,"num_object_copies":9940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":838}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15770","reported_seq":15880,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.202791+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:44.202791+0000","last_peered":"2024-09-15T21:27:44.202791+0000","last_clean":"2024-09-15T21:27:44.202791+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:44.202791+0000","last_undegraded":"2024-09-15T21:27:44.202791+0000","last_fullsized":"2024-09-15T21:27:44.202791+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20087750,"num_objects":4906,"num_object_clones":0,"num_object_copies":9812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4906,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14051,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16175","reported_seq":17740,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:44.206732+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:44.206732+0000","last_peered":"2024-09-15T21:27:44.206732+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:44.206732+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20193304,"num_objects":4931,"num_object_clones":0,"num_object_copies":9862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3551,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4931,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14275,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":717,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1380}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15564","reported_seq":15640,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.085803+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:44.085803+0000","last_peered":"2024-09-15T21:27:44.085803+0000","last_clean":"2024-09-15T21:27:44.085803+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:44.085803+0000","last_undegraded":"2024-09-15T21:27:44.085803+0000","last_fullsized":"2024-09-15T21:27:44.085803+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19456000,"num_objects":4750,"num_object_clones":0,"num_object_copies":9500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15564,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15781","reported_seq":15919,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:48.444194+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:48.444194+0000","last_peered":"2024-09-15T21:27:48.444194+0000","last_clean":"2024-09-15T21:27:48.444194+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:48.444194+0000","last_undegraded":"2024-09-15T21:27:48.444194+0000","last_fullsized":"2024-09-15T21:27:48.444194+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19349987,"num_objects":4725,"num_object_clones":0,"num_object_copies":9450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12236,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15854","reported_seq":15996,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.353750+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:48.353750+0000","last_peered":"2024-09-15T21:27:48.353750+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:48.353750+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19583459,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12530,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":706},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15962","reported_seq":16102,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.455710+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:48.455710+0000","last_peered":"2024-09-15T21:27:48.455710+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:48.455710+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19284934,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12273,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":549},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16157","reported_seq":16291,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:48.435988+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:48.435988+0000","last_peered":"2024-09-15T21:27:48.435988+0000","last_clean":"2024-09-15T21:27:48.435988+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:48.435988+0000","last_undegraded":"2024-09-15T21:27:48.435988+0000","last_fullsized":"2024-09-15T21:27:48.435988+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19296256,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12929,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15677","reported_seq":15809,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:44.151887+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:44.151887+0000","last_peered":"2024-09-15T21:27:44.151887+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:44.151887+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19808739,"num_objects":4837,"num_object_clones":0,"num_object_copies":9674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4837,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13609,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":750}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15764","reported_seq":15898,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.405808+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:48.405808+0000","last_peered":"2024-09-15T21:27:48.405808+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:48.405808+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19324928,"num_objects":4718,"num_object_clones":0,"num_object_copies":9436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12437,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":698}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15755","reported_seq":15879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:44.167983+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:44.167983+0000","last_peered":"2024-09-15T21:27:44.167983+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:44.167983+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20021731,"num_objects":4889,"num_object_clones":0,"num_object_copies":9778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4889,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13614,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":799}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15745","reported_seq":24209,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.141427+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:44.141427+0000","last_peered":"2024-09-15T21:27:44.141427+0000","last_clean":"2024-09-15T21:27:44.141427+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:44.141427+0000","last_undegraded":"2024-09-15T21:27:44.141427+0000","last_fullsized":"2024-09-15T21:27:44.141427+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19722240,"num_objects":4815,"num_object_clones":0,"num_object_copies":9630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13438,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15943","reported_seq":16082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.343346+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:48.343346+0000","last_peered":"2024-09-15T21:27:48.343346+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:48.343346+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19427328,"num_objects":4743,"num_object_clones":0,"num_object_copies":9486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12887,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":649}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15913","reported_seq":16050,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.392962+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:48.392962+0000","last_peered":"2024-09-15T21:27:48.392962+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:48.392962+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19156992,"num_objects":4677,"num_object_clones":0,"num_object_copies":9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12822,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":538}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15613","reported_seq":23928,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.412817+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:44.412817+0000","last_peered":"2024-09-15T21:27:44.412817+0000","last_clean":"2024-09-15T21:27:44.412817+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:44.412817+0000","last_undegraded":"2024-09-15T21:27:44.412817+0000","last_fullsized":"2024-09-15T21:27:44.412817+0000","mapping_epoch":46,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19558400,"num_objects":4775,"num_object_clones":0,"num_object_copies":9550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12097,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16083","reported_seq":16227,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.381406+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:48.381406+0000","last_peered":"2024-09-15T21:27:48.381406+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:48.381406+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19592134,"num_objects":4785,"num_object_clones":0,"num_object_copies":9570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12787,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":638},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":314221302,"num_objects":76724,"num_object_clones":0,"num_object_copies":153448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76724,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":213731,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8966,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":833880064,"data_stored":833734626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160938,"ondisk_log_size":160938,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603348,"kb_used_data":287036,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93768492,"statfs":{"total":96636764160,"available":96018935808,"internally_reserved":0,"allocated":293924864,"data_stored":293639606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709668,"kb_used_data":294840,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93662172,"statfs":{"total":96636764160,"available":95910064128,"internally_reserved":0,"allocated":301916160,"data_stored":301630897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591680,"kb_used_data":235120,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93780160,"statfs":{"total":96636764160,"available":96030883840,"internally_reserved":0,"allocated":240762880,"data_stored":240444641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365096623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":301101056,"data_stored":301064926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":239259648,"data_stored":239190542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":293519360,"data_stored":293479158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:52.049 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:52.603 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:52.604 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:52.625 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-15T21:27:50.819970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":310227721,"num_objects":75751,"num_object_clones":0,"num_object_copies":151502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75751,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":214708,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9063,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1906444,"kb_used_data":806260,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281209076,"statfs":{"total":289910292480,"available":287958093824,"internally_reserved":0,"allocated":825610240,"data_stored":824721480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1126484995},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,6,14,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23187456,"num_objects":-5661,"num_object_clones":0,"num_object_copies":-11322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-277,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5661,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004608"},"pg_stats":[{"pgid":"2.f","version":"47'16182","reported_seq":16263,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.391074+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:48.391074+0000","last_peered":"2024-09-15T21:27:48.391074+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:48.391074+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20357120,"num_objects":4970,"num_object_clones":0,"num_object_copies":9940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":838}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15913","reported_seq":16024,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:49.169308+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:49.169308+0000","last_peered":"2024-09-15T21:27:49.169308+0000","last_clean":"2024-09-15T21:27:49.169308+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:49.169308+0000","last_undegraded":"2024-09-15T21:27:49.169308+0000","last_fullsized":"2024-09-15T21:27:49.169308+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19502022,"num_objects":4763,"num_object_clones":0,"num_object_copies":9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4763,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14194,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16324","reported_seq":18083,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:49.194943+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:49.194943+0000","last_peered":"2024-09-15T21:27:49.194943+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:49.194943+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19583000,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14424,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":813,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1327}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15683","reported_seq":15759,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:49.186265+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:49.186265+0000","last_peered":"2024-09-15T21:27:49.186265+0000","last_clean":"2024-09-15T21:27:49.186265+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:49.186265+0000","last_undegraded":"2024-09-15T21:27:49.186265+0000","last_fullsized":"2024-09-15T21:27:49.186265+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18968576,"num_objects":4631,"num_object_clones":0,"num_object_copies":9262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15683,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15781","reported_seq":15919,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:48.444194+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:48.444194+0000","last_peered":"2024-09-15T21:27:48.444194+0000","last_clean":"2024-09-15T21:27:48.444194+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:48.444194+0000","last_undegraded":"2024-09-15T21:27:48.444194+0000","last_fullsized":"2024-09-15T21:27:48.444194+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19349987,"num_objects":4725,"num_object_clones":0,"num_object_copies":9450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12236,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15854","reported_seq":15996,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.353750+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:48.353750+0000","last_peered":"2024-09-15T21:27:48.353750+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:48.353750+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19583459,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12530,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":706},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15962","reported_seq":16102,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.455710+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:48.455710+0000","last_peered":"2024-09-15T21:27:48.455710+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:48.455710+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19284934,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12273,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":549},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16157","reported_seq":16291,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:48.435988+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:48.435988+0000","last_peered":"2024-09-15T21:27:48.435988+0000","last_clean":"2024-09-15T21:27:48.435988+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:48.435988+0000","last_undegraded":"2024-09-15T21:27:48.435988+0000","last_fullsized":"2024-09-15T21:27:48.435988+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19296256,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12929,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15811","reported_seq":15943,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:49.085072+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:49.085072+0000","last_peered":"2024-09-15T21:27:49.085072+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:49.085072+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19259875,"num_objects":4703,"num_object_clones":0,"num_object_copies":9406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4703,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13743,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":616}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15764","reported_seq":15898,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.405808+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:48.405808+0000","last_peered":"2024-09-15T21:27:48.405808+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:48.405808+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19324928,"num_objects":4718,"num_object_clones":0,"num_object_copies":9436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12437,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":698}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15896","reported_seq":16020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:49.170463+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:49.170463+0000","last_peered":"2024-09-15T21:27:49.170463+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:49.170463+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19444195,"num_objects":4748,"num_object_clones":0,"num_object_copies":9496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13755,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15897","reported_seq":24361,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:49.186395+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:49.186395+0000","last_peered":"2024-09-15T21:27:49.186395+0000","last_clean":"2024-09-15T21:27:49.186395+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:49.186395+0000","last_undegraded":"2024-09-15T21:27:49.186395+0000","last_fullsized":"2024-09-15T21:27:49.186395+0000","mapping_epoch":44,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19099648,"num_objects":4663,"num_object_clones":0,"num_object_copies":9326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13590,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15943","reported_seq":16082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.343346+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:48.343346+0000","last_peered":"2024-09-15T21:27:48.343346+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:48.343346+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19427328,"num_objects":4743,"num_object_clones":0,"num_object_copies":9486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12887,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":649}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15913","reported_seq":16050,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.392962+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:48.392962+0000","last_peered":"2024-09-15T21:27:48.392962+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:48.392962+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19156992,"num_objects":4677,"num_object_clones":0,"num_object_copies":9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12822,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":538}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15750","reported_seq":24065,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:49.477090+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:49.477090+0000","last_peered":"2024-09-15T21:27:49.477090+0000","last_clean":"2024-09-15T21:27:49.477090+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:49.477090+0000","last_undegraded":"2024-09-15T21:27:49.477090+0000","last_fullsized":"2024-09-15T21:27:49.477090+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18997248,"num_objects":4638,"num_object_clones":0,"num_object_copies":9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12234,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16083","reported_seq":16227,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.381406+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:48.381406+0000","last_peered":"2024-09-15T21:27:48.381406+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:48.381406+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19592134,"num_objects":4785,"num_object_clones":0,"num_object_copies":9570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12787,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":638},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":310227702,"num_objects":75749,"num_object_clones":0,"num_object_copies":151498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75749,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":214706,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9062,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":822054912,"data_stored":821909474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603836,"kb_used_data":283372,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93768004,"statfs":{"total":96636764160,"available":96018436096,"internally_reserved":0,"allocated":290172928,"data_stored":289887670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":328134313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709668,"kb_used_data":294840,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93662172,"statfs":{"total":96636764160,"available":95910064128,"internally_reserved":0,"allocated":301916160,"data_stored":301630897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592940,"kb_used_data":228048,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93778900,"statfs":{"total":96636764160,"available":96029593600,"internally_reserved":0,"allocated":233521152,"data_stored":233202913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":301101056,"data_stored":301064926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":232054784,"data_stored":231985678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288899072,"data_stored":288858870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:52.627 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:53.195 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:53.197 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:53.219 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-15T21:27:52.820846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":310227721,"num_objects":75751,"num_object_clones":0,"num_object_copies":151502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75751,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":214708,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9063,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1906444,"kb_used_data":806260,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281209076,"statfs":{"total":289910292480,"available":287958093824,"internally_reserved":0,"allocated":825610240,"data_stored":824721480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1126484995},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,6,14,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19161088,"num_objects":-4678,"num_object_clones":0,"num_object_copies":-9356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4678,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004675"},"pg_stats":[{"pgid":"2.f","version":"47'16182","reported_seq":16263,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.391074+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:48.391074+0000","last_peered":"2024-09-15T21:27:48.391074+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:48.391074+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20357120,"num_objects":4970,"num_object_clones":0,"num_object_copies":9940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":838}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15913","reported_seq":16024,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:49.169308+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:49.169308+0000","last_peered":"2024-09-15T21:27:49.169308+0000","last_clean":"2024-09-15T21:27:49.169308+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:49.169308+0000","last_undegraded":"2024-09-15T21:27:49.169308+0000","last_fullsized":"2024-09-15T21:27:49.169308+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19502022,"num_objects":4763,"num_object_clones":0,"num_object_copies":9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4763,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14194,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16324","reported_seq":18083,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:49.194943+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:49.194943+0000","last_peered":"2024-09-15T21:27:49.194943+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:49.194943+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19583000,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14424,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":813,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1327}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15683","reported_seq":15759,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:49.186265+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:49.186265+0000","last_peered":"2024-09-15T21:27:49.186265+0000","last_clean":"2024-09-15T21:27:49.186265+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:49.186265+0000","last_undegraded":"2024-09-15T21:27:49.186265+0000","last_fullsized":"2024-09-15T21:27:49.186265+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18968576,"num_objects":4631,"num_object_clones":0,"num_object_copies":9262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15683,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15781","reported_seq":15919,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:48.444194+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:48.444194+0000","last_peered":"2024-09-15T21:27:48.444194+0000","last_clean":"2024-09-15T21:27:48.444194+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:48.444194+0000","last_undegraded":"2024-09-15T21:27:48.444194+0000","last_fullsized":"2024-09-15T21:27:48.444194+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19349987,"num_objects":4725,"num_object_clones":0,"num_object_copies":9450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12236,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15854","reported_seq":15996,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.353750+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:48.353750+0000","last_peered":"2024-09-15T21:27:48.353750+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:48.353750+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19583459,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12530,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":706},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15962","reported_seq":16102,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.455710+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:48.455710+0000","last_peered":"2024-09-15T21:27:48.455710+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:48.455710+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19284934,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12273,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":549},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16157","reported_seq":16291,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:48.435988+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:48.435988+0000","last_peered":"2024-09-15T21:27:48.435988+0000","last_clean":"2024-09-15T21:27:48.435988+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:48.435988+0000","last_undegraded":"2024-09-15T21:27:48.435988+0000","last_fullsized":"2024-09-15T21:27:48.435988+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19296256,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12929,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15811","reported_seq":15943,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:49.085072+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:49.085072+0000","last_peered":"2024-09-15T21:27:49.085072+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:49.085072+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19259875,"num_objects":4703,"num_object_clones":0,"num_object_copies":9406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4703,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13743,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":616}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15764","reported_seq":15898,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.405808+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:48.405808+0000","last_peered":"2024-09-15T21:27:48.405808+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:48.405808+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19324928,"num_objects":4718,"num_object_clones":0,"num_object_copies":9436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12437,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":698}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15896","reported_seq":16020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:49.170463+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:49.170463+0000","last_peered":"2024-09-15T21:27:49.170463+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:49.170463+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19444195,"num_objects":4748,"num_object_clones":0,"num_object_copies":9496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13755,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15897","reported_seq":24361,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:49.186395+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:49.186395+0000","last_peered":"2024-09-15T21:27:49.186395+0000","last_clean":"2024-09-15T21:27:49.186395+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:49.186395+0000","last_undegraded":"2024-09-15T21:27:49.186395+0000","last_fullsized":"2024-09-15T21:27:49.186395+0000","mapping_epoch":44,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19099648,"num_objects":4663,"num_object_clones":0,"num_object_copies":9326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13590,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15943","reported_seq":16082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.343346+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:48.343346+0000","last_peered":"2024-09-15T21:27:48.343346+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:48.343346+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19427328,"num_objects":4743,"num_object_clones":0,"num_object_copies":9486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12887,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":649}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15913","reported_seq":16050,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.392962+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:48.392962+0000","last_peered":"2024-09-15T21:27:48.392962+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:48.392962+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19156992,"num_objects":4677,"num_object_clones":0,"num_object_copies":9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12822,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":538}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15750","reported_seq":24065,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:49.477090+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:49.477090+0000","last_peered":"2024-09-15T21:27:49.477090+0000","last_clean":"2024-09-15T21:27:49.477090+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:49.477090+0000","last_undegraded":"2024-09-15T21:27:49.477090+0000","last_fullsized":"2024-09-15T21:27:49.477090+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18997248,"num_objects":4638,"num_object_clones":0,"num_object_copies":9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12234,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16083","reported_seq":16227,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.381406+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:48.381406+0000","last_peered":"2024-09-15T21:27:48.381406+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:48.381406+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19592134,"num_objects":4785,"num_object_clones":0,"num_object_copies":9570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12787,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":638},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":310227702,"num_objects":75749,"num_object_clones":0,"num_object_copies":151498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75749,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":214706,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9062,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":822054912,"data_stored":821909474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603836,"kb_used_data":283372,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93768004,"statfs":{"total":96636764160,"available":96018436096,"internally_reserved":0,"allocated":290172928,"data_stored":289887670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":328134313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709668,"kb_used_data":294840,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93662172,"statfs":{"total":96636764160,"available":95910064128,"internally_reserved":0,"allocated":301916160,"data_stored":301630897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592940,"kb_used_data":228048,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93778900,"statfs":{"total":96636764160,"available":96029593600,"internally_reserved":0,"allocated":233521152,"data_stored":233202913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":301101056,"data_stored":301064926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":232054784,"data_stored":231985678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288899072,"data_stored":288858870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:53.220 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:53.220 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16182", 'reported_seq': 16263, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:48.391074+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:27:48.391074+0000', 'last_peered': '2024-09-15T21:27:48.391074+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:27:48.391074+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20357120, 'num_objects': 4970, 'num_object_clones': 0, 'num_object_copies': 9940, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4970, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16182, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 838}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:53.220 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:27:53.220 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'16324", 'reported_seq': 18083, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:49.194943+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:27:49.194943+0000', 'last_peered': '2024-09-15T21:27:49.194943+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:27:49.194943+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19583000, 'num_objects': 4782, 'num_object_clones': 0, 'num_object_copies': 9564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3455, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4782, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14424, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 813, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1327}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:53.221 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:53.221 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'15854", 'reported_seq': 15996, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:48.353750+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:27:48.353750+0000', 'last_peered': '2024-09-15T21:27:48.353750+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:27:48.353750+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19583459, 'num_objects': 4782, 'num_object_clones': 0, 'num_object_copies': 9564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4782, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12530, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 706}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:53.221 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:53.221 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'15962", 'reported_seq': 16102, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:48.455710+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:27:48.455710+0000', 'last_peered': '2024-09-15T21:27:48.455710+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:27:48.455710+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19284934, 'num_objects': 4710, 'num_object_clones': 0, 'num_object_copies': 9420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4710, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12273, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 549}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:53.221 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:27:53.221 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'15811", 'reported_seq': 15943, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:49.085072+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:27:49.085072+0000', 'last_peered': '2024-09-15T21:27:49.085072+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:27:49.085072+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19259875, 'num_objects': 4703, 'num_object_clones': 0, 'num_object_copies': 9406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4703, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13743, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 616}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:53.221 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:53.222 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15764", 'reported_seq': 15898, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:48.405808+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:27:48.405808+0000', 'last_peered': '2024-09-15T21:27:48.405808+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:27:48.405808+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19324928, 'num_objects': 4718, 'num_object_clones': 0, 'num_object_copies': 9436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12437, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 698}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:53.222 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:27:53.222 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'15896", 'reported_seq': 16020, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:49.170463+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:27:49.170463+0000', 'last_peered': '2024-09-15T21:27:49.170463+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:27:49.170463+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19444195, 'num_objects': 4748, 'num_object_clones': 0, 'num_object_copies': 9496, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4748, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13755, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 658}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:53.222 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:53.222 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'15943", 'reported_seq': 16082, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:48.343346+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:27:48.343346+0000', 'last_peered': '2024-09-15T21:27:48.343346+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:27:48.343346+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19427328, 'num_objects': 4743, 'num_object_clones': 0, 'num_object_copies': 9486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4743, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12887, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 649}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:53.222 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:27:53.222 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'15913", 'reported_seq': 16050, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:48.392962+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:27:48.392962+0000', 'last_peered': '2024-09-15T21:27:48.392962+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:27:48.392962+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19156992, 'num_objects': 4677, 'num_object_clones': 0, 'num_object_copies': 9354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4677, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12822, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 538}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:53.222 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:27:53.222 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'16083", 'reported_seq': 16227, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:48.381406+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:27:48.381406+0000', 'last_peered': '2024-09-15T21:27:48.381406+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:27:48.381406+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19592134, 'num_objects': 4785, 'num_object_clones': 0, 'num_object_copies': 9570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4785, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12787, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 638}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:53.223 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:53.911 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:53.911 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40412/151502 objects degraded (26.674%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":525,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":75751,"data_bytes":310227721,"bytes_used":1952198656,"bytes_avail":287958093824,"bytes_total":289910292480,"degraded_objects":40412,"degraded_total":151502,"degraded_ratio":0.26674235323626089,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":186,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":389},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:53.932 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:53.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:54.475 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:54.494 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-15T21:27:52.820846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":310227721,"num_objects":75751,"num_object_clones":0,"num_object_copies":151502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75751,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":214708,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9063,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1906444,"kb_used_data":806260,"kb_used_omap":12,"kb_used_meta":1100083,"kb_avail":281209076,"statfs":{"total":289910292480,"available":287958093824,"internally_reserved":0,"allocated":825610240,"data_stored":824721480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1126484995},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,6,14,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19161088,"num_objects":-4678,"num_object_clones":0,"num_object_copies":-9356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4678,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":186,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004675"},"pg_stats":[{"pgid":"2.f","version":"47'16182","reported_seq":16263,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.391074+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:48.391074+0000","last_peered":"2024-09-15T21:27:48.391074+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:48.391074+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20357120,"num_objects":4970,"num_object_clones":0,"num_object_copies":9940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":838}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15913","reported_seq":16024,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:49.169308+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:49.169308+0000","last_peered":"2024-09-15T21:27:49.169308+0000","last_clean":"2024-09-15T21:27:49.169308+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:49.169308+0000","last_undegraded":"2024-09-15T21:27:49.169308+0000","last_fullsized":"2024-09-15T21:27:49.169308+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19502022,"num_objects":4763,"num_object_clones":0,"num_object_copies":9526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4763,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14194,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16324","reported_seq":18083,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:49.194943+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:49.194943+0000","last_peered":"2024-09-15T21:27:49.194943+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:49.194943+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19583000,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14424,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":813,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1327}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15683","reported_seq":15759,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:49.186265+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:49.186265+0000","last_peered":"2024-09-15T21:27:49.186265+0000","last_clean":"2024-09-15T21:27:49.186265+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:49.186265+0000","last_undegraded":"2024-09-15T21:27:49.186265+0000","last_fullsized":"2024-09-15T21:27:49.186265+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18968576,"num_objects":4631,"num_object_clones":0,"num_object_copies":9262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15683,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15781","reported_seq":15919,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:48.444194+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:48.444194+0000","last_peered":"2024-09-15T21:27:48.444194+0000","last_clean":"2024-09-15T21:27:48.444194+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:48.444194+0000","last_undegraded":"2024-09-15T21:27:48.444194+0000","last_fullsized":"2024-09-15T21:27:48.444194+0000","mapping_epoch":34,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19349987,"num_objects":4725,"num_object_clones":0,"num_object_copies":9450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12236,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15854","reported_seq":15996,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.353750+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:48.353750+0000","last_peered":"2024-09-15T21:27:48.353750+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:48.353750+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19583459,"num_objects":4782,"num_object_clones":0,"num_object_copies":9564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4782,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12530,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":706},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15962","reported_seq":16102,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.455710+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:48.455710+0000","last_peered":"2024-09-15T21:27:48.455710+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:48.455710+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19284934,"num_objects":4710,"num_object_clones":0,"num_object_copies":9420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12273,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":549},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16157","reported_seq":16291,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:48.435988+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:48.435988+0000","last_peered":"2024-09-15T21:27:48.435988+0000","last_clean":"2024-09-15T21:27:48.435988+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:48.435988+0000","last_undegraded":"2024-09-15T21:27:48.435988+0000","last_fullsized":"2024-09-15T21:27:48.435988+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19296256,"num_objects":4711,"num_object_clones":0,"num_object_copies":9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12929,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15811","reported_seq":15943,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:49.085072+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:49.085072+0000","last_peered":"2024-09-15T21:27:49.085072+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:49.085072+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19259875,"num_objects":4703,"num_object_clones":0,"num_object_copies":9406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4703,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13743,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":616}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15764","reported_seq":15898,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.405808+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:48.405808+0000","last_peered":"2024-09-15T21:27:48.405808+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:48.405808+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19324928,"num_objects":4718,"num_object_clones":0,"num_object_copies":9436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12437,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":698}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15896","reported_seq":16020,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:49.170463+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:49.170463+0000","last_peered":"2024-09-15T21:27:49.170463+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:49.170463+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19444195,"num_objects":4748,"num_object_clones":0,"num_object_copies":9496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4748,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13755,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":658}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15897","reported_seq":24361,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:49.186395+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:49.186395+0000","last_peered":"2024-09-15T21:27:49.186395+0000","last_clean":"2024-09-15T21:27:49.186395+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:49.186395+0000","last_undegraded":"2024-09-15T21:27:49.186395+0000","last_fullsized":"2024-09-15T21:27:49.186395+0000","mapping_epoch":44,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19099648,"num_objects":4663,"num_object_clones":0,"num_object_copies":9326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13590,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15943","reported_seq":16082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.343346+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:48.343346+0000","last_peered":"2024-09-15T21:27:48.343346+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:48.343346+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19427328,"num_objects":4743,"num_object_clones":0,"num_object_copies":9486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12887,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":649}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15913","reported_seq":16050,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.392962+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:48.392962+0000","last_peered":"2024-09-15T21:27:48.392962+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:48.392962+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19156992,"num_objects":4677,"num_object_clones":0,"num_object_copies":9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12822,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":538}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15750","reported_seq":24065,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:49.477090+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:49.477090+0000","last_peered":"2024-09-15T21:27:49.477090+0000","last_clean":"2024-09-15T21:27:49.477090+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:49.477090+0000","last_undegraded":"2024-09-15T21:27:49.477090+0000","last_fullsized":"2024-09-15T21:27:49.477090+0000","mapping_epoch":46,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18997248,"num_objects":4638,"num_object_clones":0,"num_object_copies":9276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12234,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16083","reported_seq":16227,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:48.381406+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:48.381406+0000","last_peered":"2024-09-15T21:27:48.381406+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:48.381406+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19592134,"num_objects":4785,"num_object_clones":0,"num_object_copies":9570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12787,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":638},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":310227702,"num_objects":75749,"num_object_clones":0,"num_object_copies":151498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40412,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75749,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":214706,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9062,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":822054912,"data_stored":821909474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603836,"kb_used_data":283372,"kb_used_omap":4,"kb_used_meta":320443,"kb_avail":93768004,"statfs":{"total":96636764160,"available":96018436096,"internally_reserved":0,"allocated":290172928,"data_stored":289887670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":328134313},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709668,"kb_used_data":294840,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93662172,"statfs":{"total":96636764160,"available":95910064128,"internally_reserved":0,"allocated":301916160,"data_stored":301630897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592940,"kb_used_data":228048,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93778900,"statfs":{"total":96636764160,"available":96029593600,"internally_reserved":0,"allocated":233521152,"data_stored":233202913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373616303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,5,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":301101056,"data_stored":301064926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":232054784,"data_stored":231985678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288899072,"data_stored":288858870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:57.497 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:58.110 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:58.112 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:58.144 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-15T21:27:56.822379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":300552969,"num_objects":73389,"num_object_clones":0,"num_object_copies":146778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40307,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73389,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":217070,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9168,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160878,"ondisk_log_size":160878,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1883152,"kb_used_data":787160,"kb_used_omap":12,"kb_used_meta":1095923,"kb_avail":281232368,"statfs":{"total":289910292480,"available":287981944832,"internally_reserved":0,"allocated":806051840,"data_stored":805163080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1122225155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,6,3,10,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19165184,"num_objects":-4679,"num_object_clones":0,"num_object_copies":-9358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4679,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004582"},"pg_stats":[{"pgid":"2.f","version":"47'16360","reported_seq":16439,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.489230+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:53.489230+0000","last_peered":"2024-09-15T21:27:53.489230+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:53.489230+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19628032,"num_objects":4792,"num_object_clones":0,"num_object_copies":9584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16360,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":660}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16088","reported_seq":16199,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:54.081235+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:54.081235+0000","last_peered":"2024-09-15T21:27:54.081235+0000","last_clean":"2024-09-15T21:27:54.081235+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:54.081235+0000","last_undegraded":"2024-09-15T21:27:54.081235+0000","last_fullsized":"2024-09-15T21:27:54.081235+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18785222,"num_objects":4588,"num_object_clones":0,"num_object_copies":9176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4588,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14369,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16477","reported_seq":18446,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:54.214058+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:54.214058+0000","last_peered":"2024-09-15T21:27:54.214058+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:54.214058+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956312,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14577,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":918,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15840","reported_seq":15916,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:54.149120+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:54.149120+0000","last_peered":"2024-09-15T21:27:54.149120+0000","last_clean":"2024-09-15T21:27:54.149120+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:54.149120+0000","last_undegraded":"2024-09-15T21:27:54.149120+0000","last_fullsized":"2024-09-15T21:27:54.149120+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18325504,"num_objects":4474,"num_object_clones":0,"num_object_copies":8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15840,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15917","reported_seq":16055,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.429301+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:53.429301+0000","last_peered":"2024-09-15T21:27:53.429301+0000","last_clean":"2024-09-15T21:27:53.429301+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:53.429301+0000","last_undegraded":"2024-09-15T21:27:53.429301+0000","last_fullsized":"2024-09-15T21:27:53.429301+0000","mapping_epoch":34,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18792931,"num_objects":4589,"num_object_clones":0,"num_object_copies":9178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12372,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15991","reported_seq":16133,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.444334+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:53.444334+0000","last_peered":"2024-09-15T21:27:53.444334+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:53.444334+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19022307,"num_objects":4645,"num_object_clones":0,"num_object_copies":9290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4645,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12667,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":569},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16126","reported_seq":16268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.416099+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:53.416099+0000","last_peered":"2024-09-15T21:27:53.416099+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:53.416099+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18613190,"num_objects":4546,"num_object_clones":0,"num_object_copies":9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12437,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":385},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16313","reported_seq":16447,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.341691+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:53.341691+0000","last_peered":"2024-09-15T21:27:53.341691+0000","last_clean":"2024-09-15T21:27:53.341691+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:53.341691+0000","last_undegraded":"2024-09-15T21:27:53.341691+0000","last_fullsized":"2024-09-15T21:27:53.341691+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18657280,"num_objects":4555,"num_object_clones":0,"num_object_copies":9110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13085,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15954","reported_seq":16086,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:54.151442+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:54.151442+0000","last_peered":"2024-09-15T21:27:54.151442+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:54.151442+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18674147,"num_objects":4560,"num_object_clones":0,"num_object_copies":9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13886,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":473}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15895","reported_seq":16031,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.459797+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:53.459797+0000","last_peered":"2024-09-15T21:27:53.459797+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:53.459797+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18788352,"num_objects":4587,"num_object_clones":0,"num_object_copies":9174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12568,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16025","reported_seq":16149,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:54.131242+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:54.131242+0000","last_peered":"2024-09-15T21:27:54.131242+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:54.131242+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18915811,"num_objects":4619,"num_object_clones":0,"num_object_copies":9238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4619,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13884,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":529}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16058","reported_seq":24522,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:54.077666+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:54.077666+0000","last_peered":"2024-09-15T21:27:54.077666+0000","last_clean":"2024-09-15T21:27:54.077666+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:54.077666+0000","last_undegraded":"2024-09-15T21:27:54.077666+0000","last_fullsized":"2024-09-15T21:27:54.077666+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18440192,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13751,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16095","reported_seq":16233,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.482351+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:53.482351+0000","last_peered":"2024-09-15T21:27:53.482351+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:53.482351+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18804736,"num_objects":4591,"num_object_clones":0,"num_object_copies":9182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13039,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16034","reported_seq":16172,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.459547+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:53.459547+0000","last_peered":"2024-09-15T21:27:53.459547+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:53.459547+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18661376,"num_objects":4556,"num_object_clones":0,"num_object_copies":9112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12943,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15891","reported_seq":24205,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:54.618710+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:54.618710+0000","last_peered":"2024-09-15T21:27:54.618710+0000","last_clean":"2024-09-15T21:27:54.618710+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:54.618710+0000","last_undegraded":"2024-09-15T21:27:54.618710+0000","last_fullsized":"2024-09-15T21:27:54.618710+0000","mapping_epoch":46,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18419712,"num_objects":4497,"num_object_clones":0,"num_object_copies":8994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12375,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16211","reported_seq":16355,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.383897+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:53.383897+0000","last_peered":"2024-09-15T21:27:53.383897+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:53.383897+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19067846,"num_objects":4657,"num_object_clones":0,"num_object_copies":9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12915,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":510},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":300552950,"num_objects":73387,"num_object_clones":0,"num_object_copies":146774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40307,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73387,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":217068,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9167,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802783232,"data_stored":802637794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603056,"kb_used_data":278432,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93768784,"statfs":{"total":96636764160,"available":96019234816,"internally_reserved":0,"allocated":285114368,"data_stored":284829110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":332394153},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706716,"kb_used_data":287748,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93665124,"statfs":{"total":96636764160,"available":95913086976,"internally_reserved":0,"allocated":294653952,"data_stored":294368689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573380,"kb_used_data":220980,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93798460,"statfs":{"total":96636764160,"available":96049623040,"internally_reserved":0,"allocated":226283520,"data_stored":225965281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":293888000,"data_stored":293851870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224948224,"data_stored":224879118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":283947008,"data_stored":283906806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:58.145 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:27:58.718 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:58.718 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:27:58.742 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-15T21:27:56.822379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":300552969,"num_objects":73389,"num_object_clones":0,"num_object_copies":146778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40307,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73389,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":217070,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9168,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160878,"ondisk_log_size":160878,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1883152,"kb_used_data":787160,"kb_used_omap":12,"kb_used_meta":1095923,"kb_avail":281232368,"statfs":{"total":289910292480,"available":287981944832,"internally_reserved":0,"allocated":806051840,"data_stored":805163080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1122225155},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,6,3,10,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19165184,"num_objects":-4679,"num_object_clones":0,"num_object_copies":-9358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4679,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4679,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004582"},"pg_stats":[{"pgid":"2.f","version":"47'16360","reported_seq":16439,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.489230+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:53.489230+0000","last_peered":"2024-09-15T21:27:53.489230+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:53.489230+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19628032,"num_objects":4792,"num_object_clones":0,"num_object_copies":9584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16360,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":660}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16088","reported_seq":16199,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:54.081235+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:54.081235+0000","last_peered":"2024-09-15T21:27:54.081235+0000","last_clean":"2024-09-15T21:27:54.081235+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:54.081235+0000","last_undegraded":"2024-09-15T21:27:54.081235+0000","last_fullsized":"2024-09-15T21:27:54.081235+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18785222,"num_objects":4588,"num_object_clones":0,"num_object_copies":9176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4588,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14369,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16477","reported_seq":18446,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:54.214058+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:54.214058+0000","last_peered":"2024-09-15T21:27:54.214058+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:54.214058+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956312,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14577,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":918,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15840","reported_seq":15916,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:54.149120+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:54.149120+0000","last_peered":"2024-09-15T21:27:54.149120+0000","last_clean":"2024-09-15T21:27:54.149120+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:54.149120+0000","last_undegraded":"2024-09-15T21:27:54.149120+0000","last_fullsized":"2024-09-15T21:27:54.149120+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18325504,"num_objects":4474,"num_object_clones":0,"num_object_copies":8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15840,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15917","reported_seq":16055,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.429301+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:53.429301+0000","last_peered":"2024-09-15T21:27:53.429301+0000","last_clean":"2024-09-15T21:27:53.429301+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:53.429301+0000","last_undegraded":"2024-09-15T21:27:53.429301+0000","last_fullsized":"2024-09-15T21:27:53.429301+0000","mapping_epoch":34,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18792931,"num_objects":4589,"num_object_clones":0,"num_object_copies":9178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12372,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15991","reported_seq":16133,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.444334+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:53.444334+0000","last_peered":"2024-09-15T21:27:53.444334+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:53.444334+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19022307,"num_objects":4645,"num_object_clones":0,"num_object_copies":9290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4645,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12667,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":569},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16126","reported_seq":16268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.416099+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:53.416099+0000","last_peered":"2024-09-15T21:27:53.416099+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:53.416099+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18613190,"num_objects":4546,"num_object_clones":0,"num_object_copies":9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12437,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":385},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16313","reported_seq":16447,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:53.341691+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:53.341691+0000","last_peered":"2024-09-15T21:27:53.341691+0000","last_clean":"2024-09-15T21:27:53.341691+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:53.341691+0000","last_undegraded":"2024-09-15T21:27:53.341691+0000","last_fullsized":"2024-09-15T21:27:53.341691+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18657280,"num_objects":4555,"num_object_clones":0,"num_object_copies":9110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13085,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15954","reported_seq":16086,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:54.151442+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:54.151442+0000","last_peered":"2024-09-15T21:27:54.151442+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:54.151442+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18674147,"num_objects":4560,"num_object_clones":0,"num_object_copies":9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13886,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":473}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15895","reported_seq":16031,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.459797+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:53.459797+0000","last_peered":"2024-09-15T21:27:53.459797+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:53.459797+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18788352,"num_objects":4587,"num_object_clones":0,"num_object_copies":9174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12568,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":567}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16025","reported_seq":16149,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:54.131242+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:54.131242+0000","last_peered":"2024-09-15T21:27:54.131242+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:54.131242+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18915811,"num_objects":4619,"num_object_clones":0,"num_object_copies":9238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4619,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13884,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":529}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16058","reported_seq":24522,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:54.077666+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:54.077666+0000","last_peered":"2024-09-15T21:27:54.077666+0000","last_clean":"2024-09-15T21:27:54.077666+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:54.077666+0000","last_undegraded":"2024-09-15T21:27:54.077666+0000","last_fullsized":"2024-09-15T21:27:54.077666+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18440192,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13751,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16095","reported_seq":16233,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.482351+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:53.482351+0000","last_peered":"2024-09-15T21:27:53.482351+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:53.482351+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18804736,"num_objects":4591,"num_object_clones":0,"num_object_copies":9182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13039,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16034","reported_seq":16172,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.459547+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:53.459547+0000","last_peered":"2024-09-15T21:27:53.459547+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:53.459547+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18661376,"num_objects":4556,"num_object_clones":0,"num_object_copies":9112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12943,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":417}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15891","reported_seq":24205,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:54.618710+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:54.618710+0000","last_peered":"2024-09-15T21:27:54.618710+0000","last_clean":"2024-09-15T21:27:54.618710+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:54.618710+0000","last_undegraded":"2024-09-15T21:27:54.618710+0000","last_fullsized":"2024-09-15T21:27:54.618710+0000","mapping_epoch":46,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18419712,"num_objects":4497,"num_object_clones":0,"num_object_copies":8994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12375,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16211","reported_seq":16355,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:53.383897+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:53.383897+0000","last_peered":"2024-09-15T21:27:53.383897+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:53.383897+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19067846,"num_objects":4657,"num_object_clones":0,"num_object_copies":9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12915,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":510},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":300552950,"num_objects":73387,"num_object_clones":0,"num_object_copies":146774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40307,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73387,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":217068,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9167,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802783232,"data_stored":802637794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603056,"kb_used_data":278432,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93768784,"statfs":{"total":96636764160,"available":96019234816,"internally_reserved":0,"allocated":285114368,"data_stored":284829110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":332394153},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706716,"kb_used_data":287748,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93665124,"statfs":{"total":96636764160,"available":95913086976,"internally_reserved":0,"allocated":294653952,"data_stored":294368689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573380,"kb_used_data":220980,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93798460,"statfs":{"total":96636764160,"available":96049623040,"internally_reserved":0,"allocated":226283520,"data_stored":225965281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":293888000,"data_stored":293851870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224948224,"data_stored":224879118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":283947008,"data_stored":283906806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:27:58.743 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:27:58.743 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16360", 'reported_seq': 16439, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.489230+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:27:53.489230+0000', 'last_peered': '2024-09-15T21:27:53.489230+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:27:53.489230+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19628032, 'num_objects': 4792, 'num_object_clones': 0, 'num_object_copies': 9584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4792, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16360, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 660}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:58.743 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:27:58.743 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'16477", 'reported_seq': 18446, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:54.214058+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:27:54.214058+0000', 'last_peered': '2024-09-15T21:27:54.214058+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:27:54.214058+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18956312, 'num_objects': 4629, 'num_object_clones': 0, 'num_object_copies': 9258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3350, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4629, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14577, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 918, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1279}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:58.743 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:27:58.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'15991", 'reported_seq': 16133, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.444334+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:27:53.444334+0000', 'last_peered': '2024-09-15T21:27:53.444334+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:27:53.444334+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19022307, 'num_objects': 4645, 'num_object_clones': 0, 'num_object_copies': 9290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4645, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12667, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 569}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:58.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:27:58.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16126", 'reported_seq': 16268, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.416099+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:27:53.416099+0000', 'last_peered': '2024-09-15T21:27:53.416099+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:27:53.416099+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18613190, 'num_objects': 4546, 'num_object_clones': 0, 'num_object_copies': 9092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4546, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12437, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 385}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:58.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:27:58.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'15954", 'reported_seq': 16086, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:54.151442+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:27:54.151442+0000', 'last_peered': '2024-09-15T21:27:54.151442+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:27:54.151442+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18674147, 'num_objects': 4560, 'num_object_clones': 0, 'num_object_copies': 9120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4560, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13886, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 473}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:58.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:27:58.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'15895", 'reported_seq': 16031, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.459797+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:27:53.459797+0000', 'last_peered': '2024-09-15T21:27:53.459797+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:27:53.459797+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18788352, 'num_objects': 4587, 'num_object_clones': 0, 'num_object_copies': 9174, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4587, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12568, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 567}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:58.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:27:58.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16025", 'reported_seq': 16149, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:54.131242+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:27:54.131242+0000', 'last_peered': '2024-09-15T21:27:54.131242+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:27:54.131242+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18915811, 'num_objects': 4619, 'num_object_clones': 0, 'num_object_copies': 9238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4619, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13884, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 529}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:27:58.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:27:58.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16095", 'reported_seq': 16233, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.482351+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:27:53.482351+0000', 'last_peered': '2024-09-15T21:27:53.482351+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:27:53.482351+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18804736, 'num_objects': 4591, 'num_object_clones': 0, 'num_object_copies': 9182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4591, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13039, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 497}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:58.745 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:27:58.745 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'16034", 'reported_seq': 16172, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.459547+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:27:53.459547+0000', 'last_peered': '2024-09-15T21:27:53.459547+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:27:53.459547+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18661376, 'num_objects': 4556, 'num_object_clones': 0, 'num_object_copies': 9112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4556, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12943, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 417}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:58.745 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:27:58.745 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'16211", 'reported_seq': 16355, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:53.383897+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:27:53.383897+0000', 'last_peered': '2024-09-15T21:27:53.383897+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:27:53.383897+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19067846, 'num_objects': 4657, 'num_object_clones': 0, 'num_object_copies': 9314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4657, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12915, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 510}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:27:58.745 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:27:59.455 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:27:59.455 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40307/144140 objects degraded (27.964%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":531,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":72070,"data_bytes":295150345,"bytes_used":1931046912,"bytes_avail":287979245568,"bytes_total":289910292480,"degraded_objects":40307,"degraded_total":144140,"degraded_ratio":0.27963785208824754,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":499},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:27:59.480 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:27:59.481 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:00.048 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:00.051 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:00.072 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-15T21:27:58.823163+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":295150345,"num_objects":72070,"num_object_clones":0,"num_object_copies":144140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40307,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72070,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":218389,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9168,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160897,"ondisk_log_size":160897,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1885788,"kb_used_data":780060,"kb_used_omap":12,"kb_used_meta":1105651,"kb_avail":281229732,"statfs":{"total":289910292480,"available":287979245568,"internally_reserved":0,"allocated":798781440,"data_stored":797892680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1132186627},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,7,5,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24567808,"num_objects":-5998,"num_object_clones":0,"num_object_copies":-11996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5998,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004705"},"pg_stats":[{"pgid":"2.f","version":"47'16523","reported_seq":16604,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.473941+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:58.473941+0000","last_peered":"2024-09-15T21:27:58.473941+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:58.473941+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18960384,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16523,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":497}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16088","reported_seq":16199,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:54.081235+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:54.081235+0000","last_peered":"2024-09-15T21:27:54.081235+0000","last_clean":"2024-09-15T21:27:54.081235+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:54.081235+0000","last_undegraded":"2024-09-15T21:27:54.081235+0000","last_fullsized":"2024-09-15T21:27:54.081235+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18785222,"num_objects":4588,"num_object_clones":0,"num_object_copies":9176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4588,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14369,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16477","reported_seq":18446,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:54.214058+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:54.214058+0000","last_peered":"2024-09-15T21:27:54.214058+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:54.214058+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956312,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3350,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14577,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":918,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1279}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15840","reported_seq":15916,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:54.149120+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:54.149120+0000","last_peered":"2024-09-15T21:27:54.149120+0000","last_clean":"2024-09-15T21:27:54.149120+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:54.149120+0000","last_undegraded":"2024-09-15T21:27:54.149120+0000","last_fullsized":"2024-09-15T21:27:54.149120+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18325504,"num_objects":4474,"num_object_clones":0,"num_object_copies":8948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15840,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16064","reported_seq":16202,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:58.441807+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:58.441807+0000","last_peered":"2024-09-15T21:27:58.441807+0000","last_clean":"2024-09-15T21:27:58.441807+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:58.441807+0000","last_undegraded":"2024-09-15T21:27:58.441807+0000","last_fullsized":"2024-09-15T21:27:58.441807+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18190819,"num_objects":4442,"num_object_clones":0,"num_object_copies":8884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12519,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16134","reported_seq":16276,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.489003+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:58.489003+0000","last_peered":"2024-09-15T21:27:58.489003+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:58.489003+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18436579,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12810,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":426},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16286","reported_seq":16428,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.433241+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:58.433241+0000","last_peered":"2024-09-15T21:27:58.433241+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:58.433241+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17957830,"num_objects":4386,"num_object_clones":0,"num_object_copies":8772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12597,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":225},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16466","reported_seq":16600,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:58.430034+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:58.430034+0000","last_peered":"2024-09-15T21:27:58.430034+0000","last_clean":"2024-09-15T21:27:58.430034+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:58.430034+0000","last_undegraded":"2024-09-15T21:27:58.430034+0000","last_fullsized":"2024-09-15T21:27:58.430034+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18030592,"num_objects":4402,"num_object_clones":0,"num_object_copies":8804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13238,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15954","reported_seq":16086,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:54.151442+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:54.151442+0000","last_peered":"2024-09-15T21:27:54.151442+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:54.151442+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18674147,"num_objects":4560,"num_object_clones":0,"num_object_copies":9120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4560,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13886,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":473}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16033","reported_seq":16169,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.457447+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:58.457447+0000","last_peered":"2024-09-15T21:27:58.457447+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:58.457447+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18223104,"num_objects":4449,"num_object_clones":0,"num_object_copies":8898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12706,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":429}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16025","reported_seq":16149,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:54.131242+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:54.131242+0000","last_peered":"2024-09-15T21:27:54.131242+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:54.131242+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18915811,"num_objects":4619,"num_object_clones":0,"num_object_copies":9238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4619,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13884,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":529}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16058","reported_seq":24522,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:54.077666+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:54.077666+0000","last_peered":"2024-09-15T21:27:54.077666+0000","last_clean":"2024-09-15T21:27:54.077666+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:54.077666+0000","last_undegraded":"2024-09-15T21:27:54.077666+0000","last_fullsized":"2024-09-15T21:27:54.077666+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18440192,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13751,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16243","reported_seq":16382,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.458461+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:58.458461+0000","last_peered":"2024-09-15T21:27:58.458461+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:58.458461+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18198528,"num_objects":4443,"num_object_clones":0,"num_object_copies":8886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13187,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16164","reported_seq":16302,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.379729+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:58.379729+0000","last_peered":"2024-09-15T21:27:58.379729+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:58.379729+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18128896,"num_objects":4426,"num_object_clones":0,"num_object_copies":8852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13073,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15891","reported_seq":24205,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:54.618710+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:54.618710+0000","last_peered":"2024-09-15T21:27:54.618710+0000","last_clean":"2024-09-15T21:27:54.618710+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:54.618710+0000","last_undegraded":"2024-09-15T21:27:54.618710+0000","last_fullsized":"2024-09-15T21:27:54.618710+0000","mapping_epoch":46,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18419712,"num_objects":4497,"num_object_clones":0,"num_object_copies":8994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12375,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16348","reported_seq":16492,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.425199+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:58.425199+0000","last_peered":"2024-09-15T21:27:58.425199+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:58.425199+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18506694,"num_objects":4520,"num_object_clones":0,"num_object_copies":9040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4520,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13052,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":373},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":295150326,"num_objects":72068,"num_object_clones":0,"num_object_copies":144136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40307,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72068,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":218387,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9167,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":795500544,"data_stored":795355106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160894,"ondisk_log_size":160894,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603056,"kb_used_data":278432,"kb_used_omap":4,"kb_used_meta":324603,"kb_avail":93768784,"statfs":{"total":96636764160,"available":96019234816,"internally_reserved":0,"allocated":285114368,"data_stored":284829110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":332394153},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709352,"kb_used_data":280648,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93662488,"statfs":{"total":96636764160,"available":95910387712,"internally_reserved":0,"allocated":287383552,"data_stored":287098289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,5,0,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573380,"kb_used_data":220980,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93798460,"statfs":{"total":96636764160,"available":96049623040,"internally_reserved":0,"allocated":226283520,"data_stored":225965281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360836783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":286605312,"data_stored":286569182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224948224,"data_stored":224879118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":283947008,"data_stored":283906806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:03.075 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:03.588 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:03.590 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:03.615 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-15T21:28:02.824462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":290743049,"num_objects":70994,"num_object_clones":0,"num_object_copies":141988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70994,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":219465,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9269,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1904632,"kb_used_data":768360,"kb_used_omap":12,"kb_used_meta":1136179,"kb_avail":281210888,"statfs":{"total":289910292480,"available":287959949312,"internally_reserved":0,"allocated":786800640,"data_stored":785911880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1163447299},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,1,6,6,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19484672,"num_objects":-4757,"num_object_clones":0,"num_object_copies":-9514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4757,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004491"},"pg_stats":[{"pgid":"2.f","version":"47'16523","reported_seq":16604,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.473941+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:58.473941+0000","last_peered":"2024-09-15T21:27:58.473941+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:58.473941+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18960384,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16523,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":497}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16253","reported_seq":16364,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:59.067445+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:59.067445+0000","last_peered":"2024-09-15T21:27:59.067445+0000","last_clean":"2024-09-15T21:27:59.067445+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:59.067445+0000","last_undegraded":"2024-09-15T21:27:59.067445+0000","last_fullsized":"2024-09-15T21:27:59.067445+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18109382,"num_objects":4423,"num_object_clones":0,"num_object_copies":8846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4423,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14534,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16631","reported_seq":18802,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:59.177357+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:59.177357+0000","last_peered":"2024-09-15T21:27:59.177357+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:59.177357+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18325528,"num_objects":4475,"num_object_clones":0,"num_object_copies":8950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4475,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14731,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1019,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1226}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15978","reported_seq":16054,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:59.164236+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:59.164236+0000","last_peered":"2024-09-15T21:27:59.164236+0000","last_clean":"2024-09-15T21:27:59.164236+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:59.164236+0000","last_undegraded":"2024-09-15T21:27:59.164236+0000","last_fullsized":"2024-09-15T21:27:59.164236+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15978,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16064","reported_seq":16202,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:58.441807+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:58.441807+0000","last_peered":"2024-09-15T21:27:58.441807+0000","last_clean":"2024-09-15T21:27:58.441807+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:58.441807+0000","last_undegraded":"2024-09-15T21:27:58.441807+0000","last_fullsized":"2024-09-15T21:27:58.441807+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18190819,"num_objects":4442,"num_object_clones":0,"num_object_copies":8884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12519,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16134","reported_seq":16276,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.489003+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:58.489003+0000","last_peered":"2024-09-15T21:27:58.489003+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:58.489003+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18436579,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12810,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":426},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16286","reported_seq":16428,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.433241+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:58.433241+0000","last_peered":"2024-09-15T21:27:58.433241+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:58.433241+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17957830,"num_objects":4386,"num_object_clones":0,"num_object_copies":8772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12597,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":225},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16466","reported_seq":16600,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:58.430034+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:58.430034+0000","last_peered":"2024-09-15T21:27:58.430034+0000","last_clean":"2024-09-15T21:27:58.430034+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:58.430034+0000","last_undegraded":"2024-09-15T21:27:58.430034+0000","last_fullsized":"2024-09-15T21:27:58.430034+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18030592,"num_objects":4402,"num_object_clones":0,"num_object_copies":8804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13238,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16121","reported_seq":16253,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:59.110003+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:59.110003+0000","last_peered":"2024-09-15T21:27:59.110003+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:59.110003+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17990115,"num_objects":4393,"num_object_clones":0,"num_object_copies":8786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14053,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16033","reported_seq":16169,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.457447+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:58.457447+0000","last_peered":"2024-09-15T21:27:58.457447+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:58.457447+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18223104,"num_objects":4449,"num_object_clones":0,"num_object_copies":8898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12706,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":429}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16180","reported_seq":16304,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:59.253767+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:59.253767+0000","last_peered":"2024-09-15T21:27:59.253767+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:59.253767+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18280931,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14039,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":374}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16204","reported_seq":24668,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:59.211191+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:59.211191+0000","last_peered":"2024-09-15T21:27:59.211191+0000","last_clean":"2024-09-15T21:27:59.211191+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:59.211191+0000","last_undegraded":"2024-09-15T21:27:59.211191+0000","last_fullsized":"2024-09-15T21:27:59.211191+0000","mapping_epoch":44,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17842176,"num_objects":4356,"num_object_clones":0,"num_object_copies":8712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13897,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16243","reported_seq":16382,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.458461+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:58.458461+0000","last_peered":"2024-09-15T21:27:58.458461+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:58.458461+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18198528,"num_objects":4443,"num_object_clones":0,"num_object_copies":8886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13187,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16164","reported_seq":16302,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.379729+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:58.379729+0000","last_peered":"2024-09-15T21:27:58.379729+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:58.379729+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18128896,"num_objects":4426,"num_object_clones":0,"num_object_copies":8852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13073,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16042","reported_seq":24357,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:59.632414+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:59.632414+0000","last_peered":"2024-09-15T21:27:59.632414+0000","last_clean":"2024-09-15T21:27:59.632414+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:59.632414+0000","last_undegraded":"2024-09-15T21:27:59.632414+0000","last_fullsized":"2024-09-15T21:27:59.632414+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12526,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16348","reported_seq":16492,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.425199+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:58.425199+0000","last_peered":"2024-09-15T21:27:58.425199+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:58.425199+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18506694,"num_objects":4520,"num_object_clones":0,"num_object_copies":9040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4520,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13052,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":373},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":290743030,"num_objects":70992,"num_object_clones":0,"num_object_copies":141984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70992,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":219463,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9268,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":783142912,"data_stored":782997474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602480,"kb_used_data":273692,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93769360,"statfs":{"total":96636764160,"available":96019824640,"internally_reserved":0,"allocated":280260608,"data_stored":279975350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":336653993},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709352,"kb_used_data":280648,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93662488,"statfs":{"total":96636764160,"available":95910387712,"internally_reserved":0,"allocated":287383552,"data_stored":287098289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,5,0,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592800,"kb_used_data":214020,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93779040,"statfs":{"total":96636764160,"available":96029736960,"internally_reserved":0,"allocated":219156480,"data_stored":218838241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387837615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":286605312,"data_stored":286569182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":217579520,"data_stored":217510414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278958080,"data_stored":278917878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:03.617 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:04.153 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:04.155 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:04.184 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-15T21:28:02.824462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":290743049,"num_objects":70994,"num_object_clones":0,"num_object_copies":141988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70994,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":219465,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9269,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160773,"ondisk_log_size":160773,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1904632,"kb_used_data":768360,"kb_used_omap":12,"kb_used_meta":1136179,"kb_avail":281210888,"statfs":{"total":289910292480,"available":287959949312,"internally_reserved":0,"allocated":786800640,"data_stored":785911880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1163447299},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,1,6,6,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19484672,"num_objects":-4757,"num_object_clones":0,"num_object_copies":-9514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4757,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004491"},"pg_stats":[{"pgid":"2.f","version":"47'16523","reported_seq":16604,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.473941+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:27:58.473941+0000","last_peered":"2024-09-15T21:27:58.473941+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:27:58.473941+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18960384,"num_objects":4629,"num_object_clones":0,"num_object_copies":9258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16523,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":497}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16253","reported_seq":16364,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:59.067445+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:27:59.067445+0000","last_peered":"2024-09-15T21:27:59.067445+0000","last_clean":"2024-09-15T21:27:59.067445+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:27:59.067445+0000","last_undegraded":"2024-09-15T21:27:59.067445+0000","last_fullsized":"2024-09-15T21:27:59.067445+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18109382,"num_objects":4423,"num_object_clones":0,"num_object_copies":8846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4423,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14534,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16631","reported_seq":18802,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:59.177357+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:27:59.177357+0000","last_peered":"2024-09-15T21:27:59.177357+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:27:59.177357+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18325528,"num_objects":4475,"num_object_clones":0,"num_object_copies":8950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4475,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14731,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1019,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1226}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15978","reported_seq":16054,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:59.164236+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:27:59.164236+0000","last_peered":"2024-09-15T21:27:59.164236+0000","last_clean":"2024-09-15T21:27:59.164236+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:27:59.164236+0000","last_undegraded":"2024-09-15T21:27:59.164236+0000","last_fullsized":"2024-09-15T21:27:59.164236+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17760256,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15978,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16064","reported_seq":16202,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:58.441807+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:27:58.441807+0000","last_peered":"2024-09-15T21:27:58.441807+0000","last_clean":"2024-09-15T21:27:58.441807+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:27:58.441807+0000","last_undegraded":"2024-09-15T21:27:58.441807+0000","last_fullsized":"2024-09-15T21:27:58.441807+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18190819,"num_objects":4442,"num_object_clones":0,"num_object_copies":8884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12519,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16134","reported_seq":16276,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.489003+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:27:58.489003+0000","last_peered":"2024-09-15T21:27:58.489003+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:27:58.489003+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18436579,"num_objects":4502,"num_object_clones":0,"num_object_copies":9004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12810,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":426},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16286","reported_seq":16428,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.433241+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:27:58.433241+0000","last_peered":"2024-09-15T21:27:58.433241+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:27:58.433241+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17957830,"num_objects":4386,"num_object_clones":0,"num_object_copies":8772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12597,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":225},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16466","reported_seq":16600,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:58.430034+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:27:58.430034+0000","last_peered":"2024-09-15T21:27:58.430034+0000","last_clean":"2024-09-15T21:27:58.430034+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:27:58.430034+0000","last_undegraded":"2024-09-15T21:27:58.430034+0000","last_fullsized":"2024-09-15T21:27:58.430034+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18030592,"num_objects":4402,"num_object_clones":0,"num_object_copies":8804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4402,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13238,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16121","reported_seq":16253,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:59.110003+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:27:59.110003+0000","last_peered":"2024-09-15T21:27:59.110003+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:27:59.110003+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17990115,"num_objects":4393,"num_object_clones":0,"num_object_copies":8786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14053,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":306}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16033","reported_seq":16169,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.457447+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:27:58.457447+0000","last_peered":"2024-09-15T21:27:58.457447+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:27:58.457447+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18223104,"num_objects":4449,"num_object_clones":0,"num_object_copies":8898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12706,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":429}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16180","reported_seq":16304,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:59.253767+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:27:59.253767+0000","last_peered":"2024-09-15T21:27:59.253767+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:27:59.253767+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18280931,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14039,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":374}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16204","reported_seq":24668,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:59.211191+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:27:59.211191+0000","last_peered":"2024-09-15T21:27:59.211191+0000","last_clean":"2024-09-15T21:27:59.211191+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:27:59.211191+0000","last_undegraded":"2024-09-15T21:27:59.211191+0000","last_fullsized":"2024-09-15T21:27:59.211191+0000","mapping_epoch":44,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17842176,"num_objects":4356,"num_object_clones":0,"num_object_copies":8712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13897,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16243","reported_seq":16382,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.458461+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:27:58.458461+0000","last_peered":"2024-09-15T21:27:58.458461+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:27:58.458461+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18198528,"num_objects":4443,"num_object_clones":0,"num_object_copies":8886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13187,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16164","reported_seq":16302,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.379729+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:27:58.379729+0000","last_peered":"2024-09-15T21:27:58.379729+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:27:58.379729+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18128896,"num_objects":4426,"num_object_clones":0,"num_object_copies":8852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13073,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16042","reported_seq":24357,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:59.632414+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:27:59.632414+0000","last_peered":"2024-09-15T21:27:59.632414+0000","last_clean":"2024-09-15T21:27:59.632414+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:27:59.632414+0000","last_undegraded":"2024-09-15T21:27:59.632414+0000","last_fullsized":"2024-09-15T21:27:59.632414+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12526,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16348","reported_seq":16492,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:27:58.425199+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:27:58.425199+0000","last_peered":"2024-09-15T21:27:58.425199+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:27:58.425199+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18506694,"num_objects":4520,"num_object_clones":0,"num_object_copies":9040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4520,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13052,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":373},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":290743030,"num_objects":70992,"num_object_clones":0,"num_object_copies":141984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70992,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":219463,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9268,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":783142912,"data_stored":782997474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602480,"kb_used_data":273692,"kb_used_omap":4,"kb_used_meta":328763,"kb_avail":93769360,"statfs":{"total":96636764160,"available":96019824640,"internally_reserved":0,"allocated":280260608,"data_stored":279975350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":336653993},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709352,"kb_used_data":280648,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93662488,"statfs":{"total":96636764160,"available":95910387712,"internally_reserved":0,"allocated":287383552,"data_stored":287098289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,5,0,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592800,"kb_used_data":214020,"kb_used_omap":4,"kb_used_meta":378747,"kb_avail":93779040,"statfs":{"total":96636764160,"available":96029736960,"internally_reserved":0,"allocated":219156480,"data_stored":218838241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387837615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":286605312,"data_stored":286569182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":217579520,"data_stored":217510414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":278958080,"data_stored":278917878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:04.187 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:28:04.187 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16523", 'reported_seq': 16604, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:58.473941+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:27:58.473941+0000', 'last_peered': '2024-09-15T21:27:58.473941+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:27:58.473941+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18960384, 'num_objects': 4629, 'num_object_clones': 0, 'num_object_copies': 9258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4629, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16523, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 497}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:04.187 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:28:04.187 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'16631", 'reported_seq': 18802, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:59.177357+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:27:59.177357+0000', 'last_peered': '2024-09-15T21:27:59.177357+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:27:59.177357+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18325528, 'num_objects': 4475, 'num_object_clones': 0, 'num_object_copies': 8950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3249, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4475, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14731, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1019, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:04.188 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:04.188 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'16134", 'reported_seq': 16276, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:58.489003+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:27:58.489003+0000', 'last_peered': '2024-09-15T21:27:58.489003+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:27:58.489003+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18436579, 'num_objects': 4502, 'num_object_clones': 0, 'num_object_copies': 9004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4502, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12810, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 426}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:04.188 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:04.188 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16286", 'reported_seq': 16428, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:58.433241+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:27:58.433241+0000', 'last_peered': '2024-09-15T21:27:58.433241+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:27:58.433241+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17957830, 'num_objects': 4386, 'num_object_clones': 0, 'num_object_copies': 8772, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4386, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12597, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 225}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:04.189 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:28:04.189 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'16121", 'reported_seq': 16253, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:59.110003+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:27:59.110003+0000', 'last_peered': '2024-09-15T21:27:59.110003+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:27:59.110003+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17990115, 'num_objects': 4393, 'num_object_clones': 0, 'num_object_copies': 8786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4393, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14053, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 306}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:04.189 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:04.189 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16033", 'reported_seq': 16169, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:58.457447+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:27:58.457447+0000', 'last_peered': '2024-09-15T21:27:58.457447+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:27:58.457447+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18223104, 'num_objects': 4449, 'num_object_clones': 0, 'num_object_copies': 8898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4449, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12706, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 429}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:04.189 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:28:04.190 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16180", 'reported_seq': 16304, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:59.253767+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:27:59.253767+0000', 'last_peered': '2024-09-15T21:27:59.253767+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:27:59.253767+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18280931, 'num_objects': 4464, 'num_object_clones': 0, 'num_object_copies': 8928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4464, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14039, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 374}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:04.190 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:04.190 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16243", 'reported_seq': 16382, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:58.458461+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:27:58.458461+0000', 'last_peered': '2024-09-15T21:27:58.458461+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:27:58.458461+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18198528, 'num_objects': 4443, 'num_object_clones': 0, 'num_object_copies': 8886, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4443, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13187, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 349}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:04.190 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:28:04.190 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'16164", 'reported_seq': 16302, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:58.379729+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:27:58.379729+0000', 'last_peered': '2024-09-15T21:27:58.379729+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:27:58.379729+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18128896, 'num_objects': 4426, 'num_object_clones': 0, 'num_object_copies': 8852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13073, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 287}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:04.191 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:28:04.191 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'16348", 'reported_seq': 16492, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:27:58.425199+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:27:58.425199+0000', 'last_peered': '2024-09-15T21:27:58.425199+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:27:58.425199+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18506694, 'num_objects': 4520, 'num_object_clones': 0, 'num_object_copies': 9040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4520, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13052, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 373}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:04.191 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:04.892 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:04.892 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40206/141988 objects degraded (28.316%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":536,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":70994,"data_bytes":290743049,"bytes_used":1950343168,"bytes_avail":287959949312,"bytes_total":289910292480,"degraded_objects":40206,"degraded_total":141988,"degraded_ratio":0.28316477448798488,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":396},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:04.920 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:04.920 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:05.513 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:05.515 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:05.538 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-15T21:28:04.825216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":281195273,"num_objects":68663,"num_object_clones":0,"num_object_copies":137326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68663,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":221796,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9367,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898872,"kb_used_data":748716,"kb_used_omap":12,"kb_used_meta":1150067,"kb_avail":281216648,"statfs":{"total":289910292480,"available":287965847552,"internally_reserved":0,"allocated":766685184,"data_stored":765796424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1177668611},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,0,5,5,9,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29032448,"num_objects":-7088,"num_object_clones":0,"num_object_copies":-14176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7088,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":304,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004370"},"pg_stats":[{"pgid":"2.f","version":"47'16674","reported_seq":16755,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:03.514361+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:03.514361+0000","last_peered":"2024-09-15T21:28:03.514361+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:03.514361+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18341888,"num_objects":4478,"num_object_clones":0,"num_object_copies":8956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16674,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":346}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16406","reported_seq":16516,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:04.229830+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:04.229830+0000","last_peered":"2024-09-15T21:28:04.229830+0000","last_clean":"2024-09-15T21:28:04.229830+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:04.229830+0000","last_undegraded":"2024-09-15T21:28:04.229830+0000","last_fullsized":"2024-09-15T21:28:04.229830+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17482694,"num_objects":4270,"num_object_clones":0,"num_object_copies":8540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4270,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14687,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16770","reported_seq":19136,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:04.244971+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:04.244971+0000","last_peered":"2024-09-15T21:28:04.244971+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:04.244971+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17756184,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14870,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1185}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16107","reported_seq":16183,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:04.153010+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:04.153010+0000","last_peered":"2024-09-15T21:28:04.153010+0000","last_clean":"2024-09-15T21:28:04.153010+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:04.153010+0000","last_undegraded":"2024-09-15T21:28:04.153010+0000","last_fullsized":"2024-09-15T21:28:04.153010+0000","mapping_epoch":35,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16107,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16208","reported_seq":16346,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:03.350903+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:03.350903+0000","last_peered":"2024-09-15T21:28:03.350903+0000","last_clean":"2024-09-15T21:28:03.350903+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:03.350903+0000","last_undegraded":"2024-09-15T21:28:03.350903+0000","last_fullsized":"2024-09-15T21:28:03.350903+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17600995,"num_objects":4298,"num_object_clones":0,"num_object_copies":8596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12663,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16276","reported_seq":16418,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:03.406598+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:03.406598+0000","last_peered":"2024-09-15T21:28:03.406598+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:03.406598+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17854947,"num_objects":4360,"num_object_clones":0,"num_object_copies":8720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4360,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12952,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":284},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16423","reported_seq":16564,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:03.363900+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:03.363900+0000","last_peered":"2024-09-15T21:28:03.363900+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:03.363900+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17396678,"num_objects":4249,"num_object_clones":0,"num_object_copies":8498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12734,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":88},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16605","reported_seq":16739,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:03.466031+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:03.466031+0000","last_peered":"2024-09-15T21:28:03.466031+0000","last_clean":"2024-09-15T21:28:03.466031+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:03.466031+0000","last_undegraded":"2024-09-15T21:28:03.466031+0000","last_fullsized":"2024-09-15T21:28:03.466031+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17461248,"num_objects":4263,"num_object_clones":0,"num_object_copies":8526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13377,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16264","reported_seq":16395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:04.213276+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:04.213276+0000","last_peered":"2024-09-15T21:28:04.213276+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:04.213276+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17404387,"num_objects":4250,"num_object_clones":0,"num_object_copies":8500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14196,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":163}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16170","reported_seq":16306,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:03.466955+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:03.466955+0000","last_peered":"2024-09-15T21:28:03.466955+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:03.466955+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17661952,"num_objects":4312,"num_object_clones":0,"num_object_copies":8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12843,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":292}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16332","reported_seq":16456,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:04.197919+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:04.197919+0000","last_peered":"2024-09-15T21:28:04.197919+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:04.197919+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17658339,"num_objects":4312,"num_object_clones":0,"num_object_copies":8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14191,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":222}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16381","reported_seq":24845,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:04.118226+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:04.118226+0000","last_peered":"2024-09-15T21:28:04.118226+0000","last_clean":"2024-09-15T21:28:04.118226+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:04.118226+0000","last_undegraded":"2024-09-15T21:28:04.118226+0000","last_fullsized":"2024-09-15T21:28:04.118226+0000","mapping_epoch":44,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17117184,"num_objects":4179,"num_object_clones":0,"num_object_copies":8358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14074,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16391","reported_seq":16529,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:03.477558+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:03.477558+0000","last_peered":"2024-09-15T21:28:03.477558+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:03.477558+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17592320,"num_objects":4295,"num_object_clones":0,"num_object_copies":8590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13335,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16320","reported_seq":16455,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:03.482274+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:03.482274+0000","last_peered":"2024-09-15T21:28:03.482274+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:03.482274+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17489920,"num_objects":4270,"num_object_clones":0,"num_object_copies":8540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13229,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":131}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16184","reported_seq":24499,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:04.655152+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:04.655152+0000","last_peered":"2024-09-15T21:28:04.655152+0000","last_clean":"2024-09-15T21:28:04.655152+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:04.655152+0000","last_undegraded":"2024-09-15T21:28:04.655152+0000","last_fullsized":"2024-09-15T21:28:04.655152+0000","mapping_epoch":46,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17219584,"num_objects":4204,"num_object_clones":0,"num_object_copies":8408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12668,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16490","reported_seq":16634,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:03.409172+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:03.409172+0000","last_peered":"2024-09-15T21:28:03.409172+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:03.409172+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17925062,"num_objects":4378,"num_object_clones":0,"num_object_copies":8756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4378,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13194,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":231},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":281195254,"num_objects":68661,"num_object_clones":0,"num_object_copies":137322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68661,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":221794,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9366,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":763985920,"data_stored":763840482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160801,"ondisk_log_size":160801,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584420,"kb_used_data":268104,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93787420,"statfs":{"total":96636764160,"available":96038318080,"internally_reserved":0,"allocated":274538496,"data_stored":274253238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706624,"kb_used_data":273764,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93665216,"statfs":{"total":96636764160,"available":95913181184,"internally_reserved":0,"allocated":280334336,"data_stored":280049073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,5,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607828,"kb_used_data":206848,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93764012,"statfs":{"total":96636764160,"available":96014348288,"internally_reserved":0,"allocated":211812352,"data_stored":211494113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":410578607},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,2,3,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":279527424,"data_stored":279491294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":210325504,"data_stored":210256398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274132992,"data_stored":274092790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:08.540 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:09.119 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:09.120 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:09.144 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-15T21:28:08.826733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":275677961,"num_objects":67316,"num_object_clones":0,"num_object_copies":134632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67316,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":223143,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9367,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160751,"ondisk_log_size":160751,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1896196,"kb_used_data":741880,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281219324,"statfs":{"total":289910292480,"available":287968587776,"internally_reserved":0,"allocated":759685120,"data_stored":758796360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1181928451},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,2,0,3,6,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24875008,"num_objects":-6073,"num_object_clones":0,"num_object_copies":-12146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6073,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004353"},"pg_stats":[{"pgid":"2.f","version":"47'16853","reported_seq":16934,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.488996+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:08.488996+0000","last_peered":"2024-09-15T21:28:08.488996+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:08.488996+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17608704,"num_objects":4299,"num_object_clones":0,"num_object_copies":8598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16853,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":167}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16406","reported_seq":16516,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:04.229830+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:04.229830+0000","last_peered":"2024-09-15T21:28:04.229830+0000","last_clean":"2024-09-15T21:28:04.229830+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:04.229830+0000","last_undegraded":"2024-09-15T21:28:04.229830+0000","last_fullsized":"2024-09-15T21:28:04.229830+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17482694,"num_objects":4270,"num_object_clones":0,"num_object_copies":8540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4270,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14687,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16770","reported_seq":19136,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:04.244971+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:04.244971+0000","last_peered":"2024-09-15T21:28:04.244971+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:04.244971+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17756184,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14870,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1185}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16107","reported_seq":16183,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:04.153010+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:04.153010+0000","last_peered":"2024-09-15T21:28:04.153010+0000","last_clean":"2024-09-15T21:28:04.153010+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:04.153010+0000","last_undegraded":"2024-09-15T21:28:04.153010+0000","last_fullsized":"2024-09-15T21:28:04.153010+0000","mapping_epoch":35,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16107,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16366","reported_seq":16502,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:08.383146+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:08.383146+0000","last_peered":"2024-09-15T21:28:08.383146+0000","last_clean":"2024-09-15T21:28:08.383146+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:08.383146+0000","last_undegraded":"2024-09-15T21:28:08.383146+0000","last_fullsized":"2024-09-15T21:28:08.383146+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953827,"num_objects":4140,"num_object_clones":0,"num_object_copies":8280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12821,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16426","reported_seq":16568,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.437706+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:08.437706+0000","last_peered":"2024-09-15T21:28:08.437706+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:08.437706+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17240547,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13102,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":134},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16576","reported_seq":16718,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.497676+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:08.497676+0000","last_peered":"2024-09-15T21:28:08.497676+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:08.497676+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16769990,"num_objects":4096,"num_object_clones":0,"num_object_copies":8192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12887,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-65},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16736","reported_seq":16870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:08.539807+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:08.539807+0000","last_peered":"2024-09-15T21:28:08.539807+0000","last_clean":"2024-09-15T21:28:08.539807+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:08.539807+0000","last_undegraded":"2024-09-15T21:28:08.539807+0000","last_fullsized":"2024-09-15T21:28:08.539807+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16924672,"num_objects":4132,"num_object_clones":0,"num_object_copies":8264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13508,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16264","reported_seq":16395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:04.213276+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:04.213276+0000","last_peered":"2024-09-15T21:28:04.213276+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:04.213276+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17404387,"num_objects":4250,"num_object_clones":0,"num_object_copies":8500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14196,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":163}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16301","reported_seq":16436,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.438417+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:08.438417+0000","last_peered":"2024-09-15T21:28:08.438417+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:08.438417+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17125376,"num_objects":4181,"num_object_clones":0,"num_object_copies":8362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12974,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":161}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16332","reported_seq":16456,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:04.197919+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:04.197919+0000","last_peered":"2024-09-15T21:28:04.197919+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:04.197919+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17658339,"num_objects":4312,"num_object_clones":0,"num_object_copies":8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14191,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":222}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16381","reported_seq":24845,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:04.118226+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:04.118226+0000","last_peered":"2024-09-15T21:28:04.118226+0000","last_clean":"2024-09-15T21:28:04.118226+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:04.118226+0000","last_undegraded":"2024-09-15T21:28:04.118226+0000","last_fullsized":"2024-09-15T21:28:04.118226+0000","mapping_epoch":44,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17117184,"num_objects":4179,"num_object_clones":0,"num_object_copies":8358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14074,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16537","reported_seq":16676,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.510010+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:08.510010+0000","last_peered":"2024-09-15T21:28:08.510010+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:08.510010+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16994304,"num_objects":4149,"num_object_clones":0,"num_object_copies":8298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13481,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":55}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16487","reported_seq":16625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.503785+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:08.503785+0000","last_peered":"2024-09-15T21:28:08.503785+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:08.503785+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16805888,"num_objects":4103,"num_object_clones":0,"num_object_copies":8206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13396,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-36}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16184","reported_seq":24499,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:04.655152+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:04.655152+0000","last_peered":"2024-09-15T21:28:04.655152+0000","last_clean":"2024-09-15T21:28:04.655152+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:04.655152+0000","last_undegraded":"2024-09-15T21:28:04.655152+0000","last_fullsized":"2024-09-15T21:28:04.655152+0000","mapping_epoch":46,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17219584,"num_objects":4204,"num_object_clones":0,"num_object_copies":8408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12668,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16622","reported_seq":16766,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.457976+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:08.457976+0000","last_peered":"2024-09-15T21:28:08.457976+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:08.457976+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17384390,"num_objects":4246,"num_object_clones":0,"num_object_copies":8492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4246,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13326,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":275677942,"num_objects":67314,"num_object_clones":0,"num_object_copies":134628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67314,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":223141,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9366,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":756846592,"data_stored":756701154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160748,"ondisk_log_size":160748,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584420,"kb_used_data":268104,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93787420,"statfs":{"total":96636764160,"available":96038318080,"internally_reserved":0,"allocated":274538496,"data_stored":274253238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703948,"kb_used_data":266928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93667892,"statfs":{"total":96636764160,"available":95915921408,"internally_reserved":0,"allocated":273334272,"data_stored":273049009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,1,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607828,"kb_used_data":206848,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93764012,"statfs":{"total":96636764160,"available":96014348288,"internally_reserved":0,"allocated":211812352,"data_stored":211494113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":410578607},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,2,3,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272388096,"data_stored":272351966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":210325504,"data_stored":210256398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274132992,"data_stored":274092790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:09.145 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:09.700 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:09.703 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:09.722 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-15T21:28:08.826733+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":275677961,"num_objects":67316,"num_object_clones":0,"num_object_copies":134632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67316,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":223143,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9367,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160751,"ondisk_log_size":160751,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1896196,"kb_used_data":741880,"kb_used_omap":12,"kb_used_meta":1154227,"kb_avail":281219324,"statfs":{"total":289910292480,"available":287968587776,"internally_reserved":0,"allocated":759685120,"data_stored":758796360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1181928451},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,2,0,3,6,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24875008,"num_objects":-6073,"num_object_clones":0,"num_object_copies":-12146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6073,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004353"},"pg_stats":[{"pgid":"2.f","version":"47'16853","reported_seq":16934,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.488996+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:08.488996+0000","last_peered":"2024-09-15T21:28:08.488996+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:08.488996+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17608704,"num_objects":4299,"num_object_clones":0,"num_object_copies":8598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16853,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":167}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16406","reported_seq":16516,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:04.229830+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:04.229830+0000","last_peered":"2024-09-15T21:28:04.229830+0000","last_clean":"2024-09-15T21:28:04.229830+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:04.229830+0000","last_undegraded":"2024-09-15T21:28:04.229830+0000","last_fullsized":"2024-09-15T21:28:04.229830+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17482694,"num_objects":4270,"num_object_clones":0,"num_object_copies":8540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4270,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14687,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16770","reported_seq":19136,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:04.244971+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:04.244971+0000","last_peered":"2024-09-15T21:28:04.244971+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:04.244971+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17756184,"num_objects":4336,"num_object_clones":0,"num_object_copies":8672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3151,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4336,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14870,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1117,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1185}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16107","reported_seq":16183,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:04.153010+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:04.153010+0000","last_peered":"2024-09-15T21:28:04.153010+0000","last_clean":"2024-09-15T21:28:04.153010+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:04.153010+0000","last_undegraded":"2024-09-15T21:28:04.153010+0000","last_fullsized":"2024-09-15T21:28:04.153010+0000","mapping_epoch":35,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17231872,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16107,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16366","reported_seq":16502,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:08.383146+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:08.383146+0000","last_peered":"2024-09-15T21:28:08.383146+0000","last_clean":"2024-09-15T21:28:08.383146+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:08.383146+0000","last_undegraded":"2024-09-15T21:28:08.383146+0000","last_fullsized":"2024-09-15T21:28:08.383146+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953827,"num_objects":4140,"num_object_clones":0,"num_object_copies":8280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12821,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16426","reported_seq":16568,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.437706+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:08.437706+0000","last_peered":"2024-09-15T21:28:08.437706+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:08.437706+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17240547,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13102,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":134},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16576","reported_seq":16718,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.497676+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:08.497676+0000","last_peered":"2024-09-15T21:28:08.497676+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:08.497676+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16769990,"num_objects":4096,"num_object_clones":0,"num_object_copies":8192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12887,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-65},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16736","reported_seq":16870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:08.539807+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:08.539807+0000","last_peered":"2024-09-15T21:28:08.539807+0000","last_clean":"2024-09-15T21:28:08.539807+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:08.539807+0000","last_undegraded":"2024-09-15T21:28:08.539807+0000","last_fullsized":"2024-09-15T21:28:08.539807+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16924672,"num_objects":4132,"num_object_clones":0,"num_object_copies":8264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13508,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16264","reported_seq":16395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:04.213276+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:04.213276+0000","last_peered":"2024-09-15T21:28:04.213276+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:04.213276+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17404387,"num_objects":4250,"num_object_clones":0,"num_object_copies":8500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4250,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14196,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":163}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16301","reported_seq":16436,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.438417+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:08.438417+0000","last_peered":"2024-09-15T21:28:08.438417+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:08.438417+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17125376,"num_objects":4181,"num_object_clones":0,"num_object_copies":8362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12974,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":161}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16332","reported_seq":16456,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:04.197919+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:04.197919+0000","last_peered":"2024-09-15T21:28:04.197919+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:04.197919+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17658339,"num_objects":4312,"num_object_clones":0,"num_object_copies":8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4312,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14191,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":222}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16381","reported_seq":24845,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:04.118226+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:04.118226+0000","last_peered":"2024-09-15T21:28:04.118226+0000","last_clean":"2024-09-15T21:28:04.118226+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:04.118226+0000","last_undegraded":"2024-09-15T21:28:04.118226+0000","last_fullsized":"2024-09-15T21:28:04.118226+0000","mapping_epoch":44,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17117184,"num_objects":4179,"num_object_clones":0,"num_object_copies":8358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14074,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16537","reported_seq":16676,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.510010+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:08.510010+0000","last_peered":"2024-09-15T21:28:08.510010+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:08.510010+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16994304,"num_objects":4149,"num_object_clones":0,"num_object_copies":8298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13481,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":55}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16487","reported_seq":16625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.503785+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:08.503785+0000","last_peered":"2024-09-15T21:28:08.503785+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:08.503785+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16805888,"num_objects":4103,"num_object_clones":0,"num_object_copies":8206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13396,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-36}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16184","reported_seq":24499,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:04.655152+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:04.655152+0000","last_peered":"2024-09-15T21:28:04.655152+0000","last_clean":"2024-09-15T21:28:04.655152+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:04.655152+0000","last_undegraded":"2024-09-15T21:28:04.655152+0000","last_fullsized":"2024-09-15T21:28:04.655152+0000","mapping_epoch":46,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17219584,"num_objects":4204,"num_object_clones":0,"num_object_copies":8408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12668,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16622","reported_seq":16766,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.457976+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:08.457976+0000","last_peered":"2024-09-15T21:28:08.457976+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:08.457976+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17384390,"num_objects":4246,"num_object_clones":0,"num_object_copies":8492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4246,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13326,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":275677942,"num_objects":67314,"num_object_clones":0,"num_object_copies":134628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":40108,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67314,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":223141,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9366,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":756846592,"data_stored":756701154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160748,"ondisk_log_size":160748,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":584420,"kb_used_data":268104,"kb_used_omap":4,"kb_used_meta":316283,"kb_avail":93787420,"statfs":{"total":96636764160,"available":96038318080,"internally_reserved":0,"allocated":274538496,"data_stored":274253238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":323874473},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703948,"kb_used_data":266928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93667892,"statfs":{"total":96636764160,"available":95915921408,"internally_reserved":0,"allocated":273334272,"data_stored":273049009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,1,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607828,"kb_used_data":206848,"kb_used_omap":4,"kb_used_meta":400955,"kb_avail":93764012,"statfs":{"total":96636764160,"available":96014348288,"internally_reserved":0,"allocated":211812352,"data_stored":211494113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":410578607},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,2,3,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272388096,"data_stored":272351966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":210325504,"data_stored":210256398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":274132992,"data_stored":274092790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:09.723 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:28:09.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'16853", 'reported_seq': 16934, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:08.488996+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:28:08.488996+0000', 'last_peered': '2024-09-15T21:28:08.488996+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:28:08.488996+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17608704, 'num_objects': 4299, 'num_object_clones': 0, 'num_object_copies': 8598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4299, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16853, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 167}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:09.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:28:09.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'16770", 'reported_seq': 19136, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:04.244971+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:28:04.244971+0000', 'last_peered': '2024-09-15T21:28:04.244971+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:28:04.244971+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17756184, 'num_objects': 4336, 'num_object_clones': 0, 'num_object_copies': 8672, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3151, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4336, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14870, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1117, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1185}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:09.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:09.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'16426", 'reported_seq': 16568, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:08.437706+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:28:08.437706+0000', 'last_peered': '2024-09-15T21:28:08.437706+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:28:08.437706+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17240547, 'num_objects': 4210, 'num_object_clones': 0, 'num_object_copies': 8420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4210, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13102, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 134}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:09.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:09.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16576", 'reported_seq': 16718, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:08.497676+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:28:08.497676+0000', 'last_peered': '2024-09-15T21:28:08.497676+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:28:08.497676+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16769990, 'num_objects': 4096, 'num_object_clones': 0, 'num_object_copies': 8192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12887, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -65}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:09.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:28:09.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'16264", 'reported_seq': 16395, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:04.213276+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:28:04.213276+0000', 'last_peered': '2024-09-15T21:28:04.213276+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:28:04.213276+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17404387, 'num_objects': 4250, 'num_object_clones': 0, 'num_object_copies': 8500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4250, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14196, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 163}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:09.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:09.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16301", 'reported_seq': 16436, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:08.438417+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:28:08.438417+0000', 'last_peered': '2024-09-15T21:28:08.438417+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:28:08.438417+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17125376, 'num_objects': 4181, 'num_object_clones': 0, 'num_object_copies': 8362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4181, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12974, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 161}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:09.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:28:09.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16332", 'reported_seq': 16456, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:04.197919+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:28:04.197919+0000', 'last_peered': '2024-09-15T21:28:04.197919+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:28:04.197919+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17658339, 'num_objects': 4312, 'num_object_clones': 0, 'num_object_copies': 8624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4312, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14191, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': 222}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:09.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:09.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16537", 'reported_seq': 16676, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:08.510010+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:28:08.510010+0000', 'last_peered': '2024-09-15T21:28:08.510010+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:28:08.510010+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16994304, 'num_objects': 4149, 'num_object_clones': 0, 'num_object_copies': 8298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4149, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13481, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 55}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:09.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:28:09.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'16487", 'reported_seq': 16625, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:08.503785+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:28:08.503785+0000', 'last_peered': '2024-09-15T21:28:08.503785+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:28:08.503785+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16805888, 'num_objects': 4103, 'num_object_clones': 0, 'num_object_copies': 8206, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4103, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13396, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -36}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:09.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:28:09.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'16622", 'reported_seq': 16766, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:08.457976+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:28:08.457976+0000', 'last_peered': '2024-09-15T21:28:08.457976+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:28:08.457976+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17384390, 'num_objects': 4246, 'num_object_clones': 0, 'num_object_copies': 8492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4246, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13326, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 99}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:09.726 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:10.423 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:10.423 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 40108/134632 objects degraded (29.791%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":542,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":67316,"data_bytes":275677961,"bytes_used":1941704704,"bytes_avail":287968587776,"bytes_total":289910292480,"degraded_objects":40108,"degraded_total":134632,"degraded_ratio":0.29790837245231444,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":199,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":505},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:10.444 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:10.445 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:11.015 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:11.016 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:11.046 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-15T21:28:10.827555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":271610633,"num_objects":66323,"num_object_clones":0,"num_object_copies":132646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66323,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":224136,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9476,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160644,"ondisk_log_size":160644,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898284,"kb_used_data":730088,"kb_used_omap":12,"kb_used_meta":1168115,"kb_avail":281217236,"statfs":{"total":289910292480,"available":287966449664,"internally_reserved":0,"allocated":747610112,"data_stored":746721352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1196149763},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,2,5,6,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23539712,"num_objects":-5747,"num_object_clones":0,"num_object_copies":-11494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-308,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5747,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5747,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":308,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004391"},"pg_stats":[{"pgid":"2.f","version":"47'16853","reported_seq":16934,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.488996+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:08.488996+0000","last_peered":"2024-09-15T21:28:08.488996+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:08.488996+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17608704,"num_objects":4299,"num_object_clones":0,"num_object_copies":8598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16853,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":167}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16533","reported_seq":16644,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:09.276773+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:09.276773+0000","last_peered":"2024-09-15T21:28:09.276773+0000","last_clean":"2024-09-15T21:28:09.276773+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:09.276773+0000","last_undegraded":"2024-09-15T21:28:09.276773+0000","last_fullsized":"2024-09-15T21:28:09.276773+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16962502,"num_objects":4143,"num_object_clones":0,"num_object_copies":8286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4143,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14814,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16916","reported_seq":19500,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:09.286738+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:09.286738+0000","last_peered":"2024-09-15T21:28:09.286738+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:09.286738+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17158168,"num_objects":4190,"num_object_clones":0,"num_object_copies":8380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15016,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1148}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16242","reported_seq":16318,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:09.147831+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:09.147831+0000","last_peered":"2024-09-15T21:28:09.147831+0000","last_clean":"2024-09-15T21:28:09.147831+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:09.147831+0000","last_undegraded":"2024-09-15T21:28:09.147831+0000","last_fullsized":"2024-09-15T21:28:09.147831+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16678912,"num_objects":4072,"num_object_clones":0,"num_object_copies":8144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16242,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16366","reported_seq":16502,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:08.383146+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:08.383146+0000","last_peered":"2024-09-15T21:28:08.383146+0000","last_clean":"2024-09-15T21:28:08.383146+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:08.383146+0000","last_undegraded":"2024-09-15T21:28:08.383146+0000","last_fullsized":"2024-09-15T21:28:08.383146+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953827,"num_objects":4140,"num_object_clones":0,"num_object_copies":8280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12821,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16426","reported_seq":16568,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.437706+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:08.437706+0000","last_peered":"2024-09-15T21:28:08.437706+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:08.437706+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17240547,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13102,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":134},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16576","reported_seq":16718,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.497676+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:08.497676+0000","last_peered":"2024-09-15T21:28:08.497676+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:08.497676+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16769990,"num_objects":4096,"num_object_clones":0,"num_object_copies":8192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12887,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-65},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16736","reported_seq":16870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:08.539807+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:08.539807+0000","last_peered":"2024-09-15T21:28:08.539807+0000","last_clean":"2024-09-15T21:28:08.539807+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:08.539807+0000","last_undegraded":"2024-09-15T21:28:08.539807+0000","last_fullsized":"2024-09-15T21:28:08.539807+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16924672,"num_objects":4132,"num_object_clones":0,"num_object_copies":8264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13508,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16403","reported_seq":16535,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:09.276807+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:09.276807+0000","last_peered":"2024-09-15T21:28:09.276807+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:09.276807+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16835043,"num_objects":4111,"num_object_clones":0,"num_object_copies":8222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14335,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16301","reported_seq":16436,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.438417+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:08.438417+0000","last_peered":"2024-09-15T21:28:08.438417+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:08.438417+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17125376,"num_objects":4181,"num_object_clones":0,"num_object_copies":8362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12974,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":161}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16479","reported_seq":16603,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:09.220305+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:09.220305+0000","last_peered":"2024-09-15T21:28:09.220305+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:09.220305+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17056227,"num_objects":4165,"num_object_clones":0,"num_object_copies":8330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14338,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":75}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16536","reported_seq":25000,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:09.282088+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:09.282088+0000","last_peered":"2024-09-15T21:28:09.282088+0000","last_clean":"2024-09-15T21:28:09.282088+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:09.282088+0000","last_undegraded":"2024-09-15T21:28:09.282088+0000","last_fullsized":"2024-09-15T21:28:09.282088+0000","mapping_epoch":44,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16482304,"num_objects":4024,"num_object_clones":0,"num_object_copies":8048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14229,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16537","reported_seq":16676,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.510010+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:08.510010+0000","last_peered":"2024-09-15T21:28:08.510010+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:08.510010+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16994304,"num_objects":4149,"num_object_clones":0,"num_object_copies":8298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13481,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":55}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16487","reported_seq":16625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.503785+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:08.503785+0000","last_peered":"2024-09-15T21:28:08.503785+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:08.503785+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16805888,"num_objects":4103,"num_object_clones":0,"num_object_copies":8206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13396,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-36}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16328","reported_seq":24643,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:09.705737+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:09.705737+0000","last_peered":"2024-09-15T21:28:09.705737+0000","last_clean":"2024-09-15T21:28:09.705737+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:09.705737+0000","last_undegraded":"2024-09-15T21:28:09.705737+0000","last_fullsized":"2024-09-15T21:28:09.705737+0000","mapping_epoch":46,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16629760,"num_objects":4060,"num_object_clones":0,"num_object_copies":8120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12812,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16622","reported_seq":16766,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.457976+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:08.457976+0000","last_peered":"2024-09-15T21:28:08.457976+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:08.457976+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17384390,"num_objects":4246,"num_object_clones":0,"num_object_copies":8492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4246,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13326,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":271610614,"num_objects":66321,"num_object_clones":0,"num_object_copies":132642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66321,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":224134,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9475,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":744779776,"data_stored":744634338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160641,"ondisk_log_size":160641,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602036,"kb_used_data":263520,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93769804,"statfs":{"total":96636764160,"available":96020279296,"internally_reserved":0,"allocated":269844480,"data_stored":269559222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703948,"kb_used_data":266928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93667892,"statfs":{"total":96636764160,"available":95915921408,"internally_reserved":0,"allocated":273334272,"data_stored":273049009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,1,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592300,"kb_used_data":199640,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93779540,"statfs":{"total":96636764160,"available":96030248960,"internally_reserved":0,"allocated":204431360,"data_stored":204113121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":402058927},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272388096,"data_stored":272351966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":202948608,"data_stored":202879502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":269443072,"data_stored":269402870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:14.049 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:14.607 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:14.608 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:14.627 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-15T21:28:12.828343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":271610633,"num_objects":66323,"num_object_clones":0,"num_object_copies":132646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66323,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":224136,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9476,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160644,"ondisk_log_size":160644,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1898284,"kb_used_data":730088,"kb_used_omap":12,"kb_used_meta":1168115,"kb_avail":281217236,"statfs":{"total":289910292480,"available":287966449664,"internally_reserved":0,"allocated":747610112,"data_stored":746721352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1196149763},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,2,5,6,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19132416,"num_objects":-4671,"num_object_clones":0,"num_object_copies":-9342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4671,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004569"},"pg_stats":[{"pgid":"2.f","version":"47'16853","reported_seq":16934,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.488996+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:08.488996+0000","last_peered":"2024-09-15T21:28:08.488996+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:08.488996+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17608704,"num_objects":4299,"num_object_clones":0,"num_object_copies":8598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4299,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16853,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":167}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16533","reported_seq":16644,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:09.276773+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:09.276773+0000","last_peered":"2024-09-15T21:28:09.276773+0000","last_clean":"2024-09-15T21:28:09.276773+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:09.276773+0000","last_undegraded":"2024-09-15T21:28:09.276773+0000","last_fullsized":"2024-09-15T21:28:09.276773+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16962502,"num_objects":4143,"num_object_clones":0,"num_object_copies":8286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4143,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14814,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16916","reported_seq":19500,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:09.286738+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:09.286738+0000","last_peered":"2024-09-15T21:28:09.286738+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:09.286738+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17158168,"num_objects":4190,"num_object_clones":0,"num_object_copies":8380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3042,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4190,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15016,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1226,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1148}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16242","reported_seq":16318,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:09.147831+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:09.147831+0000","last_peered":"2024-09-15T21:28:09.147831+0000","last_clean":"2024-09-15T21:28:09.147831+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:09.147831+0000","last_undegraded":"2024-09-15T21:28:09.147831+0000","last_fullsized":"2024-09-15T21:28:09.147831+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16678912,"num_objects":4072,"num_object_clones":0,"num_object_copies":8144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16242,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16366","reported_seq":16502,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:08.383146+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:08.383146+0000","last_peered":"2024-09-15T21:28:08.383146+0000","last_clean":"2024-09-15T21:28:08.383146+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:08.383146+0000","last_undegraded":"2024-09-15T21:28:08.383146+0000","last_fullsized":"2024-09-15T21:28:08.383146+0000","mapping_epoch":34,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16953827,"num_objects":4140,"num_object_clones":0,"num_object_copies":8280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4140,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12821,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16426","reported_seq":16568,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.437706+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:08.437706+0000","last_peered":"2024-09-15T21:28:08.437706+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:08.437706+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17240547,"num_objects":4210,"num_object_clones":0,"num_object_copies":8420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4210,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13102,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":134},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16576","reported_seq":16718,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.497676+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:08.497676+0000","last_peered":"2024-09-15T21:28:08.497676+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:08.497676+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16769990,"num_objects":4096,"num_object_clones":0,"num_object_copies":8192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12887,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-65},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16736","reported_seq":16870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:08.539807+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:08.539807+0000","last_peered":"2024-09-15T21:28:08.539807+0000","last_clean":"2024-09-15T21:28:08.539807+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:08.539807+0000","last_undegraded":"2024-09-15T21:28:08.539807+0000","last_fullsized":"2024-09-15T21:28:08.539807+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16924672,"num_objects":4132,"num_object_clones":0,"num_object_copies":8264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13508,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16403","reported_seq":16535,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:09.276807+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:09.276807+0000","last_peered":"2024-09-15T21:28:09.276807+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:09.276807+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16835043,"num_objects":4111,"num_object_clones":0,"num_object_copies":8222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14335,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":24}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16301","reported_seq":16436,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.438417+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:08.438417+0000","last_peered":"2024-09-15T21:28:08.438417+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:08.438417+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17125376,"num_objects":4181,"num_object_clones":0,"num_object_copies":8362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12974,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":161}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16479","reported_seq":16603,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:09.220305+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:09.220305+0000","last_peered":"2024-09-15T21:28:09.220305+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:09.220305+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17056227,"num_objects":4165,"num_object_clones":0,"num_object_copies":8330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14338,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":75}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16536","reported_seq":25000,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:09.282088+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:09.282088+0000","last_peered":"2024-09-15T21:28:09.282088+0000","last_clean":"2024-09-15T21:28:09.282088+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:09.282088+0000","last_undegraded":"2024-09-15T21:28:09.282088+0000","last_fullsized":"2024-09-15T21:28:09.282088+0000","mapping_epoch":44,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16482304,"num_objects":4024,"num_object_clones":0,"num_object_copies":8048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14229,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16537","reported_seq":16676,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.510010+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:08.510010+0000","last_peered":"2024-09-15T21:28:08.510010+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:08.510010+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16994304,"num_objects":4149,"num_object_clones":0,"num_object_copies":8298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13481,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":55}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16487","reported_seq":16625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.503785+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:08.503785+0000","last_peered":"2024-09-15T21:28:08.503785+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:08.503785+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16805888,"num_objects":4103,"num_object_clones":0,"num_object_copies":8206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13396,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-36}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16328","reported_seq":24643,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:09.705737+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:09.705737+0000","last_peered":"2024-09-15T21:28:09.705737+0000","last_clean":"2024-09-15T21:28:09.705737+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:09.705737+0000","last_undegraded":"2024-09-15T21:28:09.705737+0000","last_fullsized":"2024-09-15T21:28:09.705737+0000","mapping_epoch":46,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16629760,"num_objects":4060,"num_object_clones":0,"num_object_copies":8120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12812,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16622","reported_seq":16766,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:08.457976+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:08.457976+0000","last_peered":"2024-09-15T21:28:08.457976+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:08.457976+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17384390,"num_objects":4246,"num_object_clones":0,"num_object_copies":8492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4246,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13326,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":99},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":271610614,"num_objects":66321,"num_object_clones":0,"num_object_copies":132642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66321,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":224134,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9475,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":744779776,"data_stored":744634338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160641,"ondisk_log_size":160641,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":602036,"kb_used_data":263520,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93769804,"statfs":{"total":96636764160,"available":96020279296,"internally_reserved":0,"allocated":269844480,"data_stored":269559222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":703948,"kb_used_data":266928,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93667892,"statfs":{"total":96636764160,"available":95915921408,"internally_reserved":0,"allocated":273334272,"data_stored":273049009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,0,1,0,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592300,"kb_used_data":199640,"kb_used_omap":4,"kb_used_meta":392635,"kb_avail":93779540,"statfs":{"total":96636764160,"available":96030248960,"internally_reserved":0,"allocated":204431360,"data_stored":204113121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":402058927},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":272388096,"data_stored":272351966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":202948608,"data_stored":202879502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":269443072,"data_stored":269402870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:14.629 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:15.196 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:15.197 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:15.230 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-15T21:28:14.829001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":261980937,"num_objects":63972,"num_object_clones":0,"num_object_copies":127944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63972,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":226487,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9576,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1879804,"kb_used_data":711608,"kb_used_omap":12,"kb_used_meta":1168115,"kb_avail":281235716,"statfs":{"total":289910292480,"available":287985373184,"internally_reserved":0,"allocated":728686592,"data_stored":727797832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1196149763},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,9,9,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28762112,"num_objects":-7022,"num_object_clones":0,"num_object_copies":-14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-307,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7022,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004539"},"pg_stats":[{"pgid":"2.f","version":"47'17023","reported_seq":17104,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.501211+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:13.501211+0000","last_peered":"2024-09-15T21:28:13.501211+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:13.501211+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16912384,"num_objects":4129,"num_object_clones":0,"num_object_copies":8258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17023,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16681","reported_seq":16792,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.254595+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:14.254595+0000","last_peered":"2024-09-15T21:28:14.254595+0000","last_clean":"2024-09-15T21:28:14.254595+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:14.254595+0000","last_undegraded":"2024-09-15T21:28:14.254595+0000","last_fullsized":"2024-09-15T21:28:14.254595+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16356294,"num_objects":3995,"num_object_clones":0,"num_object_copies":7990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3995,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14962,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17075","reported_seq":19858,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:14.278755+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:14.278755+0000","last_peered":"2024-09-15T21:28:14.278755+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:14.278755+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16506904,"num_objects":4031,"num_object_clones":0,"num_object_copies":8062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4031,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15175,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1089}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16387","reported_seq":16463,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.222798+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:14.222798+0000","last_peered":"2024-09-15T21:28:14.222798+0000","last_clean":"2024-09-15T21:28:14.222798+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:14.222798+0000","last_undegraded":"2024-09-15T21:28:14.222798+0000","last_fullsized":"2024-09-15T21:28:14.222798+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16084992,"num_objects":3927,"num_object_clones":0,"num_object_copies":7854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16387,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16529","reported_seq":16667,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:13.451205+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:13.451205+0000","last_peered":"2024-09-15T21:28:13.451205+0000","last_clean":"2024-09-15T21:28:13.451205+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:13.451205+0000","last_undegraded":"2024-09-15T21:28:13.451205+0000","last_fullsized":"2024-09-15T21:28:13.451205+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16286179,"num_objects":3977,"num_object_clones":0,"num_object_copies":7954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12984,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16587","reported_seq":16729,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.551928+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:13.551928+0000","last_peered":"2024-09-15T21:28:13.551928+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:13.551928+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16581091,"num_objects":4049,"num_object_clones":0,"num_object_copies":8098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13263,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-27},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16727","reported_seq":16869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.501155+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:13.501155+0000","last_peered":"2024-09-15T21:28:13.501155+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:13.501155+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16151494,"num_objects":3945,"num_object_clones":0,"num_object_copies":7890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13038,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-216},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16898","reported_seq":17032,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:13.535686+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:13.535686+0000","last_peered":"2024-09-15T21:28:13.535686+0000","last_clean":"2024-09-15T21:28:13.535686+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:13.535686+0000","last_undegraded":"2024-09-15T21:28:13.535686+0000","last_fullsized":"2024-09-15T21:28:13.535686+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16261120,"num_objects":3970,"num_object_clones":0,"num_object_copies":7940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13670,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16545","reported_seq":16677,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:14.138953+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:14.138953+0000","last_peered":"2024-09-15T21:28:14.138953+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:14.138953+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16253411,"num_objects":3969,"num_object_clones":0,"num_object_copies":7938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3969,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14477,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16451","reported_seq":16587,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.510994+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:13.510994+0000","last_peered":"2024-09-15T21:28:13.510994+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:13.510994+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16510976,"num_objects":4031,"num_object_clones":0,"num_object_copies":8062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13124,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":11}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16618","reported_seq":16742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:14.238968+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:14.238968+0000","last_peered":"2024-09-15T21:28:14.238968+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:14.238968+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16486883,"num_objects":4026,"num_object_clones":0,"num_object_copies":8052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14477,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-64}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16670","reported_seq":25134,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.232971+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:14.232971+0000","last_peered":"2024-09-15T21:28:14.232971+0000","last_clean":"2024-09-15T21:28:14.232971+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:14.232971+0000","last_undegraded":"2024-09-15T21:28:14.232971+0000","last_fullsized":"2024-09-15T21:28:14.232971+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15933440,"num_objects":3890,"num_object_clones":0,"num_object_copies":7780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14363,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16685","reported_seq":16824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.452668+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:13.452668+0000","last_peered":"2024-09-15T21:28:13.452668+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:13.452668+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16388096,"num_objects":4001,"num_object_clones":0,"num_object_copies":8002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13629,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-93}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16621","reported_seq":16759,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.558832+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:13.558832+0000","last_peered":"2024-09-15T21:28:13.558832+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:13.558832+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16257024,"num_objects":3969,"num_object_clones":0,"num_object_copies":7938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13530,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-170}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16448","reported_seq":24763,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.806544+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:14.806544+0000","last_peered":"2024-09-15T21:28:14.806544+0000","last_clean":"2024-09-15T21:28:14.806544+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:14.806544+0000","last_undegraded":"2024-09-15T21:28:14.806544+0000","last_fullsized":"2024-09-15T21:28:14.806544+0000","mapping_epoch":46,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16138240,"num_objects":3940,"num_object_clones":0,"num_object_copies":7880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12932,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16747","reported_seq":16891,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.419711+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:13.419711+0000","last_peered":"2024-09-15T21:28:13.419711+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:13.419711+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16872390,"num_objects":4121,"num_object_clones":0,"num_object_copies":8242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13451,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-26},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":261980918,"num_objects":63970,"num_object_clones":0,"num_object_copies":127940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63970,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":226485,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9575,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":725426176,"data_stored":725280738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601420,"kb_used_data":258748,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93770420,"statfs":{"total":96636764160,"available":96020910080,"internally_reserved":0,"allocated":264957952,"data_stored":264672694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688700,"kb_used_data":260004,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93683140,"statfs":{"total":96636764160,"available":95931535360,"internally_reserved":0,"allocated":266244096,"data_stored":265958833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,6,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589684,"kb_used_data":192856,"kb_used_omap":4,"kb_used_meta":396795,"kb_avail":93782156,"statfs":{"total":96636764160,"available":96032927744,"internally_reserved":0,"allocated":197484544,"data_stored":197166305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":406318767},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":265097216,"data_stored":265061086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":195801088,"data_stored":195731982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":264527872,"data_stored":264487670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:15.231 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:28:15.231 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17023", 'reported_seq': 17104, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:13.501211+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:28:13.501211+0000', 'last_peered': '2024-09-15T21:28:13.501211+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:28:13.501211+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16912384, 'num_objects': 4129, 'num_object_clones': 0, 'num_object_copies': 8258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4129, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17023, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:15.231 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:28:15.231 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'17075", 'reported_seq': 19858, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:14.278755+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:28:14.278755+0000', 'last_peered': '2024-09-15T21:28:14.278755+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:28:14.278755+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16506904, 'num_objects': 4031, 'num_object_clones': 0, 'num_object_copies': 8062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4031, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15175, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1089}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:15.231 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:15.231 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'16587", 'reported_seq': 16729, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:13.551928+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:28:13.551928+0000', 'last_peered': '2024-09-15T21:28:13.551928+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:28:13.551928+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16581091, 'num_objects': 4049, 'num_object_clones': 0, 'num_object_copies': 8098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4049, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13263, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -27}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:15.232 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:15.232 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16727", 'reported_seq': 16869, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:13.501155+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:28:13.501155+0000', 'last_peered': '2024-09-15T21:28:13.501155+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:28:13.501155+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16151494, 'num_objects': 3945, 'num_object_clones': 0, 'num_object_copies': 7890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3945, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13038, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -216}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:15.232 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:28:15.232 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'16545", 'reported_seq': 16677, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:14.138953+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:28:14.138953+0000', 'last_peered': '2024-09-15T21:28:14.138953+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:28:14.138953+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16253411, 'num_objects': 3969, 'num_object_clones': 0, 'num_object_copies': 7938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3969, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14477, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -118}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:15.232 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:15.232 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16451", 'reported_seq': 16587, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:13.510994+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:28:13.510994+0000', 'last_peered': '2024-09-15T21:28:13.510994+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:28:13.510994+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16510976, 'num_objects': 4031, 'num_object_clones': 0, 'num_object_copies': 8062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4031, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13124, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 11}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:15.232 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:28:15.232 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16618", 'reported_seq': 16742, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:14.238968+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:28:14.238968+0000', 'last_peered': '2024-09-15T21:28:14.238968+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:28:14.238968+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16486883, 'num_objects': 4026, 'num_object_clones': 0, 'num_object_copies': 8052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4026, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14477, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -64}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:15.232 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:15.232 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16685", 'reported_seq': 16824, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:13.452668+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:28:13.452668+0000', 'last_peered': '2024-09-15T21:28:13.452668+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:28:13.452668+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16388096, 'num_objects': 4001, 'num_object_clones': 0, 'num_object_copies': 8002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4001, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13629, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -93}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:15.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:28:15.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'16621", 'reported_seq': 16759, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:13.558832+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:28:13.558832+0000', 'last_peered': '2024-09-15T21:28:13.558832+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:28:13.558832+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16257024, 'num_objects': 3969, 'num_object_clones': 0, 'num_object_copies': 7938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3969, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13530, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -170}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:15.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:28:15.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'16747", 'reported_seq': 16891, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:13.419711+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:28:13.419711+0000', 'last_peered': '2024-09-15T21:28:13.419711+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:28:13.419711+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16872390, 'num_objects': 4121, 'num_object_clones': 0, 'num_object_copies': 8242, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4121, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13451, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -26}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:15.233 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:15.938 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:15.938 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39899/127944 objects degraded (31.185%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":547,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":63972,"data_bytes":261980937,"bytes_used":1924919296,"bytes_avail":287985373184,"bytes_total":289910292480,"degraded_objects":39899,"degraded_total":127944,"degraded_ratio":0.31184737072469204,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":584},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:15.958 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:15.958 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:16.507 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:16.526 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-09-15T21:28:14.829001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":261980937,"num_objects":63972,"num_object_clones":0,"num_object_copies":127944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63972,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":226487,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9576,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160895,"ondisk_log_size":160895,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1879804,"kb_used_data":711608,"kb_used_omap":12,"kb_used_meta":1168115,"kb_avail":281235716,"statfs":{"total":289910292480,"available":287985373184,"internally_reserved":0,"allocated":728686592,"data_stored":727797832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1196149763},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,8,9,9,13,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28762112,"num_objects":-7022,"num_object_clones":0,"num_object_copies":-14044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-307,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7022,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7022,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004539"},"pg_stats":[{"pgid":"2.f","version":"47'17023","reported_seq":17104,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.501211+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:13.501211+0000","last_peered":"2024-09-15T21:28:13.501211+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:13.501211+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16912384,"num_objects":4129,"num_object_clones":0,"num_object_copies":8258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17023,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16681","reported_seq":16792,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.254595+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:14.254595+0000","last_peered":"2024-09-15T21:28:14.254595+0000","last_clean":"2024-09-15T21:28:14.254595+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:14.254595+0000","last_undegraded":"2024-09-15T21:28:14.254595+0000","last_fullsized":"2024-09-15T21:28:14.254595+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16356294,"num_objects":3995,"num_object_clones":0,"num_object_copies":7990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3995,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14962,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17075","reported_seq":19858,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:14.278755+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:14.278755+0000","last_peered":"2024-09-15T21:28:14.278755+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:14.278755+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16506904,"num_objects":4031,"num_object_clones":0,"num_object_copies":8062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4031,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15175,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1089}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16387","reported_seq":16463,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.222798+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:14.222798+0000","last_peered":"2024-09-15T21:28:14.222798+0000","last_clean":"2024-09-15T21:28:14.222798+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:14.222798+0000","last_undegraded":"2024-09-15T21:28:14.222798+0000","last_fullsized":"2024-09-15T21:28:14.222798+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16084992,"num_objects":3927,"num_object_clones":0,"num_object_copies":7854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16387,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16529","reported_seq":16667,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:13.451205+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:13.451205+0000","last_peered":"2024-09-15T21:28:13.451205+0000","last_clean":"2024-09-15T21:28:13.451205+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:13.451205+0000","last_undegraded":"2024-09-15T21:28:13.451205+0000","last_fullsized":"2024-09-15T21:28:13.451205+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16286179,"num_objects":3977,"num_object_clones":0,"num_object_copies":7954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12984,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16587","reported_seq":16729,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.551928+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:13.551928+0000","last_peered":"2024-09-15T21:28:13.551928+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:13.551928+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16581091,"num_objects":4049,"num_object_clones":0,"num_object_copies":8098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4049,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13263,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-27},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16727","reported_seq":16869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.501155+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:13.501155+0000","last_peered":"2024-09-15T21:28:13.501155+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:13.501155+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16151494,"num_objects":3945,"num_object_clones":0,"num_object_copies":7890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13038,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-216},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16898","reported_seq":17032,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:13.535686+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:13.535686+0000","last_peered":"2024-09-15T21:28:13.535686+0000","last_clean":"2024-09-15T21:28:13.535686+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:13.535686+0000","last_undegraded":"2024-09-15T21:28:13.535686+0000","last_fullsized":"2024-09-15T21:28:13.535686+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16261120,"num_objects":3970,"num_object_clones":0,"num_object_copies":7940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13670,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16545","reported_seq":16677,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:14.138953+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:14.138953+0000","last_peered":"2024-09-15T21:28:14.138953+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:14.138953+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16253411,"num_objects":3969,"num_object_clones":0,"num_object_copies":7938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3969,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14477,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16451","reported_seq":16587,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.510994+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:13.510994+0000","last_peered":"2024-09-15T21:28:13.510994+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:13.510994+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16510976,"num_objects":4031,"num_object_clones":0,"num_object_copies":8062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13124,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":11}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16618","reported_seq":16742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:14.238968+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:14.238968+0000","last_peered":"2024-09-15T21:28:14.238968+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:14.238968+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16486883,"num_objects":4026,"num_object_clones":0,"num_object_copies":8052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14477,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-64}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16670","reported_seq":25134,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.232971+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:14.232971+0000","last_peered":"2024-09-15T21:28:14.232971+0000","last_clean":"2024-09-15T21:28:14.232971+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:14.232971+0000","last_undegraded":"2024-09-15T21:28:14.232971+0000","last_fullsized":"2024-09-15T21:28:14.232971+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15933440,"num_objects":3890,"num_object_clones":0,"num_object_copies":7780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14363,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16685","reported_seq":16824,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.452668+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:13.452668+0000","last_peered":"2024-09-15T21:28:13.452668+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:13.452668+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16388096,"num_objects":4001,"num_object_clones":0,"num_object_copies":8002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13629,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-93}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16621","reported_seq":16759,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.558832+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:13.558832+0000","last_peered":"2024-09-15T21:28:13.558832+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:13.558832+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16257024,"num_objects":3969,"num_object_clones":0,"num_object_copies":7938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13530,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-170}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16448","reported_seq":24763,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.806544+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:14.806544+0000","last_peered":"2024-09-15T21:28:14.806544+0000","last_clean":"2024-09-15T21:28:14.806544+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:14.806544+0000","last_undegraded":"2024-09-15T21:28:14.806544+0000","last_fullsized":"2024-09-15T21:28:14.806544+0000","mapping_epoch":46,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16138240,"num_objects":3940,"num_object_clones":0,"num_object_copies":7880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12932,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16747","reported_seq":16891,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:13.419711+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:13.419711+0000","last_peered":"2024-09-15T21:28:13.419711+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:13.419711+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16872390,"num_objects":4121,"num_object_clones":0,"num_object_copies":8242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13451,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-26},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":261980918,"num_objects":63970,"num_object_clones":0,"num_object_copies":127940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63970,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":226485,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9575,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":725426176,"data_stored":725280738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160892,"ondisk_log_size":160892,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601420,"kb_used_data":258748,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93770420,"statfs":{"total":96636764160,"available":96020910080,"internally_reserved":0,"allocated":264957952,"data_stored":264672694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688700,"kb_used_data":260004,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93683140,"statfs":{"total":96636764160,"available":95931535360,"internally_reserved":0,"allocated":266244096,"data_stored":265958833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,6,4,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589684,"kb_used_data":192856,"kb_used_omap":4,"kb_used_meta":396795,"kb_avail":93782156,"statfs":{"total":96636764160,"available":96032927744,"internally_reserved":0,"allocated":197484544,"data_stored":197166305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":406318767},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":265097216,"data_stored":265061086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":195801088,"data_stored":195731982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":264527872,"data_stored":264487670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:19.529 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:20.093 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:20.095 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:20.124 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-15T21:28:18.830480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":256488201,"num_objects":62631,"num_object_clones":0,"num_object_copies":125262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62631,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":227828,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9576,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1876916,"kb_used_data":704548,"kb_used_omap":12,"kb_used_meta":1172275,"kb_avail":281238604,"statfs":{"total":289910292480,"available":287988330496,"internally_reserved":0,"allocated":721457152,"data_stored":720568392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1200409603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,6,7,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24707072,"num_objects":-6032,"num_object_clones":0,"num_object_copies":-12064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6032,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004508"},"pg_stats":[{"pgid":"2.f","version":"47'17150","reported_seq":17229,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.596215+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:18.596215+0000","last_peered":"2024-09-15T21:28:18.596215+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:18.596215+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16392192,"num_objects":4002,"num_object_clones":0,"num_object_copies":8004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17150,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-130}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16681","reported_seq":16792,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.254595+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:14.254595+0000","last_peered":"2024-09-15T21:28:14.254595+0000","last_clean":"2024-09-15T21:28:14.254595+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:14.254595+0000","last_undegraded":"2024-09-15T21:28:14.254595+0000","last_fullsized":"2024-09-15T21:28:14.254595+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16356294,"num_objects":3995,"num_object_clones":0,"num_object_copies":7990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3995,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14962,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17075","reported_seq":19858,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:14.278755+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:14.278755+0000","last_peered":"2024-09-15T21:28:14.278755+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:14.278755+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16506904,"num_objects":4031,"num_object_clones":0,"num_object_copies":8062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4031,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15175,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1089}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16387","reported_seq":16463,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.222798+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:14.222798+0000","last_peered":"2024-09-15T21:28:14.222798+0000","last_clean":"2024-09-15T21:28:14.222798+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:14.222798+0000","last_undegraded":"2024-09-15T21:28:14.222798+0000","last_fullsized":"2024-09-15T21:28:14.222798+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16084992,"num_objects":3927,"num_object_clones":0,"num_object_copies":7854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16387,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16701","reported_seq":16839,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:18.567221+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:18.567221+0000","last_peered":"2024-09-15T21:28:18.567221+0000","last_clean":"2024-09-15T21:28:18.567221+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:18.567221+0000","last_undegraded":"2024-09-15T21:28:18.567221+0000","last_fullsized":"2024-09-15T21:28:18.567221+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15581667,"num_objects":3805,"num_object_clones":0,"num_object_copies":7610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13156,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16739","reported_seq":16881,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.558737+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:18.558737+0000","last_peered":"2024-09-15T21:28:18.558737+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:18.558737+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15958499,"num_objects":3897,"num_object_clones":0,"num_object_copies":7794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13415,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-179},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16862","reported_seq":17004,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.531425+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:18.531425+0000","last_peered":"2024-09-15T21:28:18.531425+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:18.531425+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15598534,"num_objects":3810,"num_object_clones":0,"num_object_copies":7620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13173,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-351},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17042","reported_seq":17176,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:18.593373+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:18.593373+0000","last_peered":"2024-09-15T21:28:18.593373+0000","last_clean":"2024-09-15T21:28:18.593373+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:18.593373+0000","last_undegraded":"2024-09-15T21:28:18.593373+0000","last_fullsized":"2024-09-15T21:28:18.593373+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15671296,"num_objects":3826,"num_object_clones":0,"num_object_copies":7652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13814,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16545","reported_seq":16677,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:14.138953+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:14.138953+0000","last_peered":"2024-09-15T21:28:14.138953+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:14.138953+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16253411,"num_objects":3969,"num_object_clones":0,"num_object_copies":7938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3969,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14477,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16613","reported_seq":16749,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.557306+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:18.557306+0000","last_peered":"2024-09-15T21:28:18.557306+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:18.557306+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15847424,"num_objects":3869,"num_object_clones":0,"num_object_copies":7738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13286,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-151}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16618","reported_seq":16742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:14.238968+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:14.238968+0000","last_peered":"2024-09-15T21:28:14.238968+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:14.238968+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16486883,"num_objects":4026,"num_object_clones":0,"num_object_copies":8052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14477,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-64}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16670","reported_seq":25134,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.232971+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:14.232971+0000","last_peered":"2024-09-15T21:28:14.232971+0000","last_clean":"2024-09-15T21:28:14.232971+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:14.232971+0000","last_undegraded":"2024-09-15T21:28:14.232971+0000","last_fullsized":"2024-09-15T21:28:14.232971+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15933440,"num_objects":3890,"num_object_clones":0,"num_object_copies":7780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14363,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16846","reported_seq":16985,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.588610+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:18.588610+0000","last_peered":"2024-09-15T21:28:18.588610+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:18.588610+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15728640,"num_objects":3840,"num_object_clones":0,"num_object_copies":7680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13790,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-254}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16775","reported_seq":16913,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.610441+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:18.610441+0000","last_peered":"2024-09-15T21:28:18.610441+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:18.610441+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15626240,"num_objects":3815,"num_object_clones":0,"num_object_copies":7630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13684,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-324}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16448","reported_seq":24763,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.806544+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:14.806544+0000","last_peered":"2024-09-15T21:28:14.806544+0000","last_clean":"2024-09-15T21:28:14.806544+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:14.806544+0000","last_undegraded":"2024-09-15T21:28:14.806544+0000","last_fullsized":"2024-09-15T21:28:14.806544+0000","mapping_epoch":46,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16138240,"num_objects":3940,"num_object_clones":0,"num_object_copies":7880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12932,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16881","reported_seq":17025,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.602421+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:18.602421+0000","last_peered":"2024-09-15T21:28:18.602421+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:18.602421+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16323526,"num_objects":3987,"num_object_clones":0,"num_object_copies":7974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3987,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13585,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-160},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":256488182,"num_objects":62629,"num_object_clones":0,"num_object_copies":125258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62629,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":227826,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9575,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":718151680,"data_stored":718006242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601420,"kb_used_data":258748,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93770420,"statfs":{"total":96636764160,"available":96020910080,"internally_reserved":0,"allocated":264957952,"data_stored":264672694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685812,"kb_used_data":252944,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93686028,"statfs":{"total":96636764160,"available":95934492672,"internally_reserved":0,"allocated":259014656,"data_stored":258729393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589684,"kb_used_data":192856,"kb_used_omap":4,"kb_used_meta":396795,"kb_avail":93782156,"statfs":{"total":96636764160,"available":96032927744,"internally_reserved":0,"allocated":197484544,"data_stored":197166305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":406318767},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257822720,"data_stored":257786590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":195801088,"data_stored":195731982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":264527872,"data_stored":264487670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:20.127 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:20.680 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:20.684 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:20.703 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-15T21:28:18.830480+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":256488201,"num_objects":62631,"num_object_clones":0,"num_object_copies":125262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62631,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":227828,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9576,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1876916,"kb_used_data":704548,"kb_used_omap":12,"kb_used_meta":1172275,"kb_avail":281238604,"statfs":{"total":289910292480,"available":287988330496,"internally_reserved":0,"allocated":721457152,"data_stored":720568392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1200409603},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,6,7,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24707072,"num_objects":-6032,"num_object_clones":0,"num_object_copies":-12064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-209,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6032,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004508"},"pg_stats":[{"pgid":"2.f","version":"47'17150","reported_seq":17229,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.596215+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:18.596215+0000","last_peered":"2024-09-15T21:28:18.596215+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:18.596215+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16392192,"num_objects":4002,"num_object_clones":0,"num_object_copies":8004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17150,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-130}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16681","reported_seq":16792,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.254595+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:14.254595+0000","last_peered":"2024-09-15T21:28:14.254595+0000","last_clean":"2024-09-15T21:28:14.254595+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:14.254595+0000","last_undegraded":"2024-09-15T21:28:14.254595+0000","last_fullsized":"2024-09-15T21:28:14.254595+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16356294,"num_objects":3995,"num_object_clones":0,"num_object_copies":7990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3995,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":14962,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17075","reported_seq":19858,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:14.278755+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:14.278755+0000","last_peered":"2024-09-15T21:28:14.278755+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:14.278755+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16506904,"num_objects":4031,"num_object_clones":0,"num_object_copies":8062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4031,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15175,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1326,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1089}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16387","reported_seq":16463,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.222798+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:14.222798+0000","last_peered":"2024-09-15T21:28:14.222798+0000","last_clean":"2024-09-15T21:28:14.222798+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:14.222798+0000","last_undegraded":"2024-09-15T21:28:14.222798+0000","last_fullsized":"2024-09-15T21:28:14.222798+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16084992,"num_objects":3927,"num_object_clones":0,"num_object_copies":7854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16387,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16701","reported_seq":16839,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:18.567221+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:18.567221+0000","last_peered":"2024-09-15T21:28:18.567221+0000","last_clean":"2024-09-15T21:28:18.567221+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:18.567221+0000","last_undegraded":"2024-09-15T21:28:18.567221+0000","last_fullsized":"2024-09-15T21:28:18.567221+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15581667,"num_objects":3805,"num_object_clones":0,"num_object_copies":7610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13156,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16739","reported_seq":16881,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.558737+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:18.558737+0000","last_peered":"2024-09-15T21:28:18.558737+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:18.558737+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15958499,"num_objects":3897,"num_object_clones":0,"num_object_copies":7794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13415,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-179},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16862","reported_seq":17004,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.531425+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:18.531425+0000","last_peered":"2024-09-15T21:28:18.531425+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:18.531425+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15598534,"num_objects":3810,"num_object_clones":0,"num_object_copies":7620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13173,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-351},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17042","reported_seq":17176,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:18.593373+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:18.593373+0000","last_peered":"2024-09-15T21:28:18.593373+0000","last_clean":"2024-09-15T21:28:18.593373+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:18.593373+0000","last_undegraded":"2024-09-15T21:28:18.593373+0000","last_fullsized":"2024-09-15T21:28:18.593373+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15671296,"num_objects":3826,"num_object_clones":0,"num_object_copies":7652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13814,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16545","reported_seq":16677,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:14.138953+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:14.138953+0000","last_peered":"2024-09-15T21:28:14.138953+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:14.138953+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16253411,"num_objects":3969,"num_object_clones":0,"num_object_copies":7938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3969,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14477,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16613","reported_seq":16749,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.557306+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:18.557306+0000","last_peered":"2024-09-15T21:28:18.557306+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:18.557306+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15847424,"num_objects":3869,"num_object_clones":0,"num_object_copies":7738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13286,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-151}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16618","reported_seq":16742,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:14.238968+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:14.238968+0000","last_peered":"2024-09-15T21:28:14.238968+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:14.238968+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16486883,"num_objects":4026,"num_object_clones":0,"num_object_copies":8052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4026,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14477,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-64}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16670","reported_seq":25134,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.232971+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:14.232971+0000","last_peered":"2024-09-15T21:28:14.232971+0000","last_clean":"2024-09-15T21:28:14.232971+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:14.232971+0000","last_undegraded":"2024-09-15T21:28:14.232971+0000","last_fullsized":"2024-09-15T21:28:14.232971+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15933440,"num_objects":3890,"num_object_clones":0,"num_object_copies":7780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14363,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16846","reported_seq":16985,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.588610+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:18.588610+0000","last_peered":"2024-09-15T21:28:18.588610+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:18.588610+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15728640,"num_objects":3840,"num_object_clones":0,"num_object_copies":7680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13790,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-254}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16775","reported_seq":16913,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.610441+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:18.610441+0000","last_peered":"2024-09-15T21:28:18.610441+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:18.610441+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15626240,"num_objects":3815,"num_object_clones":0,"num_object_copies":7630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13684,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-324}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16448","reported_seq":24763,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:14.806544+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:14.806544+0000","last_peered":"2024-09-15T21:28:14.806544+0000","last_clean":"2024-09-15T21:28:14.806544+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:14.806544+0000","last_undegraded":"2024-09-15T21:28:14.806544+0000","last_fullsized":"2024-09-15T21:28:14.806544+0000","mapping_epoch":46,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16138240,"num_objects":3940,"num_object_clones":0,"num_object_copies":7880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12932,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16881","reported_seq":17025,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.602421+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:18.602421+0000","last_peered":"2024-09-15T21:28:18.602421+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:18.602421+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16323526,"num_objects":3987,"num_object_clones":0,"num_object_copies":7974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3987,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13585,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-160},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":256488182,"num_objects":62629,"num_object_clones":0,"num_object_copies":125258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62629,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":227826,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9575,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":718151680,"data_stored":718006242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601420,"kb_used_data":258748,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93770420,"statfs":{"total":96636764160,"available":96020910080,"internally_reserved":0,"allocated":264957952,"data_stored":264672694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685812,"kb_used_data":252944,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93686028,"statfs":{"total":96636764160,"available":95934492672,"internally_reserved":0,"allocated":259014656,"data_stored":258729393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589684,"kb_used_data":192856,"kb_used_omap":4,"kb_used_meta":396795,"kb_avail":93782156,"statfs":{"total":96636764160,"available":96032927744,"internally_reserved":0,"allocated":197484544,"data_stored":197166305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":406318767},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257822720,"data_stored":257786590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":195801088,"data_stored":195731982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":264527872,"data_stored":264487670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:20.704 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:28:20.704 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17150", 'reported_seq': 17229, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:18.596215+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:28:18.596215+0000', 'last_peered': '2024-09-15T21:28:18.596215+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:28:18.596215+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16392192, 'num_objects': 4002, 'num_object_clones': 0, 'num_object_copies': 8004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4002, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17150, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -130}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:20.704 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:28:20.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'17075", 'reported_seq': 19858, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:14.278755+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:28:14.278755+0000', 'last_peered': '2024-09-15T21:28:14.278755+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:28:14.278755+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16506904, 'num_objects': 4031, 'num_object_clones': 0, 'num_object_copies': 8062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4031, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15175, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1326, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1089}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:20.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:20.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'16739", 'reported_seq': 16881, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:18.558737+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:28:18.558737+0000', 'last_peered': '2024-09-15T21:28:18.558737+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:28:18.558737+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15958499, 'num_objects': 3897, 'num_object_clones': 0, 'num_object_copies': 7794, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3897, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13415, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -179}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:20.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:20.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'16862", 'reported_seq': 17004, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:18.531425+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:28:18.531425+0000', 'last_peered': '2024-09-15T21:28:18.531425+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:28:18.531425+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15598534, 'num_objects': 3810, 'num_object_clones': 0, 'num_object_copies': 7620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3810, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13173, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -351}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:20.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:28:20.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'16545", 'reported_seq': 16677, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:14.138953+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:28:14.138953+0000', 'last_peered': '2024-09-15T21:28:14.138953+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:28:14.138953+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16253411, 'num_objects': 3969, 'num_object_clones': 0, 'num_object_copies': 7938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3969, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14477, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -118}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:20.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:20.705 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16613", 'reported_seq': 16749, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:18.557306+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:28:18.557306+0000', 'last_peered': '2024-09-15T21:28:18.557306+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:28:18.557306+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15847424, 'num_objects': 3869, 'num_object_clones': 0, 'num_object_copies': 7738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3869, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13286, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -151}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:20.705 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:28:20.706 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16618", 'reported_seq': 16742, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:14.238968+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:28:14.238968+0000', 'last_peered': '2024-09-15T21:28:14.238968+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:28:14.238968+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16486883, 'num_objects': 4026, 'num_object_clones': 0, 'num_object_copies': 8052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4026, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14477, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -64}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:20.706 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:20.706 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16846", 'reported_seq': 16985, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:18.588610+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:28:18.588610+0000', 'last_peered': '2024-09-15T21:28:18.588610+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:28:18.588610+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15728640, 'num_objects': 3840, 'num_object_clones': 0, 'num_object_copies': 7680, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3840, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13790, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -254}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:20.706 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:28:20.706 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'16775", 'reported_seq': 16913, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:18.610441+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:28:18.610441+0000', 'last_peered': '2024-09-15T21:28:18.610441+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:28:18.610441+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15626240, 'num_objects': 3815, 'num_object_clones': 0, 'num_object_copies': 7630, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3815, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13684, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -324}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:20.706 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:28:20.706 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'16881", 'reported_seq': 17025, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:18.602421+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:28:18.602421+0000', 'last_peered': '2024-09-15T21:28:18.602421+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:28:18.602421+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16323526, 'num_objects': 3987, 'num_object_clones': 0, 'num_object_copies': 7974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3987, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13585, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -160}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:20.706 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:21.367 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:21.367 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39899/125262 objects degraded (31.852%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":553,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":62631,"data_bytes":256488201,"bytes_used":1921961984,"bytes_avail":287988330496,"bytes_total":289910292480,"degraded_objects":39899,"degraded_total":125262,"degraded_ratio":0.31852437291437147,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":209,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":502},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:21.395 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:21.395 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:22.007 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:22.007 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:22.036 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-15T21:28:20.831176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":252269321,"num_objects":61601,"num_object_clones":0,"num_object_copies":123202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39798,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61601,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":228858,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9677,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160766,"ondisk_log_size":160766,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1877672,"kb_used_data":692824,"kb_used_omap":12,"kb_used_meta":1184755,"kb_avail":281237848,"statfs":{"total":289910292480,"available":287987556352,"internally_reserved":0,"allocated":709451776,"data_stored":708563016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1213189123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,2,3,5,6,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23408640,"num_objects":-5715,"num_object_clones":0,"num_object_copies":-11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5715,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004443"},"pg_stats":[{"pgid":"2.f","version":"47'17150","reported_seq":17229,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.596215+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:18.596215+0000","last_peered":"2024-09-15T21:28:18.596215+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:18.596215+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16392192,"num_objects":4002,"num_object_clones":0,"num_object_copies":8004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17150,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-130}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16839","reported_seq":16950,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.253876+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:19.253876+0000","last_peered":"2024-09-15T21:28:19.253876+0000","last_clean":"2024-09-15T21:28:19.253876+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:19.253876+0000","last_undegraded":"2024-09-15T21:28:19.253876+0000","last_fullsized":"2024-09-15T21:28:19.253876+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15709126,"num_objects":3837,"num_object_clones":0,"num_object_copies":7674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3837,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15120,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17219","reported_seq":20206,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.259637+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:19.259637+0000","last_peered":"2024-09-15T21:28:19.259637+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:19.259637+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15917080,"num_objects":3887,"num_object_clones":0,"num_object_copies":7774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2841,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3887,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15319,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1046}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16540","reported_seq":16616,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.302096+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:19.302096+0000","last_peered":"2024-09-15T21:28:19.302096+0000","last_clean":"2024-09-15T21:28:19.302096+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:19.302096+0000","last_undegraded":"2024-09-15T21:28:19.302096+0000","last_fullsized":"2024-09-15T21:28:19.302096+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15458304,"num_objects":3774,"num_object_clones":0,"num_object_copies":7548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16540,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16701","reported_seq":16839,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:18.567221+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:18.567221+0000","last_peered":"2024-09-15T21:28:18.567221+0000","last_clean":"2024-09-15T21:28:18.567221+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:18.567221+0000","last_undegraded":"2024-09-15T21:28:18.567221+0000","last_fullsized":"2024-09-15T21:28:18.567221+0000","mapping_epoch":34,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15581667,"num_objects":3805,"num_object_clones":0,"num_object_copies":7610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13156,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16739","reported_seq":16881,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.558737+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:18.558737+0000","last_peered":"2024-09-15T21:28:18.558737+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:18.558737+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15958499,"num_objects":3897,"num_object_clones":0,"num_object_copies":7794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13415,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-179},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16862","reported_seq":17004,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.531425+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:18.531425+0000","last_peered":"2024-09-15T21:28:18.531425+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:18.531425+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15598534,"num_objects":3810,"num_object_clones":0,"num_object_copies":7620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13173,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-351},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17042","reported_seq":17176,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:18.593373+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:18.593373+0000","last_peered":"2024-09-15T21:28:18.593373+0000","last_clean":"2024-09-15T21:28:18.593373+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:18.593373+0000","last_undegraded":"2024-09-15T21:28:18.593373+0000","last_fullsized":"2024-09-15T21:28:18.593373+0000","mapping_epoch":34,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15671296,"num_objects":3826,"num_object_clones":0,"num_object_copies":7652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13814,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16685","reported_seq":16817,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.221634+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:19.221634+0000","last_peered":"2024-09-15T21:28:19.221634+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:19.221634+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15679971,"num_objects":3829,"num_object_clones":0,"num_object_copies":7658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3829,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14617,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-258}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16613","reported_seq":16749,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.557306+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:18.557306+0000","last_peered":"2024-09-15T21:28:18.557306+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:18.557306+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15847424,"num_objects":3869,"num_object_clones":0,"num_object_copies":7738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13286,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-151}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318585+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318585+0000","last_peered":"2024-09-15T21:27:09.318585+0000","last_clean":"2024-09-15T21:27:09.318585+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318585+0000","last_undegraded":"2024-09-15T21:27:09.318585+0000","last_fullsized":"2024-09-15T21:27:09.318585+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16762","reported_seq":16886,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:19.265800+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:19.265800+0000","last_peered":"2024-09-15T21:28:19.265800+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:19.265800+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15897059,"num_objects":3882,"num_object_clones":0,"num_object_copies":7764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14621,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-208}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16825","reported_seq":25289,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.240212+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:19.240212+0000","last_peered":"2024-09-15T21:28:19.240212+0000","last_clean":"2024-09-15T21:28:19.240212+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:19.240212+0000","last_undegraded":"2024-09-15T21:28:19.240212+0000","last_fullsized":"2024-09-15T21:28:19.240212+0000","mapping_epoch":44,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15298560,"num_objects":3735,"num_object_clones":0,"num_object_copies":7470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14518,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16846","reported_seq":16985,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.588610+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:18.588610+0000","last_peered":"2024-09-15T21:28:18.588610+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:18.588610+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15728640,"num_objects":3840,"num_object_clones":0,"num_object_copies":7680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13790,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-254}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16775","reported_seq":16913,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.610441+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:18.610441+0000","last_peered":"2024-09-15T21:28:18.610441+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:18.610441+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15626240,"num_objects":3815,"num_object_clones":0,"num_object_copies":7630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13684,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-324}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16584","reported_seq":24899,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.787020+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:19.787020+0000","last_peered":"2024-09-15T21:28:19.787020+0000","last_clean":"2024-09-15T21:28:19.787020+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:19.787020+0000","last_undegraded":"2024-09-15T21:28:19.787020+0000","last_fullsized":"2024-09-15T21:28:19.787020+0000","mapping_epoch":46,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15581184,"num_objects":3804,"num_object_clones":0,"num_object_copies":7608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13068,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16881","reported_seq":17025,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:18.602421+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:18.602421+0000","last_peered":"2024-09-15T21:28:18.602421+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:18.602421+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16323526,"num_objects":3987,"num_object_clones":0,"num_object_copies":7974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3987,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13585,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-160},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":252269302,"num_objects":61599,"num_object_clones":0,"num_object_copies":123198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39798,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61599,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":228856,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9676,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":705982464,"data_stored":705837026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160763,"ondisk_log_size":160763,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600804,"kb_used_data":253972,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93771036,"statfs":{"total":96636764160,"available":96021540864,"internally_reserved":0,"allocated":260067328,"data_stored":259782070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685812,"kb_used_data":252944,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93686028,"statfs":{"total":96636764160,"available":95934492672,"internally_reserved":0,"allocated":259014656,"data_stored":258729393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591056,"kb_used_data":185908,"kb_used_omap":4,"kb_used_meta":405115,"kb_avail":93780784,"statfs":{"total":96636764160,"available":96031522816,"internally_reserved":0,"allocated":190369792,"data_stored":190051553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":414838447},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":257822720,"data_stored":257786590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188637184,"data_stored":188568078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":259522560,"data_stored":259482358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:25.038 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:25.601 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:25.602 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:25.629 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-15T21:28:24.832766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":243168009,"num_objects":59379,"num_object_clones":0,"num_object_copies":118758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39692,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59379,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":231080,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9783,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160788,"ondisk_log_size":160788,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1857272,"kb_used_data":680752,"kb_used_omap":12,"kb_used_meta":1176435,"kb_avail":281258248,"statfs":{"total":289910292480,"available":288008445952,"internally_reserved":0,"allocated":697090048,"data_stored":696201288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1204669443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,5,3,1,3,10,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28442624,"num_objects":-6944,"num_object_clones":0,"num_object_copies":-13888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-307,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6944,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004422"},"pg_stats":[{"pgid":"2.f","version":"47'17315","reported_seq":17395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.575752+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:23.575752+0000","last_peered":"2024-09-15T21:28:23.575752+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:23.575752+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15716352,"num_objects":3837,"num_object_clones":0,"num_object_copies":7674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17315,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-295}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16992","reported_seq":17103,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:24.294758+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:24.294758+0000","last_peered":"2024-09-15T21:28:24.294758+0000","last_clean":"2024-09-15T21:28:24.294758+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:24.294758+0000","last_undegraded":"2024-09-15T21:28:24.294758+0000","last_fullsized":"2024-09-15T21:28:24.294758+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15082438,"num_objects":3684,"num_object_clones":0,"num_object_copies":7368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3684,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15273,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17364","reported_seq":20563,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:24.322249+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:24.322249+0000","last_peered":"2024-09-15T21:28:24.322249+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:24.322249+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15323160,"num_objects":3742,"num_object_clones":0,"num_object_copies":7484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3742,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15464,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1007}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16684","reported_seq":16760,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:24.242342+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:24.242342+0000","last_peered":"2024-09-15T21:28:24.242342+0000","last_clean":"2024-09-15T21:28:24.242342+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:24.242342+0000","last_undegraded":"2024-09-15T21:28:24.242342+0000","last_fullsized":"2024-09-15T21:28:24.242342+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14868480,"num_objects":3630,"num_object_clones":0,"num_object_copies":7260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16684,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16845","reported_seq":16983,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:23.637506+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:23.637506+0000","last_peered":"2024-09-15T21:28:23.637506+0000","last_clean":"2024-09-15T21:28:23.637506+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:23.637506+0000","last_undegraded":"2024-09-15T21:28:23.637506+0000","last_fullsized":"2024-09-15T21:28:23.637506+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14991843,"num_objects":3661,"num_object_clones":0,"num_object_copies":7322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13300,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16884","reported_seq":17026,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.608132+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:23.608132+0000","last_peered":"2024-09-15T21:28:23.608132+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:23.608132+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15364579,"num_objects":3752,"num_object_clones":0,"num_object_copies":7504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13560,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-324},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17005","reported_seq":17147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.589921+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:23.589921+0000","last_peered":"2024-09-15T21:28:23.589921+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:23.589921+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15012806,"num_objects":3667,"num_object_clones":0,"num_object_copies":7334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13316,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-494},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17204","reported_seq":17338,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:23.570178+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:23.570178+0000","last_peered":"2024-09-15T21:28:23.570178+0000","last_clean":"2024-09-15T21:28:23.570178+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:23.570178+0000","last_undegraded":"2024-09-15T21:28:23.570178+0000","last_fullsized":"2024-09-15T21:28:23.570178+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":3664,"num_object_clones":0,"num_object_copies":7328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13976,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16835","reported_seq":16966,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:24.287602+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:24.287602+0000","last_peered":"2024-09-15T21:28:24.287602+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:24.287602+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15065571,"num_objects":3679,"num_object_clones":0,"num_object_copies":7358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14767,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16731","reported_seq":16867,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.646328+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:23.646328+0000","last_peered":"2024-09-15T21:28:23.646328+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:23.646328+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15364096,"num_objects":3751,"num_object_clones":0,"num_object_copies":7502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13404,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-269}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16922","reported_seq":17043,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:24.276803+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:24.276803+0000","last_peered":"2024-09-15T21:28:24.276803+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:24.276803+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15241699,"num_objects":3722,"num_object_clones":0,"num_object_copies":7444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14781,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-368}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16978","reported_seq":25441,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:24.286107+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:24.286107+0000","last_peered":"2024-09-15T21:28:24.286107+0000","last_clean":"2024-09-15T21:28:24.286107+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:24.286107+0000","last_undegraded":"2024-09-15T21:28:24.286107+0000","last_fullsized":"2024-09-15T21:28:24.286107+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14671872,"num_objects":3582,"num_object_clones":0,"num_object_copies":7164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14671,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16976","reported_seq":17114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.586954+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:23.586954+0000","last_peered":"2024-09-15T21:28:23.586954+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:23.586954+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15196160,"num_objects":3710,"num_object_clones":0,"num_object_copies":7420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13920,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16940","reported_seq":17078,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.586908+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:23.586908+0000","last_peered":"2024-09-15T21:28:23.586908+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:23.586908+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14950400,"num_objects":3650,"num_object_clones":0,"num_object_copies":7300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13849,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-489}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16584","reported_seq":24899,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.787020+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:19.787020+0000","last_peered":"2024-09-15T21:28:19.787020+0000","last_clean":"2024-09-15T21:28:19.787020+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:19.787020+0000","last_undegraded":"2024-09-15T21:28:19.787020+0000","last_fullsized":"2024-09-15T21:28:19.787020+0000","mapping_epoch":46,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15581184,"num_objects":3804,"num_object_clones":0,"num_object_copies":7608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13068,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17026","reported_seq":17170,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.592288+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:23.592288+0000","last_peered":"2024-09-15T21:28:23.592288+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:23.592288+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15729606,"num_objects":3842,"num_object_clones":0,"num_object_copies":7684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13730,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-305},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":243167990,"num_objects":59377,"num_object_clones":0,"num_object_copies":118754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39692,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59377,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":231078,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9782,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":693862400,"data_stored":693716962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160785,"ondisk_log_size":160785,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583572,"kb_used_data":249220,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93788268,"statfs":{"total":96636764160,"available":96039186432,"internally_reserved":0,"allocated":255201280,"data_stored":254916022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682644,"kb_used_data":245624,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93689196,"statfs":{"total":96636764160,"available":95937736704,"internally_reserved":0,"allocated":251518976,"data_stored":251233713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,0,1,6,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591056,"kb_used_data":185908,"kb_used_omap":4,"kb_used_meta":405115,"kb_avail":93780784,"statfs":{"total":96636764160,"available":96031522816,"internally_reserved":0,"allocated":190369792,"data_stored":190051553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":414838447},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":250548224,"data_stored":250512094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188637184,"data_stored":188568078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":254676992,"data_stored":254636790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:25.630 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:26.203 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:26.205 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:26.227 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-15T21:28:24.832766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":243168009,"num_objects":59379,"num_object_clones":0,"num_object_copies":118758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39692,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59379,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":231080,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9783,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160788,"ondisk_log_size":160788,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1857272,"kb_used_data":680752,"kb_used_omap":12,"kb_used_meta":1176435,"kb_avail":281258248,"statfs":{"total":289910292480,"available":288008445952,"internally_reserved":0,"allocated":697090048,"data_stored":696201288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1204669443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,5,3,1,3,10,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28442624,"num_objects":-6944,"num_object_clones":0,"num_object_copies":-13888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-307,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6944,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004422"},"pg_stats":[{"pgid":"2.f","version":"47'17315","reported_seq":17395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.575752+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:23.575752+0000","last_peered":"2024-09-15T21:28:23.575752+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:23.575752+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15716352,"num_objects":3837,"num_object_clones":0,"num_object_copies":7674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17315,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-295}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16992","reported_seq":17103,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:24.294758+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:24.294758+0000","last_peered":"2024-09-15T21:28:24.294758+0000","last_clean":"2024-09-15T21:28:24.294758+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:24.294758+0000","last_undegraded":"2024-09-15T21:28:24.294758+0000","last_fullsized":"2024-09-15T21:28:24.294758+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15082438,"num_objects":3684,"num_object_clones":0,"num_object_copies":7368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3684,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15273,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17364","reported_seq":20563,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:24.322249+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:24.322249+0000","last_peered":"2024-09-15T21:28:24.322249+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:24.322249+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15323160,"num_objects":3742,"num_object_clones":0,"num_object_copies":7484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3742,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15464,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1007}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16684","reported_seq":16760,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:24.242342+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:24.242342+0000","last_peered":"2024-09-15T21:28:24.242342+0000","last_clean":"2024-09-15T21:28:24.242342+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:24.242342+0000","last_undegraded":"2024-09-15T21:28:24.242342+0000","last_fullsized":"2024-09-15T21:28:24.242342+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14868480,"num_objects":3630,"num_object_clones":0,"num_object_copies":7260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16684,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16845","reported_seq":16983,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:23.637506+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:23.637506+0000","last_peered":"2024-09-15T21:28:23.637506+0000","last_clean":"2024-09-15T21:28:23.637506+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:23.637506+0000","last_undegraded":"2024-09-15T21:28:23.637506+0000","last_fullsized":"2024-09-15T21:28:23.637506+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14991843,"num_objects":3661,"num_object_clones":0,"num_object_copies":7322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13300,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16884","reported_seq":17026,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.608132+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:23.608132+0000","last_peered":"2024-09-15T21:28:23.608132+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:23.608132+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15364579,"num_objects":3752,"num_object_clones":0,"num_object_copies":7504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13560,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-324},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17005","reported_seq":17147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.589921+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:23.589921+0000","last_peered":"2024-09-15T21:28:23.589921+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:23.589921+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15012806,"num_objects":3667,"num_object_clones":0,"num_object_copies":7334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13316,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-494},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17204","reported_seq":17338,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:23.570178+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:23.570178+0000","last_peered":"2024-09-15T21:28:23.570178+0000","last_clean":"2024-09-15T21:28:23.570178+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:23.570178+0000","last_undegraded":"2024-09-15T21:28:23.570178+0000","last_fullsized":"2024-09-15T21:28:23.570178+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":3664,"num_object_clones":0,"num_object_copies":7328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13976,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16835","reported_seq":16966,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:24.287602+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:24.287602+0000","last_peered":"2024-09-15T21:28:24.287602+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:24.287602+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15065571,"num_objects":3679,"num_object_clones":0,"num_object_copies":7358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14767,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16731","reported_seq":16867,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.646328+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:23.646328+0000","last_peered":"2024-09-15T21:28:23.646328+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:23.646328+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15364096,"num_objects":3751,"num_object_clones":0,"num_object_copies":7502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13404,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-269}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16922","reported_seq":17043,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:24.276803+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:24.276803+0000","last_peered":"2024-09-15T21:28:24.276803+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:24.276803+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15241699,"num_objects":3722,"num_object_clones":0,"num_object_copies":7444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14781,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-368}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16978","reported_seq":25441,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:24.286107+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:24.286107+0000","last_peered":"2024-09-15T21:28:24.286107+0000","last_clean":"2024-09-15T21:28:24.286107+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:24.286107+0000","last_undegraded":"2024-09-15T21:28:24.286107+0000","last_fullsized":"2024-09-15T21:28:24.286107+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14671872,"num_objects":3582,"num_object_clones":0,"num_object_copies":7164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14671,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16976","reported_seq":17114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.586954+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:23.586954+0000","last_peered":"2024-09-15T21:28:23.586954+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:23.586954+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15196160,"num_objects":3710,"num_object_clones":0,"num_object_copies":7420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13920,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16940","reported_seq":17078,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.586908+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:23.586908+0000","last_peered":"2024-09-15T21:28:23.586908+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:23.586908+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14950400,"num_objects":3650,"num_object_clones":0,"num_object_copies":7300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13849,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-489}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16584","reported_seq":24899,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:19.787020+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:19.787020+0000","last_peered":"2024-09-15T21:28:19.787020+0000","last_clean":"2024-09-15T21:28:19.787020+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:19.787020+0000","last_undegraded":"2024-09-15T21:28:19.787020+0000","last_fullsized":"2024-09-15T21:28:19.787020+0000","mapping_epoch":46,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15581184,"num_objects":3804,"num_object_clones":0,"num_object_copies":7608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13068,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17026","reported_seq":17170,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.592288+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:23.592288+0000","last_peered":"2024-09-15T21:28:23.592288+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:23.592288+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15729606,"num_objects":3842,"num_object_clones":0,"num_object_copies":7684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13730,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-305},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":243167990,"num_objects":59377,"num_object_clones":0,"num_object_copies":118754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39692,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59377,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":231078,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9782,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":693862400,"data_stored":693716962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160785,"ondisk_log_size":160785,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583572,"kb_used_data":249220,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93788268,"statfs":{"total":96636764160,"available":96039186432,"internally_reserved":0,"allocated":255201280,"data_stored":254916022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682644,"kb_used_data":245624,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93689196,"statfs":{"total":96636764160,"available":95937736704,"internally_reserved":0,"allocated":251518976,"data_stored":251233713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,0,1,6,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591056,"kb_used_data":185908,"kb_used_omap":4,"kb_used_meta":405115,"kb_avail":93780784,"statfs":{"total":96636764160,"available":96031522816,"internally_reserved":0,"allocated":190369792,"data_stored":190051553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":414838447},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,1,1,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":250548224,"data_stored":250512094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188637184,"data_stored":188568078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":254676992,"data_stored":254636790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:26.229 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:28:26.229 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17315", 'reported_seq': 17395, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:23.575752+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:28:23.575752+0000', 'last_peered': '2024-09-15T21:28:23.575752+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:28:23.575752+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15716352, 'num_objects': 3837, 'num_object_clones': 0, 'num_object_copies': 7674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3837, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17315, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -295}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:26.229 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:28:26.229 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'17364", 'reported_seq': 20563, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:24.322249+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:28:24.322249+0000', 'last_peered': '2024-09-15T21:28:24.322249+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:28:24.322249+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15323160, 'num_objects': 3742, 'num_object_clones': 0, 'num_object_copies': 7484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2735, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3742, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15464, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1533, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1007}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:26.229 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:26.229 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'16884", 'reported_seq': 17026, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:23.608132+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:28:23.608132+0000', 'last_peered': '2024-09-15T21:28:23.608132+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:28:23.608132+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15364579, 'num_objects': 3752, 'num_object_clones': 0, 'num_object_copies': 7504, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3752, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13560, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -324}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:26.229 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:26.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17005", 'reported_seq': 17147, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:23.589921+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:28:23.589921+0000', 'last_peered': '2024-09-15T21:28:23.589921+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:28:23.589921+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15012806, 'num_objects': 3667, 'num_object_clones': 0, 'num_object_copies': 7334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3667, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13316, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -494}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:26.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:28:26.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'16835", 'reported_seq': 16966, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:24.287602+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:28:24.287602+0000', 'last_peered': '2024-09-15T21:28:24.287602+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:28:24.287602+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15065571, 'num_objects': 3679, 'num_object_clones': 0, 'num_object_copies': 7358, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3679, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14767, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -408}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:26.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:26.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16731", 'reported_seq': 16867, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:23.646328+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:28:23.646328+0000', 'last_peered': '2024-09-15T21:28:23.646328+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:28:23.646328+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15364096, 'num_objects': 3751, 'num_object_clones': 0, 'num_object_copies': 7502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3751, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13404, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -269}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:26.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:28:26.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'16922", 'reported_seq': 17043, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:24.276803+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:28:24.276803+0000', 'last_peered': '2024-09-15T21:28:24.276803+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:28:24.276803+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15241699, 'num_objects': 3722, 'num_object_clones': 0, 'num_object_copies': 7444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3722, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14781, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -368}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:26.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:26.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'16976", 'reported_seq': 17114, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:23.586954+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:28:23.586954+0000', 'last_peered': '2024-09-15T21:28:23.586954+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:28:23.586954+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15196160, 'num_objects': 3710, 'num_object_clones': 0, 'num_object_copies': 7420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3710, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13920, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -384}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:26.230 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:28:26.230 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'16940", 'reported_seq': 17078, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:23.586908+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:28:23.586908+0000', 'last_peered': '2024-09-15T21:28:23.586908+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:28:23.586908+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14950400, 'num_objects': 3650, 'num_object_clones': 0, 'num_object_copies': 7300, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3650, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13849, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -489}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:26.231 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:28:26.231 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'17026", 'reported_seq': 17170, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:23.592288+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:28:23.592288+0000', 'last_peered': '2024-09-15T21:28:23.592288+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:28:23.592288+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15729606, 'num_objects': 3842, 'num_object_clones': 0, 'num_object_copies': 7684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3842, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13730, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -305}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:26.231 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:26.945 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:26.945 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39692/118758 objects degraded (33.423%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":558,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":59379,"data_bytes":243168009,"bytes_used":1901846528,"bytes_avail":288008445952,"bytes_total":289910292480,"degraded_objects":39692,"degraded_total":118758,"degraded_ratio":0.33422590478115161,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":307,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":578},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:26.971 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:26.972 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:27.535 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:27.537 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:27.566 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":319,"stamp":"2024-09-15T21:28:26.833568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":242549513,"num_objects":59228,"num_object_clones":0,"num_object_copies":118456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39692,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59228,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":231231,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9783,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1854520,"kb_used_data":672440,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281261000,"statfs":{"total":289910292480,"available":288011264000,"internally_reserved":0,"allocated":688578560,"data_stored":687689800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1210371075},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,4,2,3,11,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19431424,"num_objects":-4744,"num_object_clones":0,"num_object_copies":-9488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-207,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4744,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004566"},"pg_stats":[{"pgid":"2.f","version":"47'17315","reported_seq":17395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.575752+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:23.575752+0000","last_peered":"2024-09-15T21:28:23.575752+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:23.575752+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15716352,"num_objects":3837,"num_object_clones":0,"num_object_copies":7674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17315,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-295}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16992","reported_seq":17103,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:24.294758+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:24.294758+0000","last_peered":"2024-09-15T21:28:24.294758+0000","last_clean":"2024-09-15T21:28:24.294758+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:24.294758+0000","last_undegraded":"2024-09-15T21:28:24.294758+0000","last_fullsized":"2024-09-15T21:28:24.294758+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15082438,"num_objects":3684,"num_object_clones":0,"num_object_copies":7368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3684,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15273,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17364","reported_seq":20563,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:24.322249+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:24.322249+0000","last_peered":"2024-09-15T21:28:24.322249+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:24.322249+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15323160,"num_objects":3742,"num_object_clones":0,"num_object_copies":7484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2735,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3742,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15464,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1007}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16684","reported_seq":16760,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:24.242342+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:24.242342+0000","last_peered":"2024-09-15T21:28:24.242342+0000","last_clean":"2024-09-15T21:28:24.242342+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:24.242342+0000","last_undegraded":"2024-09-15T21:28:24.242342+0000","last_fullsized":"2024-09-15T21:28:24.242342+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14868480,"num_objects":3630,"num_object_clones":0,"num_object_copies":7260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16684,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16845","reported_seq":16983,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:23.637506+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:23.637506+0000","last_peered":"2024-09-15T21:28:23.637506+0000","last_clean":"2024-09-15T21:28:23.637506+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:23.637506+0000","last_undegraded":"2024-09-15T21:28:23.637506+0000","last_fullsized":"2024-09-15T21:28:23.637506+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14991843,"num_objects":3661,"num_object_clones":0,"num_object_copies":7322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13300,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16884","reported_seq":17026,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.608132+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:23.608132+0000","last_peered":"2024-09-15T21:28:23.608132+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:23.608132+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15364579,"num_objects":3752,"num_object_clones":0,"num_object_copies":7504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3752,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13560,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-324},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17005","reported_seq":17147,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.589921+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:23.589921+0000","last_peered":"2024-09-15T21:28:23.589921+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:23.589921+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15012806,"num_objects":3667,"num_object_clones":0,"num_object_copies":7334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13316,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-494},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17204","reported_seq":17338,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:23.570178+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:23.570178+0000","last_peered":"2024-09-15T21:28:23.570178+0000","last_clean":"2024-09-15T21:28:23.570178+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:23.570178+0000","last_undegraded":"2024-09-15T21:28:23.570178+0000","last_fullsized":"2024-09-15T21:28:23.570178+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15007744,"num_objects":3664,"num_object_clones":0,"num_object_copies":7328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13976,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16835","reported_seq":16966,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:24.287602+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:24.287602+0000","last_peered":"2024-09-15T21:28:24.287602+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:24.287602+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15065571,"num_objects":3679,"num_object_clones":0,"num_object_copies":7358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14767,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16731","reported_seq":16867,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.646328+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:23.646328+0000","last_peered":"2024-09-15T21:28:23.646328+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:23.646328+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15364096,"num_objects":3751,"num_object_clones":0,"num_object_copies":7502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13404,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-269}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16922","reported_seq":17043,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:24.276803+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:24.276803+0000","last_peered":"2024-09-15T21:28:24.276803+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:24.276803+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15241699,"num_objects":3722,"num_object_clones":0,"num_object_copies":7444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14781,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-368}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16978","reported_seq":25441,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:24.286107+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:24.286107+0000","last_peered":"2024-09-15T21:28:24.286107+0000","last_clean":"2024-09-15T21:28:24.286107+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:24.286107+0000","last_undegraded":"2024-09-15T21:28:24.286107+0000","last_fullsized":"2024-09-15T21:28:24.286107+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14671872,"num_objects":3582,"num_object_clones":0,"num_object_copies":7164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14671,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16976","reported_seq":17114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.586954+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:23.586954+0000","last_peered":"2024-09-15T21:28:23.586954+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:23.586954+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15196160,"num_objects":3710,"num_object_clones":0,"num_object_copies":7420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13920,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16940","reported_seq":17078,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.586908+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:23.586908+0000","last_peered":"2024-09-15T21:28:23.586908+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:23.586908+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14950400,"num_objects":3650,"num_object_clones":0,"num_object_copies":7300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13849,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-489}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16735","reported_seq":25050,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:24.871647+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:24.871647+0000","last_peered":"2024-09-15T21:28:24.871647+0000","last_clean":"2024-09-15T21:28:24.871647+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:24.871647+0000","last_undegraded":"2024-09-15T21:28:24.871647+0000","last_fullsized":"2024-09-15T21:28:24.871647+0000","mapping_epoch":46,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14962688,"num_objects":3653,"num_object_clones":0,"num_object_copies":7306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13219,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17026","reported_seq":17170,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:23.592288+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:23.592288+0000","last_peered":"2024-09-15T21:28:23.592288+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:23.592288+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15729606,"num_objects":3842,"num_object_clones":0,"num_object_copies":7684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3842,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13730,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-305},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":242549494,"num_objects":59226,"num_object_clones":0,"num_object_copies":118452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39692,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59226,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":231229,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9782,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":686501888,"data_stored":686356450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160736,"ondisk_log_size":160736,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583572,"kb_used_data":249220,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93788268,"statfs":{"total":96636764160,"available":96039186432,"internally_reserved":0,"allocated":255201280,"data_stored":254916022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682644,"kb_used_data":245624,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93689196,"statfs":{"total":96636764160,"available":95937736704,"internally_reserved":0,"allocated":251518976,"data_stored":251233713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,0,1,6,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":588304,"kb_used_data":177596,"kb_used_omap":4,"kb_used_meta":410683,"kb_avail":93783536,"statfs":{"total":96636764160,"available":96034340864,"internally_reserved":0,"allocated":181858304,"data_stored":181540065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":420540079},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":250548224,"data_stored":250512094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":181276672,"data_stored":181207566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":254676992,"data_stored":254636790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:30.569 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:31.139 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:31.142 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:31.171 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-15T21:28:30.835104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":233022217,"num_objects":56902,"num_object_clones":0,"num_object_copies":113804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56902,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":233557,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9891,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1840364,"kb_used_data":654136,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281275156,"statfs":{"total":289910292480,"available":288025759744,"internally_reserved":0,"allocated":669835264,"data_stored":668946504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1214630915},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,8,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23465984,"num_objects":-5729,"num_object_clones":0,"num_object_copies":-11458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-315,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5729,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004623"},"pg_stats":[{"pgid":"2.f","version":"47'17480","reported_seq":17561,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.648873+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:28.648873+0000","last_peered":"2024-09-15T21:28:28.648873+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:28.648873+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15040512,"num_objects":3672,"num_object_clones":0,"num_object_copies":7344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17480,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-460}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17133","reported_seq":17242,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:29.292679+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:29.292679+0000","last_peered":"2024-09-15T21:28:29.292679+0000","last_clean":"2024-09-15T21:28:29.292679+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:29.292679+0000","last_undegraded":"2024-09-15T21:28:29.292679+0000","last_fullsized":"2024-09-15T21:28:29.292679+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14504902,"num_objects":3543,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15414,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17527","reported_seq":20942,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:29.316874+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:29.316874+0000","last_peered":"2024-09-15T21:28:29.316874+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:29.316874+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14655512,"num_objects":3579,"num_object_clones":0,"num_object_copies":7158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15627,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1641,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":952}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16838","reported_seq":16914,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:29.293096+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:29.293096+0000","last_peered":"2024-09-15T21:28:29.293096+0000","last_clean":"2024-09-15T21:28:29.293096+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:29.293096+0000","last_undegraded":"2024-09-15T21:28:29.293096+0000","last_fullsized":"2024-09-15T21:28:29.293096+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14237696,"num_objects":3476,"num_object_clones":0,"num_object_copies":6952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16838,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16992","reported_seq":17130,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:28.695182+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:28.695182+0000","last_peered":"2024-09-15T21:28:28.695182+0000","last_clean":"2024-09-15T21:28:28.695182+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:28.695182+0000","last_undegraded":"2024-09-15T21:28:28.695182+0000","last_fullsized":"2024-09-15T21:28:28.695182+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14389731,"num_objects":3514,"num_object_clones":0,"num_object_copies":7028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13447,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17025","reported_seq":17167,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.624143+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:28.624143+0000","last_peered":"2024-09-15T21:28:28.624143+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:28.624143+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14787043,"num_objects":3611,"num_object_clones":0,"num_object_copies":7222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13701,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-465},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17165","reported_seq":17301,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.662355+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:28.662355+0000","last_peered":"2024-09-15T21:28:28.662355+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:28.662355+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14357446,"num_objects":3507,"num_object_clones":0,"num_object_copies":7014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13476,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-654},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17331","reported_seq":17465,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:28.574920+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:28.574920+0000","last_peered":"2024-09-15T21:28:28.574920+0000","last_clean":"2024-09-15T21:28:28.574920+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:28.574920+0000","last_undegraded":"2024-09-15T21:28:28.574920+0000","last_fullsized":"2024-09-15T21:28:28.574920+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14487552,"num_objects":3537,"num_object_clones":0,"num_object_copies":7074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14103,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16986","reported_seq":17118,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:29.298291+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:29.298291+0000","last_peered":"2024-09-15T21:28:29.298291+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:29.298291+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14447075,"num_objects":3528,"num_object_clones":0,"num_object_copies":7056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14918,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-559}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16865","reported_seq":17002,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.671855+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:28.671855+0000","last_peered":"2024-09-15T21:28:28.671855+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:28.671855+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14815232,"num_objects":3617,"num_object_clones":0,"num_object_copies":7234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13538,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-403}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17068","reported_seq":17192,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:29.273115+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:29.273115+0000","last_peered":"2024-09-15T21:28:29.273115+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:29.273115+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14643683,"num_objects":3576,"num_object_clones":0,"num_object_copies":7152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3576,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14927,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-514}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17104","reported_seq":25568,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:29.272437+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:29.272437+0000","last_peered":"2024-09-15T21:28:29.272437+0000","last_clean":"2024-09-15T21:28:29.272437+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:29.272437+0000","last_undegraded":"2024-09-15T21:28:29.272437+0000","last_fullsized":"2024-09-15T21:28:29.272437+0000","mapping_epoch":44,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14155776,"num_objects":3456,"num_object_clones":0,"num_object_copies":6912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14797,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17118","reported_seq":17257,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.647640+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:28.647640+0000","last_peered":"2024-09-15T21:28:28.647640+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:28.647640+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14614528,"num_objects":3568,"num_object_clones":0,"num_object_copies":7136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14062,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17087","reported_seq":17225,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.536360+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:28.536360+0000","last_peered":"2024-09-15T21:28:28.536360+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:28.536360+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14348288,"num_objects":3503,"num_object_clones":0,"num_object_copies":7006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13996,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-636}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16874","reported_seq":25189,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:29.904466+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:29.904466+0000","last_peered":"2024-09-15T21:28:29.904466+0000","last_clean":"2024-09-15T21:28:29.904466+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:29.904466+0000","last_undegraded":"2024-09-15T21:28:29.904466+0000","last_fullsized":"2024-09-15T21:28:29.904466+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14393344,"num_objects":3514,"num_object_clones":0,"num_object_copies":7028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13358,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17169","reported_seq":17313,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.666334+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:28.666334+0000","last_peered":"2024-09-15T21:28:28.666334+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:28.666334+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15143878,"num_objects":3699,"num_object_clones":0,"num_object_copies":7398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3699,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13873,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-448},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":233022198,"num_objects":56900,"num_object_clones":0,"num_object_copies":113800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56900,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":233555,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9890,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":667447296,"data_stored":667301858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583120,"kb_used_data":244608,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93788720,"statfs":{"total":96636764160,"available":96039649280,"internally_reserved":0,"allocated":250478592,"data_stored":250193334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667428,"kb_used_data":238736,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93704412,"statfs":{"total":96636764160,"available":95953317888,"internally_reserved":0,"allocated":244465664,"data_stored":244180401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589816,"kb_used_data":170792,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93782024,"statfs":{"total":96636764160,"available":96032792576,"internally_reserved":0,"allocated":174891008,"data_stored":174572769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":429059759},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":243511296,"data_stored":243475166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":174002176,"data_stored":173933070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":249933824,"data_stored":249893622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:31.172 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:31.726 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:31.727 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:31.749 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-09-15T21:28:30.835104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":233022217,"num_objects":56902,"num_object_clones":0,"num_object_copies":113804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56902,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":233557,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9891,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1840364,"kb_used_data":654136,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281275156,"statfs":{"total":289910292480,"available":288025759744,"internally_reserved":0,"allocated":669835264,"data_stored":668946504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1214630915},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,8,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23465984,"num_objects":-5729,"num_object_clones":0,"num_object_copies":-11458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-315,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5729,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004623"},"pg_stats":[{"pgid":"2.f","version":"47'17480","reported_seq":17561,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.648873+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:28.648873+0000","last_peered":"2024-09-15T21:28:28.648873+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:28.648873+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15040512,"num_objects":3672,"num_object_clones":0,"num_object_copies":7344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17480,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-460}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17133","reported_seq":17242,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:29.292679+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:29.292679+0000","last_peered":"2024-09-15T21:28:29.292679+0000","last_clean":"2024-09-15T21:28:29.292679+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:29.292679+0000","last_undegraded":"2024-09-15T21:28:29.292679+0000","last_fullsized":"2024-09-15T21:28:29.292679+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14504902,"num_objects":3543,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15414,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17527","reported_seq":20942,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:29.316874+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:29.316874+0000","last_peered":"2024-09-15T21:28:29.316874+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:29.316874+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14655512,"num_objects":3579,"num_object_clones":0,"num_object_copies":7158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15627,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1641,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":952}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16838","reported_seq":16914,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:29.293096+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:29.293096+0000","last_peered":"2024-09-15T21:28:29.293096+0000","last_clean":"2024-09-15T21:28:29.293096+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:29.293096+0000","last_undegraded":"2024-09-15T21:28:29.293096+0000","last_fullsized":"2024-09-15T21:28:29.293096+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14237696,"num_objects":3476,"num_object_clones":0,"num_object_copies":6952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16838,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16992","reported_seq":17130,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:28.695182+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:28.695182+0000","last_peered":"2024-09-15T21:28:28.695182+0000","last_clean":"2024-09-15T21:28:28.695182+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:28.695182+0000","last_undegraded":"2024-09-15T21:28:28.695182+0000","last_fullsized":"2024-09-15T21:28:28.695182+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14389731,"num_objects":3514,"num_object_clones":0,"num_object_copies":7028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13447,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17025","reported_seq":17167,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.624143+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:28.624143+0000","last_peered":"2024-09-15T21:28:28.624143+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:28.624143+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14787043,"num_objects":3611,"num_object_clones":0,"num_object_copies":7222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13701,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-465},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17165","reported_seq":17301,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.662355+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:28.662355+0000","last_peered":"2024-09-15T21:28:28.662355+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:28.662355+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14357446,"num_objects":3507,"num_object_clones":0,"num_object_copies":7014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13476,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-654},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17331","reported_seq":17465,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:28.574920+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:28.574920+0000","last_peered":"2024-09-15T21:28:28.574920+0000","last_clean":"2024-09-15T21:28:28.574920+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:28.574920+0000","last_undegraded":"2024-09-15T21:28:28.574920+0000","last_fullsized":"2024-09-15T21:28:28.574920+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14487552,"num_objects":3537,"num_object_clones":0,"num_object_copies":7074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14103,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16986","reported_seq":17118,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:29.298291+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:29.298291+0000","last_peered":"2024-09-15T21:28:29.298291+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:29.298291+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14447075,"num_objects":3528,"num_object_clones":0,"num_object_copies":7056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14918,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-559}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16865","reported_seq":17002,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.671855+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:28.671855+0000","last_peered":"2024-09-15T21:28:28.671855+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:28.671855+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14815232,"num_objects":3617,"num_object_clones":0,"num_object_copies":7234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13538,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-403}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17068","reported_seq":17192,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:29.273115+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:29.273115+0000","last_peered":"2024-09-15T21:28:29.273115+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:29.273115+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14643683,"num_objects":3576,"num_object_clones":0,"num_object_copies":7152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3576,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14927,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-514}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17104","reported_seq":25568,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:29.272437+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:29.272437+0000","last_peered":"2024-09-15T21:28:29.272437+0000","last_clean":"2024-09-15T21:28:29.272437+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:29.272437+0000","last_undegraded":"2024-09-15T21:28:29.272437+0000","last_fullsized":"2024-09-15T21:28:29.272437+0000","mapping_epoch":44,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14155776,"num_objects":3456,"num_object_clones":0,"num_object_copies":6912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14797,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17118","reported_seq":17257,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.647640+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:28.647640+0000","last_peered":"2024-09-15T21:28:28.647640+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:28.647640+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14614528,"num_objects":3568,"num_object_clones":0,"num_object_copies":7136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14062,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17087","reported_seq":17225,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.536360+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:28.536360+0000","last_peered":"2024-09-15T21:28:28.536360+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:28.536360+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14348288,"num_objects":3503,"num_object_clones":0,"num_object_copies":7006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13996,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-636}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16874","reported_seq":25189,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:29.904466+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:29.904466+0000","last_peered":"2024-09-15T21:28:29.904466+0000","last_clean":"2024-09-15T21:28:29.904466+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:29.904466+0000","last_undegraded":"2024-09-15T21:28:29.904466+0000","last_fullsized":"2024-09-15T21:28:29.904466+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14393344,"num_objects":3514,"num_object_clones":0,"num_object_copies":7028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13358,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17169","reported_seq":17313,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.666334+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:28.666334+0000","last_peered":"2024-09-15T21:28:28.666334+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:28.666334+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15143878,"num_objects":3699,"num_object_clones":0,"num_object_copies":7398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3699,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13873,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-448},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":233022198,"num_objects":56900,"num_object_clones":0,"num_object_copies":113800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56900,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":233555,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9890,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":667447296,"data_stored":667301858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583120,"kb_used_data":244608,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93788720,"statfs":{"total":96636764160,"available":96039649280,"internally_reserved":0,"allocated":250478592,"data_stored":250193334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667428,"kb_used_data":238736,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93704412,"statfs":{"total":96636764160,"available":95953317888,"internally_reserved":0,"allocated":244465664,"data_stored":244180401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589816,"kb_used_data":170792,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93782024,"statfs":{"total":96636764160,"available":96032792576,"internally_reserved":0,"allocated":174891008,"data_stored":174572769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":429059759},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":243511296,"data_stored":243475166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":174002176,"data_stored":173933070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":249933824,"data_stored":249893622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:31.750 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:28:31.751 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17480", 'reported_seq': 17561, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:28.648873+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:28:28.648873+0000', 'last_peered': '2024-09-15T21:28:28.648873+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:28:28.648873+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15040512, 'num_objects': 3672, 'num_object_clones': 0, 'num_object_copies': 7344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17480, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -460}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:31.751 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:28:31.751 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'17527", 'reported_seq': 20942, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:29.316874+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:28:29.316874+0000', 'last_peered': '2024-09-15T21:28:29.316874+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:28:29.316874+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14655512, 'num_objects': 3579, 'num_object_clones': 0, 'num_object_copies': 7158, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2627, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3579, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15627, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1641, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 952}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:31.751 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:31.751 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17025", 'reported_seq': 17167, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:28.624143+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:28:28.624143+0000', 'last_peered': '2024-09-15T21:28:28.624143+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:28:28.624143+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14787043, 'num_objects': 3611, 'num_object_clones': 0, 'num_object_copies': 7222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3611, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13701, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -465}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:31.751 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:31.751 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17165", 'reported_seq': 17301, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:28.662355+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:28:28.662355+0000', 'last_peered': '2024-09-15T21:28:28.662355+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:28:28.662355+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14357446, 'num_objects': 3507, 'num_object_clones': 0, 'num_object_copies': 7014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3507, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13476, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -654}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:31.751 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:28:31.751 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'16986", 'reported_seq': 17118, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:29.298291+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:28:29.298291+0000', 'last_peered': '2024-09-15T21:28:29.298291+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:28:29.298291+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14447075, 'num_objects': 3528, 'num_object_clones': 0, 'num_object_copies': 7056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3528, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14918, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -559}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:31.751 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:31.752 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'16865", 'reported_seq': 17002, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:28.671855+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:28:28.671855+0000', 'last_peered': '2024-09-15T21:28:28.671855+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:28:28.671855+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14815232, 'num_objects': 3617, 'num_object_clones': 0, 'num_object_copies': 7234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3617, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13538, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -403}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:31.752 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:28:31.752 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17068", 'reported_seq': 17192, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:29.273115+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:28:29.273115+0000', 'last_peered': '2024-09-15T21:28:29.273115+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:28:29.273115+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14643683, 'num_objects': 3576, 'num_object_clones': 0, 'num_object_copies': 7152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3576, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14927, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -514}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:31.752 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:31.752 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17118", 'reported_seq': 17257, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:28.647640+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:28:28.647640+0000', 'last_peered': '2024-09-15T21:28:28.647640+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:28:28.647640+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14614528, 'num_objects': 3568, 'num_object_clones': 0, 'num_object_copies': 7136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3568, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14062, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -526}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:31.752 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:28:31.752 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'17087", 'reported_seq': 17225, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:28.536360+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:28:28.536360+0000', 'last_peered': '2024-09-15T21:28:28.536360+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:28:28.536360+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14348288, 'num_objects': 3503, 'num_object_clones': 0, 'num_object_copies': 7006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3503, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13996, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -636}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:31.752 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:28:31.752 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'17169", 'reported_seq': 17313, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:28.666334+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:28:28.666334+0000', 'last_peered': '2024-09-15T21:28:28.666334+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:28:28.666334+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15143878, 'num_objects': 3699, 'num_object_clones': 0, 'num_object_copies': 7398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3699, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13873, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -448}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:31.753 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:32.474 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:32.474 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39584/113804 objects degraded (34.783%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":564,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":56902,"data_bytes":233022217,"bytes_used":1884532736,"bytes_avail":288025759744,"bytes_total":289910292480,"degraded_objects":39584,"degraded_total":113804,"degraded_ratio":0.34782608695652173,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":315,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":477},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:32.493 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:32.493 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:33.059 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:33.059 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:33.089 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-15T21:28:32.835909+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":233022217,"num_objects":56902,"num_object_clones":0,"num_object_copies":113804,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56902,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":233557,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9891,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160865,"ondisk_log_size":160865,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1840364,"kb_used_data":654136,"kb_used_omap":12,"kb_used_meta":1186163,"kb_avail":281275156,"statfs":{"total":289910292480,"available":288025759744,"internally_reserved":0,"allocated":669835264,"data_stored":668946504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1214630915},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,8,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19247104,"num_objects":-4699,"num_object_clones":0,"num_object_copies":-9398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-214,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4699,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4699,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004733"},"pg_stats":[{"pgid":"2.f","version":"47'17480","reported_seq":17561,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.648873+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:28.648873+0000","last_peered":"2024-09-15T21:28:28.648873+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:28.648873+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15040512,"num_objects":3672,"num_object_clones":0,"num_object_copies":7344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17480,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-460}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17133","reported_seq":17242,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:29.292679+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:29.292679+0000","last_peered":"2024-09-15T21:28:29.292679+0000","last_clean":"2024-09-15T21:28:29.292679+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:29.292679+0000","last_undegraded":"2024-09-15T21:28:29.292679+0000","last_fullsized":"2024-09-15T21:28:29.292679+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14504902,"num_objects":3543,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15414,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17527","reported_seq":20942,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:29.316874+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:29.316874+0000","last_peered":"2024-09-15T21:28:29.316874+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:29.316874+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14655512,"num_objects":3579,"num_object_clones":0,"num_object_copies":7158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2627,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3579,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15627,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1641,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":952}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16838","reported_seq":16914,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:29.293096+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:29.293096+0000","last_peered":"2024-09-15T21:28:29.293096+0000","last_clean":"2024-09-15T21:28:29.293096+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:29.293096+0000","last_undegraded":"2024-09-15T21:28:29.293096+0000","last_fullsized":"2024-09-15T21:28:29.293096+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14237696,"num_objects":3476,"num_object_clones":0,"num_object_copies":6952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3476,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16838,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16992","reported_seq":17130,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:28.695182+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:28.695182+0000","last_peered":"2024-09-15T21:28:28.695182+0000","last_clean":"2024-09-15T21:28:28.695182+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:28.695182+0000","last_undegraded":"2024-09-15T21:28:28.695182+0000","last_fullsized":"2024-09-15T21:28:28.695182+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14389731,"num_objects":3514,"num_object_clones":0,"num_object_copies":7028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13447,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17025","reported_seq":17167,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.624143+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:28.624143+0000","last_peered":"2024-09-15T21:28:28.624143+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:28.624143+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14787043,"num_objects":3611,"num_object_clones":0,"num_object_copies":7222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13701,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-465},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17165","reported_seq":17301,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.662355+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:28.662355+0000","last_peered":"2024-09-15T21:28:28.662355+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:28.662355+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14357446,"num_objects":3507,"num_object_clones":0,"num_object_copies":7014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13476,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-654},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17331","reported_seq":17465,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:28.574920+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:28.574920+0000","last_peered":"2024-09-15T21:28:28.574920+0000","last_clean":"2024-09-15T21:28:28.574920+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:28.574920+0000","last_undegraded":"2024-09-15T21:28:28.574920+0000","last_fullsized":"2024-09-15T21:28:28.574920+0000","mapping_epoch":34,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14487552,"num_objects":3537,"num_object_clones":0,"num_object_copies":7074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14103,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16986","reported_seq":17118,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:29.298291+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:29.298291+0000","last_peered":"2024-09-15T21:28:29.298291+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:29.298291+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14447075,"num_objects":3528,"num_object_clones":0,"num_object_copies":7056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3528,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14918,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-559}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16865","reported_seq":17002,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.671855+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:28.671855+0000","last_peered":"2024-09-15T21:28:28.671855+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:28.671855+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14815232,"num_objects":3617,"num_object_clones":0,"num_object_copies":7234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13538,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-403}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17068","reported_seq":17192,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:29.273115+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:29.273115+0000","last_peered":"2024-09-15T21:28:29.273115+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:29.273115+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14643683,"num_objects":3576,"num_object_clones":0,"num_object_copies":7152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3576,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14927,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-514}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17104","reported_seq":25568,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:29.272437+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:29.272437+0000","last_peered":"2024-09-15T21:28:29.272437+0000","last_clean":"2024-09-15T21:28:29.272437+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:29.272437+0000","last_undegraded":"2024-09-15T21:28:29.272437+0000","last_fullsized":"2024-09-15T21:28:29.272437+0000","mapping_epoch":44,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14155776,"num_objects":3456,"num_object_clones":0,"num_object_copies":6912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14797,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17118","reported_seq":17257,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.647640+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:28.647640+0000","last_peered":"2024-09-15T21:28:28.647640+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:28.647640+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14614528,"num_objects":3568,"num_object_clones":0,"num_object_copies":7136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14062,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17087","reported_seq":17225,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.536360+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:28.536360+0000","last_peered":"2024-09-15T21:28:28.536360+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:28.536360+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14348288,"num_objects":3503,"num_object_clones":0,"num_object_copies":7006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13996,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-636}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16874","reported_seq":25189,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:29.904466+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:29.904466+0000","last_peered":"2024-09-15T21:28:29.904466+0000","last_clean":"2024-09-15T21:28:29.904466+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:29.904466+0000","last_undegraded":"2024-09-15T21:28:29.904466+0000","last_fullsized":"2024-09-15T21:28:29.904466+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14393344,"num_objects":3514,"num_object_clones":0,"num_object_copies":7028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13358,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17169","reported_seq":17313,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:28.666334+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:28.666334+0000","last_peered":"2024-09-15T21:28:28.666334+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:28.666334+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15143878,"num_objects":3699,"num_object_clones":0,"num_object_copies":7398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3699,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13873,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-448},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":233022198,"num_objects":56900,"num_object_clones":0,"num_object_copies":113800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56900,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":233555,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9890,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":667447296,"data_stored":667301858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583120,"kb_used_data":244608,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93788720,"statfs":{"total":96636764160,"available":96039649280,"internally_reserved":0,"allocated":250478592,"data_stored":250193334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667428,"kb_used_data":238736,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93704412,"statfs":{"total":96636764160,"available":95953317888,"internally_reserved":0,"allocated":244465664,"data_stored":244180401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589816,"kb_used_data":170792,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93782024,"statfs":{"total":96636764160,"available":96032792576,"internally_reserved":0,"allocated":174891008,"data_stored":174572769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":429059759},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":243511296,"data_stored":243475166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":174002176,"data_stored":173933070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":249933824,"data_stored":249893622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:36.091 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:36.682 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:36.685 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:36.706 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-15T21:28:34.836597+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":224154377,"num_objects":54737,"num_object_clones":0,"num_object_copies":109474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54737,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":235722,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10001,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160730,"ondisk_log_size":160730,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1837496,"kb_used_data":642948,"kb_used_omap":12,"kb_used_meta":1194483,"kb_avail":281278024,"statfs":{"total":289910292480,"available":288028696576,"internally_reserved":0,"allocated":658378752,"data_stored":657489992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1223150595},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,9,6,8,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28114944,"num_objects":-6864,"num_object_clones":0,"num_object_copies":-13728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-324,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6864,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":324,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004602"},"pg_stats":[{"pgid":"2.f","version":"47'17647","reported_seq":17728,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.680705+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:33.680705+0000","last_peered":"2024-09-15T21:28:33.680705+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:33.680705+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14356480,"num_objects":3505,"num_object_clones":0,"num_object_copies":7010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17647,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-627}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17264","reported_seq":17375,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:34.212663+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:34.212663+0000","last_peered":"2024-09-15T21:28:34.212663+0000","last_clean":"2024-09-15T21:28:34.212663+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:34.212663+0000","last_undegraded":"2024-09-15T21:28:34.212663+0000","last_fullsized":"2024-09-15T21:28:34.212663+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13968326,"num_objects":3412,"num_object_clones":0,"num_object_copies":6824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3412,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15545,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17674","reported_seq":21309,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:34.323133+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:34.323133+0000","last_peered":"2024-09-15T21:28:34.323133+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:34.323133+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14053400,"num_objects":3432,"num_object_clones":0,"num_object_copies":6864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3432,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15774,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1751,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":915}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16970","reported_seq":17046,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:34.316737+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:34.316737+0000","last_peered":"2024-09-15T21:28:34.316737+0000","last_clean":"2024-09-15T21:28:34.316737+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:34.316737+0000","last_undegraded":"2024-09-15T21:28:34.316737+0000","last_fullsized":"2024-09-15T21:28:34.316737+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13697024,"num_objects":3344,"num_object_clones":0,"num_object_copies":6688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16970,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17127","reported_seq":17265,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:33.621940+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:33.621940+0000","last_peered":"2024-09-15T21:28:33.621940+0000","last_clean":"2024-09-15T21:28:33.621940+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:33.621940+0000","last_undegraded":"2024-09-15T21:28:33.621940+0000","last_fullsized":"2024-09-15T21:28:33.621940+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13836771,"num_objects":3379,"num_object_clones":0,"num_object_copies":6758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13582,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17184","reported_seq":17326,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.541299+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:33.541299+0000","last_peered":"2024-09-15T21:28:33.541299+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:33.541299+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14135779,"num_objects":3452,"num_object_clones":0,"num_object_copies":6904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13860,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-624},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17323","reported_seq":17460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.646815+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:33.646815+0000","last_peered":"2024-09-15T21:28:33.646815+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:33.646815+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13710278,"num_objects":3349,"num_object_clones":0,"num_object_copies":6698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13634,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-812},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17460","reported_seq":17594,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:33.644757+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:33.644757+0000","last_peered":"2024-09-15T21:28:33.644757+0000","last_clean":"2024-09-15T21:28:33.644757+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:33.644757+0000","last_undegraded":"2024-09-15T21:28:33.644757+0000","last_fullsized":"2024-09-15T21:28:33.644757+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13959168,"num_objects":3408,"num_object_clones":0,"num_object_copies":6816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14232,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17121","reported_seq":17253,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:34.259553+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:34.259553+0000","last_peered":"2024-09-15T21:28:34.259553+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:34.259553+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13894115,"num_objects":3393,"num_object_clones":0,"num_object_copies":6786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15053,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-694}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17009","reported_seq":17146,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.669923+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:33.669923+0000","last_peered":"2024-09-15T21:28:33.669923+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:33.669923+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14225408,"num_objects":3473,"num_object_clones":0,"num_object_copies":6946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13682,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-547}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17224","reported_seq":17348,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:34.329619+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:34.329619+0000","last_peered":"2024-09-15T21:28:34.329619+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:34.329619+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14004707,"num_objects":3420,"num_object_clones":0,"num_object_copies":6840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3420,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15083,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-670}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17256","reported_seq":25720,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:34.304358+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:34.304358+0000","last_peered":"2024-09-15T21:28:34.304358+0000","last_clean":"2024-09-15T21:28:34.304358+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:34.304358+0000","last_undegraded":"2024-09-15T21:28:34.304358+0000","last_fullsized":"2024-09-15T21:28:34.304358+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13533184,"num_objects":3304,"num_object_clones":0,"num_object_copies":6608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14949,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17257","reported_seq":17396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.722059+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:33.722059+0000","last_peered":"2024-09-15T21:28:33.722059+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:33.722059+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14045184,"num_objects":3429,"num_object_clones":0,"num_object_copies":6858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14201,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-665}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17220","reported_seq":17357,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.679847+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:33.679847+0000","last_peered":"2024-09-15T21:28:33.679847+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:33.679847+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13803520,"num_objects":3370,"num_object_clones":0,"num_object_copies":6740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14129,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16874","reported_seq":25189,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:29.904466+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:29.904466+0000","last_peered":"2024-09-15T21:28:29.904466+0000","last_clean":"2024-09-15T21:28:29.904466+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:29.904466+0000","last_undegraded":"2024-09-15T21:28:29.904466+0000","last_fullsized":"2024-09-15T21:28:29.904466+0000","mapping_epoch":46,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14393344,"num_objects":3514,"num_object_clones":0,"num_object_copies":7028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13358,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17317","reported_seq":17461,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.723821+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:33.723821+0000","last_peered":"2024-09-15T21:28:33.723821+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:33.723821+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14537670,"num_objects":3551,"num_object_clones":0,"num_object_copies":7102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3551,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14021,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-596},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":224154358,"num_objects":54735,"num_object_clones":0,"num_object_copies":109470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54735,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":235720,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10000,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":655753216,"data_stored":655607778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160727,"ondisk_log_size":160727,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582856,"kb_used_data":240180,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93788984,"statfs":{"total":96636764160,"available":96039919616,"internally_reserved":0,"allocated":245944320,"data_stored":245659062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664824,"kb_used_data":231976,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93707016,"statfs":{"total":96636764160,"available":95955984384,"internally_reserved":0,"allocated":237543424,"data_stored":237258161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,1,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589816,"kb_used_data":170792,"kb_used_omap":4,"kb_used_meta":419003,"kb_avail":93782024,"statfs":{"total":96636764160,"available":96032792576,"internally_reserved":0,"allocated":174891008,"data_stored":174572769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":429059759},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,2,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":236396544,"data_stored":236360414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":174002176,"data_stored":173933070,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":245354496,"data_stored":245314294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:36.707 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:37.280 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:37.282 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:37.300 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-15T21:28:36.837309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":223630089,"num_objects":54609,"num_object_clones":0,"num_object_copies":109218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54609,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":235850,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10001,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160658,"ondisk_log_size":160658,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1817900,"kb_used_data":635828,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281297620,"statfs":{"total":289910292480,"available":288048762880,"internally_reserved":0,"allocated":651087872,"data_stored":650199112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1210371075},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,3,9,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19537920,"num_objects":-4770,"num_object_clones":0,"num_object_copies":-9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4770,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004542"},"pg_stats":[{"pgid":"2.f","version":"47'17647","reported_seq":17728,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.680705+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:33.680705+0000","last_peered":"2024-09-15T21:28:33.680705+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:33.680705+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14356480,"num_objects":3505,"num_object_clones":0,"num_object_copies":7010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17647,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-627}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17264","reported_seq":17375,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:34.212663+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:34.212663+0000","last_peered":"2024-09-15T21:28:34.212663+0000","last_clean":"2024-09-15T21:28:34.212663+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:34.212663+0000","last_undegraded":"2024-09-15T21:28:34.212663+0000","last_fullsized":"2024-09-15T21:28:34.212663+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13968326,"num_objects":3412,"num_object_clones":0,"num_object_copies":6824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3412,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15545,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17674","reported_seq":21309,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:34.323133+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:34.323133+0000","last_peered":"2024-09-15T21:28:34.323133+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:34.323133+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14053400,"num_objects":3432,"num_object_clones":0,"num_object_copies":6864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3432,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15774,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1751,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":915}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16970","reported_seq":17046,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:34.316737+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:34.316737+0000","last_peered":"2024-09-15T21:28:34.316737+0000","last_clean":"2024-09-15T21:28:34.316737+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:34.316737+0000","last_undegraded":"2024-09-15T21:28:34.316737+0000","last_fullsized":"2024-09-15T21:28:34.316737+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13697024,"num_objects":3344,"num_object_clones":0,"num_object_copies":6688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16970,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17127","reported_seq":17265,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:33.621940+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:33.621940+0000","last_peered":"2024-09-15T21:28:33.621940+0000","last_clean":"2024-09-15T21:28:33.621940+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:33.621940+0000","last_undegraded":"2024-09-15T21:28:33.621940+0000","last_fullsized":"2024-09-15T21:28:33.621940+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13836771,"num_objects":3379,"num_object_clones":0,"num_object_copies":6758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13582,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17184","reported_seq":17326,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.541299+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:33.541299+0000","last_peered":"2024-09-15T21:28:33.541299+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:33.541299+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14135779,"num_objects":3452,"num_object_clones":0,"num_object_copies":6904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13860,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-624},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17323","reported_seq":17460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.646815+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:33.646815+0000","last_peered":"2024-09-15T21:28:33.646815+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:33.646815+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13710278,"num_objects":3349,"num_object_clones":0,"num_object_copies":6698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13634,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-812},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17460","reported_seq":17594,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:33.644757+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:33.644757+0000","last_peered":"2024-09-15T21:28:33.644757+0000","last_clean":"2024-09-15T21:28:33.644757+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:33.644757+0000","last_undegraded":"2024-09-15T21:28:33.644757+0000","last_fullsized":"2024-09-15T21:28:33.644757+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13959168,"num_objects":3408,"num_object_clones":0,"num_object_copies":6816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14232,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17121","reported_seq":17253,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:34.259553+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:34.259553+0000","last_peered":"2024-09-15T21:28:34.259553+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:34.259553+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13894115,"num_objects":3393,"num_object_clones":0,"num_object_copies":6786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15053,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-694}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17009","reported_seq":17146,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.669923+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:33.669923+0000","last_peered":"2024-09-15T21:28:33.669923+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:33.669923+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14225408,"num_objects":3473,"num_object_clones":0,"num_object_copies":6946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13682,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-547}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17224","reported_seq":17348,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:34.329619+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:34.329619+0000","last_peered":"2024-09-15T21:28:34.329619+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:34.329619+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14004707,"num_objects":3420,"num_object_clones":0,"num_object_copies":6840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3420,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15083,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-670}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17256","reported_seq":25720,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:34.304358+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:34.304358+0000","last_peered":"2024-09-15T21:28:34.304358+0000","last_clean":"2024-09-15T21:28:34.304358+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:34.304358+0000","last_undegraded":"2024-09-15T21:28:34.304358+0000","last_fullsized":"2024-09-15T21:28:34.304358+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13533184,"num_objects":3304,"num_object_clones":0,"num_object_copies":6608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14949,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17257","reported_seq":17396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.722059+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:33.722059+0000","last_peered":"2024-09-15T21:28:33.722059+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:33.722059+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14045184,"num_objects":3429,"num_object_clones":0,"num_object_copies":6858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14201,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-665}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17220","reported_seq":17357,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.679847+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:33.679847+0000","last_peered":"2024-09-15T21:28:33.679847+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:33.679847+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13803520,"num_objects":3370,"num_object_clones":0,"num_object_copies":6740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14129,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17002","reported_seq":25316,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:34.973553+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:34.973553+0000","last_peered":"2024-09-15T21:28:34.973553+0000","last_clean":"2024-09-15T21:28:34.973553+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:34.973553+0000","last_undegraded":"2024-09-15T21:28:34.973553+0000","last_fullsized":"2024-09-15T21:28:34.973553+0000","mapping_epoch":46,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13869056,"num_objects":3386,"num_object_clones":0,"num_object_copies":6772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13486,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17317","reported_seq":17461,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.723821+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:33.723821+0000","last_peered":"2024-09-15T21:28:33.723821+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:33.723821+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14537670,"num_objects":3551,"num_object_clones":0,"num_object_copies":7102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3551,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14021,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-596},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":223630070,"num_objects":54607,"num_object_clones":0,"num_object_copies":109214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54607,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":235848,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10000,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":648536064,"data_stored":648390626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160655,"ondisk_log_size":160655,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582856,"kb_used_data":240180,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93788984,"statfs":{"total":96636764160,"available":96039919616,"internally_reserved":0,"allocated":245944320,"data_stored":245659062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664824,"kb_used_data":231976,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93707016,"statfs":{"total":96636764160,"available":95955984384,"internally_reserved":0,"allocated":237543424,"data_stored":237258161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,1,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570220,"kb_used_data":163672,"kb_used_omap":4,"kb_used_meta":406523,"kb_avail":93801620,"statfs":{"total":96636764160,"available":96052858880,"internally_reserved":0,"allocated":167600128,"data_stored":167281889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":416280239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":236396544,"data_stored":236360414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":166785024,"data_stored":166715918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":245354496,"data_stored":245314294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:37.301 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:28:37.302 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17647", 'reported_seq': 17728, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:33.680705+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:28:33.680705+0000', 'last_peered': '2024-09-15T21:28:33.680705+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:28:33.680705+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14356480, 'num_objects': 3505, 'num_object_clones': 0, 'num_object_copies': 7010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3505, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17647, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -627}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:37.302 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:28:37.302 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'17674", 'reported_seq': 21309, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:34.323133+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:28:34.323133+0000', 'last_peered': '2024-09-15T21:28:34.323133+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:28:34.323133+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14053400, 'num_objects': 3432, 'num_object_clones': 0, 'num_object_copies': 6864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2517, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3432, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15774, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1751, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 915}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:37.302 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:37.302 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17184", 'reported_seq': 17326, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:33.541299+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:28:33.541299+0000', 'last_peered': '2024-09-15T21:28:33.541299+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:28:33.541299+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14135779, 'num_objects': 3452, 'num_object_clones': 0, 'num_object_copies': 6904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3452, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13860, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -624}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:37.303 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:37.303 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17323", 'reported_seq': 17460, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:33.646815+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:28:33.646815+0000', 'last_peered': '2024-09-15T21:28:33.646815+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:28:33.646815+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13710278, 'num_objects': 3349, 'num_object_clones': 0, 'num_object_copies': 6698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3349, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13634, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -812}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:37.303 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:28:37.303 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'17121", 'reported_seq': 17253, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:34.259553+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:28:34.259553+0000', 'last_peered': '2024-09-15T21:28:34.259553+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:28:34.259553+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13894115, 'num_objects': 3393, 'num_object_clones': 0, 'num_object_copies': 6786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3393, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15053, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -694}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:37.303 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:37.303 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17009", 'reported_seq': 17146, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:33.669923+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:28:33.669923+0000', 'last_peered': '2024-09-15T21:28:33.669923+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:28:33.669923+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14225408, 'num_objects': 3473, 'num_object_clones': 0, 'num_object_copies': 6946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13682, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -547}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:37.303 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:28:37.303 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17224", 'reported_seq': 17348, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:34.329619+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:28:34.329619+0000', 'last_peered': '2024-09-15T21:28:34.329619+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:28:34.329619+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14004707, 'num_objects': 3420, 'num_object_clones': 0, 'num_object_copies': 6840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3420, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15083, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -670}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:37.303 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:37.303 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17257", 'reported_seq': 17396, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:33.722059+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:28:33.722059+0000', 'last_peered': '2024-09-15T21:28:33.722059+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:28:33.722059+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14045184, 'num_objects': 3429, 'num_object_clones': 0, 'num_object_copies': 6858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3429, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14201, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -665}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:37.304 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:28:37.304 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'17220", 'reported_seq': 17357, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:33.679847+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:28:33.679847+0000', 'last_peered': '2024-09-15T21:28:33.679847+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:28:33.679847+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13803520, 'num_objects': 3370, 'num_object_clones': 0, 'num_object_copies': 6740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3370, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14129, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -769}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:37.304 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:28:37.304 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'17317", 'reported_seq': 17461, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:33.723821+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:28:33.723821+0000', 'last_peered': '2024-09-15T21:28:33.723821+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:28:33.723821+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14537670, 'num_objects': 3551, 'num_object_clones': 0, 'num_object_copies': 7102, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3551, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14021, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -596}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:37.304 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:37.995 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39474/109218 objects degraded (36.142%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":569,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":54609,"data_bytes":223630089,"bytes_used":1861529600,"bytes_avail":288048762880,"bytes_total":289910292480,"degraded_objects":39474,"degraded_total":109218,"degraded_ratio":0.36142394110860848,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":397},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:38.021 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:38.021 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:38.613 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:38.615 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:38.637 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-15T21:28:36.837309+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":223630089,"num_objects":54609,"num_object_clones":0,"num_object_copies":109218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54609,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":235850,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10001,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160658,"ondisk_log_size":160658,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1817900,"kb_used_data":635828,"kb_used_omap":12,"kb_used_meta":1182003,"kb_avail":281297620,"statfs":{"total":289910292480,"available":288048762880,"internally_reserved":0,"allocated":651087872,"data_stored":650199112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1210371075},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,8,3,9,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19537920,"num_objects":-4770,"num_object_clones":0,"num_object_copies":-9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-218,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4770,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":218,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004542"},"pg_stats":[{"pgid":"2.f","version":"47'17647","reported_seq":17728,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.680705+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:33.680705+0000","last_peered":"2024-09-15T21:28:33.680705+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:33.680705+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14356480,"num_objects":3505,"num_object_clones":0,"num_object_copies":7010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17647,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-627}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17264","reported_seq":17375,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:34.212663+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:34.212663+0000","last_peered":"2024-09-15T21:28:34.212663+0000","last_clean":"2024-09-15T21:28:34.212663+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:34.212663+0000","last_undegraded":"2024-09-15T21:28:34.212663+0000","last_fullsized":"2024-09-15T21:28:34.212663+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13968326,"num_objects":3412,"num_object_clones":0,"num_object_copies":6824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3412,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15545,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17674","reported_seq":21309,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:34.323133+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:34.323133+0000","last_peered":"2024-09-15T21:28:34.323133+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:34.323133+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14053400,"num_objects":3432,"num_object_clones":0,"num_object_copies":6864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2517,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3432,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15774,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1751,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":915}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16970","reported_seq":17046,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:34.316737+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:34.316737+0000","last_peered":"2024-09-15T21:28:34.316737+0000","last_clean":"2024-09-15T21:28:34.316737+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:34.316737+0000","last_undegraded":"2024-09-15T21:28:34.316737+0000","last_fullsized":"2024-09-15T21:28:34.316737+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13697024,"num_objects":3344,"num_object_clones":0,"num_object_copies":6688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16970,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17127","reported_seq":17265,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:33.621940+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:33.621940+0000","last_peered":"2024-09-15T21:28:33.621940+0000","last_clean":"2024-09-15T21:28:33.621940+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:33.621940+0000","last_undegraded":"2024-09-15T21:28:33.621940+0000","last_fullsized":"2024-09-15T21:28:33.621940+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13836771,"num_objects":3379,"num_object_clones":0,"num_object_copies":6758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13582,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17184","reported_seq":17326,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.541299+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:33.541299+0000","last_peered":"2024-09-15T21:28:33.541299+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:33.541299+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14135779,"num_objects":3452,"num_object_clones":0,"num_object_copies":6904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3452,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13860,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-624},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17323","reported_seq":17460,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.646815+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:33.646815+0000","last_peered":"2024-09-15T21:28:33.646815+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:33.646815+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13710278,"num_objects":3349,"num_object_clones":0,"num_object_copies":6698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13634,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-812},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17460","reported_seq":17594,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:33.644757+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:33.644757+0000","last_peered":"2024-09-15T21:28:33.644757+0000","last_clean":"2024-09-15T21:28:33.644757+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:33.644757+0000","last_undegraded":"2024-09-15T21:28:33.644757+0000","last_fullsized":"2024-09-15T21:28:33.644757+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13959168,"num_objects":3408,"num_object_clones":0,"num_object_copies":6816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14232,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17121","reported_seq":17253,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:34.259553+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:34.259553+0000","last_peered":"2024-09-15T21:28:34.259553+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:34.259553+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13894115,"num_objects":3393,"num_object_clones":0,"num_object_copies":6786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15053,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-694}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367862+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367862+0000","last_peered":"2024-09-15T21:27:09.367862+0000","last_clean":"2024-09-15T21:27:09.367862+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367862+0000","last_undegraded":"2024-09-15T21:27:09.367862+0000","last_fullsized":"2024-09-15T21:27:09.367862+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17009","reported_seq":17146,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.669923+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:33.669923+0000","last_peered":"2024-09-15T21:28:33.669923+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:33.669923+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14225408,"num_objects":3473,"num_object_clones":0,"num_object_copies":6946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13682,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-547}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17224","reported_seq":17348,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:34.329619+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:34.329619+0000","last_peered":"2024-09-15T21:28:34.329619+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:34.329619+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14004707,"num_objects":3420,"num_object_clones":0,"num_object_copies":6840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3420,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15083,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-670}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17256","reported_seq":25720,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:34.304358+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:34.304358+0000","last_peered":"2024-09-15T21:28:34.304358+0000","last_clean":"2024-09-15T21:28:34.304358+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:34.304358+0000","last_undegraded":"2024-09-15T21:28:34.304358+0000","last_fullsized":"2024-09-15T21:28:34.304358+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13533184,"num_objects":3304,"num_object_clones":0,"num_object_copies":6608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14949,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17257","reported_seq":17396,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.722059+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:33.722059+0000","last_peered":"2024-09-15T21:28:33.722059+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:33.722059+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14045184,"num_objects":3429,"num_object_clones":0,"num_object_copies":6858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3429,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14201,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-665}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17220","reported_seq":17357,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.679847+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:33.679847+0000","last_peered":"2024-09-15T21:28:33.679847+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:33.679847+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13803520,"num_objects":3370,"num_object_clones":0,"num_object_copies":6740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14129,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-769}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370029+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370029+0000","last_peered":"2024-09-15T21:27:09.370029+0000","last_clean":"2024-09-15T21:27:09.370029+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370029+0000","last_undegraded":"2024-09-15T21:27:09.370029+0000","last_fullsized":"2024-09-15T21:27:09.370029+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17002","reported_seq":25316,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:34.973553+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:34.973553+0000","last_peered":"2024-09-15T21:28:34.973553+0000","last_clean":"2024-09-15T21:28:34.973553+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:34.973553+0000","last_undegraded":"2024-09-15T21:28:34.973553+0000","last_fullsized":"2024-09-15T21:28:34.973553+0000","mapping_epoch":46,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13869056,"num_objects":3386,"num_object_clones":0,"num_object_copies":6772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13486,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17317","reported_seq":17461,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:33.723821+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:33.723821+0000","last_peered":"2024-09-15T21:28:33.723821+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:33.723821+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14537670,"num_objects":3551,"num_object_clones":0,"num_object_copies":7102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3551,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14021,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-596},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":223630070,"num_objects":54607,"num_object_clones":0,"num_object_copies":109214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39474,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54607,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":235848,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10000,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":648536064,"data_stored":648390626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160655,"ondisk_log_size":160655,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582856,"kb_used_data":240180,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93788984,"statfs":{"total":96636764160,"available":96039919616,"internally_reserved":0,"allocated":245944320,"data_stored":245659062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664824,"kb_used_data":231976,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93707016,"statfs":{"total":96636764160,"available":95955984384,"internally_reserved":0,"allocated":237543424,"data_stored":237258161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,1,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":570220,"kb_used_data":163672,"kb_used_omap":4,"kb_used_meta":406523,"kb_avail":93801620,"statfs":{"total":96636764160,"available":96052858880,"internally_reserved":0,"allocated":167600128,"data_stored":167281889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":416280239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":236396544,"data_stored":236360414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":166785024,"data_stored":166715918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":245354496,"data_stored":245314294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:41.640 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:42.189 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:42.189 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:42.224 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-15T21:28:40.839826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":214008585,"num_objects":52260,"num_object_clones":0,"num_object_copies":104520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39363,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52260,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":238199,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10112,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1815932,"kb_used_data":617224,"kb_used_omap":12,"kb_used_meta":1198643,"kb_avail":281299588,"statfs":{"total":289910292480,"available":288050778112,"internally_reserved":0,"allocated":632037376,"data_stored":631148616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1227410435},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,5,9,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23191552,"num_objects":-5662,"num_object_clones":0,"num_object_copies":-11324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5662,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005407"},"pg_stats":[{"pgid":"2.f","version":"47'17798","reported_seq":17879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.418577+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:38.418577+0000","last_peered":"2024-09-15T21:28:38.418577+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:38.418577+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13737984,"num_objects":3354,"num_object_clones":0,"num_object_copies":6708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17798,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-778}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17420","reported_seq":17530,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:39.352943+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:39.352943+0000","last_peered":"2024-09-15T21:28:39.352943+0000","last_clean":"2024-09-15T21:28:39.352943+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:39.352943+0000","last_undegraded":"2024-09-15T21:28:39.352943+0000","last_fullsized":"2024-09-15T21:28:39.352943+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13329350,"num_objects":3256,"num_object_clones":0,"num_object_copies":6512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3256,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15701,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17814","reported_seq":21671,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:39.360148+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:39.360148+0000","last_peered":"2024-09-15T21:28:39.360148+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:39.360148+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13479960,"num_objects":3292,"num_object_clones":0,"num_object_copies":6584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2406,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3292,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15914,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1862,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":886}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17113","reported_seq":17189,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:39.327331+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:39.327331+0000","last_peered":"2024-09-15T21:28:39.327331+0000","last_clean":"2024-09-15T21:28:39.327331+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:39.327331+0000","last_undegraded":"2024-09-15T21:28:39.327331+0000","last_fullsized":"2024-09-15T21:28:39.327331+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13111296,"num_objects":3201,"num_object_clones":0,"num_object_copies":6402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17113,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17266","reported_seq":17404,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:38.756020+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:38.756020+0000","last_peered":"2024-09-15T21:28:38.756020+0000","last_clean":"2024-09-15T21:28:38.756020+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:38.756020+0000","last_undegraded":"2024-09-15T21:28:38.756020+0000","last_fullsized":"2024-09-15T21:28:38.756020+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13267427,"num_objects":3240,"num_object_clones":0,"num_object_copies":6480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13721,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17327","reported_seq":17469,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.632873+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:38.632873+0000","last_peered":"2024-09-15T21:28:38.632873+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:38.632873+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13550051,"num_objects":3309,"num_object_clones":0,"num_object_copies":6618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14003,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-767},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17451","reported_seq":17593,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.665379+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:38.665379+0000","last_peered":"2024-09-15T21:28:38.665379+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:38.665379+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13185990,"num_objects":3221,"num_object_clones":0,"num_object_copies":6442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13762,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-940},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17631","reported_seq":17765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:38.718400+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:38.718400+0000","last_peered":"2024-09-15T21:28:38.718400+0000","last_clean":"2024-09-15T21:28:38.718400+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:38.718400+0000","last_undegraded":"2024-09-15T21:28:38.718400+0000","last_fullsized":"2024-09-15T21:28:38.718400+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13258752,"num_objects":3237,"num_object_clones":0,"num_object_copies":6474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14403,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17292","reported_seq":17424,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:39.186121+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:39.186121+0000","last_peered":"2024-09-15T21:28:39.186121+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:39.186121+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13193699,"num_objects":3222,"num_object_clones":0,"num_object_copies":6444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3222,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15224,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-865}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17160","reported_seq":17297,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.728972+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:38.728972+0000","last_peered":"2024-09-15T21:28:38.728972+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:38.728972+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13606912,"num_objects":3322,"num_object_clones":0,"num_object_copies":6644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13833,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-698}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17388","reported_seq":17512,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:39.339748+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:39.339748+0000","last_peered":"2024-09-15T21:28:39.339748+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:39.339748+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13332963,"num_objects":3256,"num_object_clones":0,"num_object_copies":6512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3256,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15247,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-834}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17405","reported_seq":25869,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:39.339088+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:39.339088+0000","last_peered":"2024-09-15T21:28:39.339088+0000","last_clean":"2024-09-15T21:28:39.339088+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:39.339088+0000","last_undegraded":"2024-09-15T21:28:39.339088+0000","last_fullsized":"2024-09-15T21:28:39.339088+0000","mapping_epoch":44,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12922880,"num_objects":3155,"num_object_clones":0,"num_object_copies":6310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15098,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17397","reported_seq":17536,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.725990+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:38.725990+0000","last_peered":"2024-09-15T21:28:38.725990+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:38.725990+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13471744,"num_objects":3289,"num_object_clones":0,"num_object_copies":6578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14341,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17357","reported_seq":17491,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.778691+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:38.778691+0000","last_peered":"2024-09-15T21:28:38.778691+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:38.778691+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13242368,"num_objects":3233,"num_object_clones":0,"num_object_copies":6466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14266,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17131","reported_seq":25446,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:39.942214+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:39.942214+0000","last_peered":"2024-09-15T21:28:39.942214+0000","last_clean":"2024-09-15T21:28:39.942214+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:39.942214+0000","last_undegraded":"2024-09-15T21:28:39.942214+0000","last_fullsized":"2024-09-15T21:28:39.942214+0000","mapping_epoch":46,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13340672,"num_objects":3257,"num_object_clones":0,"num_object_copies":6514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13615,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17454","reported_seq":17599,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.720777+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:38.720777+0000","last_peered":"2024-09-15T21:28:38.720777+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:38.720777+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13976518,"num_objects":3414,"num_object_clones":0,"num_object_copies":6828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3414,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14158,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-733},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":214008566,"num_objects":52258,"num_object_clones":0,"num_object_copies":104516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39363,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52258,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":238197,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10111,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":629334016,"data_stored":629188578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582140,"kb_used_data":235308,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93789700,"statfs":{"total":96636764160,"available":96040652800,"internally_reserved":0,"allocated":240955392,"data_stored":240670134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662096,"kb_used_data":225084,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93709744,"statfs":{"total":96636764160,"available":95958777856,"internally_reserved":0,"allocated":230486016,"data_stored":230200753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571696,"kb_used_data":156832,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93800144,"statfs":{"total":96636764160,"available":96051347456,"internally_reserved":0,"allocated":160595968,"data_stored":160277729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":424799919},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":229384192,"data_stored":229348062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":159653888,"data_stored":159584782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":240295936,"data_stored":240255734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:42.226 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:42.790 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:42.793 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:42.818 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-15T21:28:40.839826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":214008585,"num_objects":52260,"num_object_clones":0,"num_object_copies":104520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39363,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52260,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":238199,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10112,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1815932,"kb_used_data":617224,"kb_used_omap":12,"kb_used_meta":1198643,"kb_avail":281299588,"statfs":{"total":289910292480,"available":288050778112,"internally_reserved":0,"allocated":632037376,"data_stored":631148616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1227410435},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,5,9,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23191552,"num_objects":-5662,"num_object_clones":0,"num_object_copies":-11324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5662,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005407"},"pg_stats":[{"pgid":"2.f","version":"47'17798","reported_seq":17879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.418577+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:38.418577+0000","last_peered":"2024-09-15T21:28:38.418577+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:38.418577+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13737984,"num_objects":3354,"num_object_clones":0,"num_object_copies":6708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17798,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-778}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17420","reported_seq":17530,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:39.352943+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:39.352943+0000","last_peered":"2024-09-15T21:28:39.352943+0000","last_clean":"2024-09-15T21:28:39.352943+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:39.352943+0000","last_undegraded":"2024-09-15T21:28:39.352943+0000","last_fullsized":"2024-09-15T21:28:39.352943+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13329350,"num_objects":3256,"num_object_clones":0,"num_object_copies":6512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3256,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15701,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17814","reported_seq":21671,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:39.360148+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:39.360148+0000","last_peered":"2024-09-15T21:28:39.360148+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:39.360148+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13479960,"num_objects":3292,"num_object_clones":0,"num_object_copies":6584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2406,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3292,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15914,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1862,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":886}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17113","reported_seq":17189,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:39.327331+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:39.327331+0000","last_peered":"2024-09-15T21:28:39.327331+0000","last_clean":"2024-09-15T21:28:39.327331+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:39.327331+0000","last_undegraded":"2024-09-15T21:28:39.327331+0000","last_fullsized":"2024-09-15T21:28:39.327331+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13111296,"num_objects":3201,"num_object_clones":0,"num_object_copies":6402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17113,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17266","reported_seq":17404,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:38.756020+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:38.756020+0000","last_peered":"2024-09-15T21:28:38.756020+0000","last_clean":"2024-09-15T21:28:38.756020+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:38.756020+0000","last_undegraded":"2024-09-15T21:28:38.756020+0000","last_fullsized":"2024-09-15T21:28:38.756020+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13267427,"num_objects":3240,"num_object_clones":0,"num_object_copies":6480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13721,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17327","reported_seq":17469,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.632873+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:38.632873+0000","last_peered":"2024-09-15T21:28:38.632873+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:38.632873+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13550051,"num_objects":3309,"num_object_clones":0,"num_object_copies":6618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14003,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-767},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17451","reported_seq":17593,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.665379+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:38.665379+0000","last_peered":"2024-09-15T21:28:38.665379+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:38.665379+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13185990,"num_objects":3221,"num_object_clones":0,"num_object_copies":6442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13762,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-940},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17631","reported_seq":17765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:38.718400+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:38.718400+0000","last_peered":"2024-09-15T21:28:38.718400+0000","last_clean":"2024-09-15T21:28:38.718400+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:38.718400+0000","last_undegraded":"2024-09-15T21:28:38.718400+0000","last_fullsized":"2024-09-15T21:28:38.718400+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13258752,"num_objects":3237,"num_object_clones":0,"num_object_copies":6474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14403,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17292","reported_seq":17424,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:39.186121+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:39.186121+0000","last_peered":"2024-09-15T21:28:39.186121+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:39.186121+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13193699,"num_objects":3222,"num_object_clones":0,"num_object_copies":6444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3222,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15224,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-865}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17160","reported_seq":17297,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.728972+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:38.728972+0000","last_peered":"2024-09-15T21:28:38.728972+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:38.728972+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13606912,"num_objects":3322,"num_object_clones":0,"num_object_copies":6644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13833,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-698}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17388","reported_seq":17512,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:39.339748+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:39.339748+0000","last_peered":"2024-09-15T21:28:39.339748+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:39.339748+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13332963,"num_objects":3256,"num_object_clones":0,"num_object_copies":6512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3256,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15247,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-834}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17405","reported_seq":25869,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:39.339088+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:39.339088+0000","last_peered":"2024-09-15T21:28:39.339088+0000","last_clean":"2024-09-15T21:28:39.339088+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:39.339088+0000","last_undegraded":"2024-09-15T21:28:39.339088+0000","last_fullsized":"2024-09-15T21:28:39.339088+0000","mapping_epoch":44,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12922880,"num_objects":3155,"num_object_clones":0,"num_object_copies":6310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15098,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17397","reported_seq":17536,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.725990+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:38.725990+0000","last_peered":"2024-09-15T21:28:38.725990+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:38.725990+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13471744,"num_objects":3289,"num_object_clones":0,"num_object_copies":6578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14341,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17357","reported_seq":17491,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.778691+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:38.778691+0000","last_peered":"2024-09-15T21:28:38.778691+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:38.778691+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13242368,"num_objects":3233,"num_object_clones":0,"num_object_copies":6466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14266,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17131","reported_seq":25446,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:39.942214+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:39.942214+0000","last_peered":"2024-09-15T21:28:39.942214+0000","last_clean":"2024-09-15T21:28:39.942214+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:39.942214+0000","last_undegraded":"2024-09-15T21:28:39.942214+0000","last_fullsized":"2024-09-15T21:28:39.942214+0000","mapping_epoch":46,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13340672,"num_objects":3257,"num_object_clones":0,"num_object_copies":6514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13615,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17454","reported_seq":17599,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.720777+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:38.720777+0000","last_peered":"2024-09-15T21:28:38.720777+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:38.720777+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13976518,"num_objects":3414,"num_object_clones":0,"num_object_copies":6828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3414,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14158,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-733},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":214008566,"num_objects":52258,"num_object_clones":0,"num_object_copies":104516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39363,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52258,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":238197,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10111,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":629334016,"data_stored":629188578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582140,"kb_used_data":235308,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93789700,"statfs":{"total":96636764160,"available":96040652800,"internally_reserved":0,"allocated":240955392,"data_stored":240670134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662096,"kb_used_data":225084,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93709744,"statfs":{"total":96636764160,"available":95958777856,"internally_reserved":0,"allocated":230486016,"data_stored":230200753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571696,"kb_used_data":156832,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93800144,"statfs":{"total":96636764160,"available":96051347456,"internally_reserved":0,"allocated":160595968,"data_stored":160277729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":424799919},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":229384192,"data_stored":229348062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":159653888,"data_stored":159584782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":240295936,"data_stored":240255734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:42.819 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:28:42.820 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17798", 'reported_seq': 17879, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:38.418577+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:28:38.418577+0000', 'last_peered': '2024-09-15T21:28:38.418577+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:28:38.418577+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13737984, 'num_objects': 3354, 'num_object_clones': 0, 'num_object_copies': 6708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3354, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17798, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -778}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:42.820 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:28:42.820 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'17814", 'reported_seq': 21671, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:39.360148+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:28:39.360148+0000', 'last_peered': '2024-09-15T21:28:39.360148+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:28:39.360148+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13479960, 'num_objects': 3292, 'num_object_clones': 0, 'num_object_copies': 6584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2406, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3292, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15914, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1862, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 886}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:42.820 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:42.820 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17327", 'reported_seq': 17469, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:38.632873+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:28:38.632873+0000', 'last_peered': '2024-09-15T21:28:38.632873+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:28:38.632873+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13550051, 'num_objects': 3309, 'num_object_clones': 0, 'num_object_copies': 6618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3309, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14003, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -767}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:42.820 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:42.820 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17451", 'reported_seq': 17593, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:38.665379+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:28:38.665379+0000', 'last_peered': '2024-09-15T21:28:38.665379+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:28:38.665379+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13185990, 'num_objects': 3221, 'num_object_clones': 0, 'num_object_copies': 6442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3221, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13762, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -940}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:42.820 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:28:42.821 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'17292", 'reported_seq': 17424, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:39.186121+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:28:39.186121+0000', 'last_peered': '2024-09-15T21:28:39.186121+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:28:39.186121+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13193699, 'num_objects': 3222, 'num_object_clones': 0, 'num_object_copies': 6444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3222, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15224, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -865}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:42.821 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:42.821 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17160", 'reported_seq': 17297, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:38.728972+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:28:38.728972+0000', 'last_peered': '2024-09-15T21:28:38.728972+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:28:38.728972+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13606912, 'num_objects': 3322, 'num_object_clones': 0, 'num_object_copies': 6644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3322, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13833, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -698}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:42.821 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:28:42.821 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17388", 'reported_seq': 17512, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:39.339748+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:28:39.339748+0000', 'last_peered': '2024-09-15T21:28:39.339748+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:28:39.339748+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13332963, 'num_objects': 3256, 'num_object_clones': 0, 'num_object_copies': 6512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3256, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15247, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -834}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:42.821 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:42.821 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17397", 'reported_seq': 17536, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:38.725990+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:28:38.725990+0000', 'last_peered': '2024-09-15T21:28:38.725990+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:28:38.725990+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13471744, 'num_objects': 3289, 'num_object_clones': 0, 'num_object_copies': 6578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3289, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14341, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -805}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:42.821 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:28:42.822 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'17357", 'reported_seq': 17491, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:38.778691+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:28:38.778691+0000', 'last_peered': '2024-09-15T21:28:38.778691+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:28:38.778691+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13242368, 'num_objects': 3233, 'num_object_clones': 0, 'num_object_copies': 6466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3233, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14266, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -906}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:42.822 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:28:42.822 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'17454", 'reported_seq': 17599, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:38.720777+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:28:38.720777+0000', 'last_peered': '2024-09-15T21:28:38.720777+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:28:38.720777+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13976518, 'num_objects': 3414, 'num_object_clones': 0, 'num_object_copies': 6828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3414, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14158, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -733}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:42.822 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:43.572 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:43.573 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39363/104520 objects degraded (37.661%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":575,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":52260,"data_bytes":214008585,"bytes_used":1859514368,"bytes_avail":288050778112,"bytes_total":289910292480,"degraded_objects":39363,"degraded_total":104520,"degraded_ratio":0.37660734787600458,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":386},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:43.606 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:43.606 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:44.176 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:44.178 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:44.197 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-15T21:28:42.840576+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":214008585,"num_objects":52260,"num_object_clones":0,"num_object_copies":104520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39363,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52260,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":238199,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10112,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160807,"ondisk_log_size":160807,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1815932,"kb_used_data":617224,"kb_used_omap":12,"kb_used_meta":1198643,"kb_avail":281299588,"statfs":{"total":289910292480,"available":288050778112,"internally_reserved":0,"allocated":632037376,"data_stored":631148616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1227410435},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,5,9,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19013632,"num_objects":-4642,"num_object_clones":0,"num_object_copies":-9284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-221,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4642,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":221,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005472"},"pg_stats":[{"pgid":"2.f","version":"47'17798","reported_seq":17879,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.418577+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:38.418577+0000","last_peered":"2024-09-15T21:28:38.418577+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:38.418577+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13737984,"num_objects":3354,"num_object_clones":0,"num_object_copies":6708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17798,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-778}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17420","reported_seq":17530,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:39.352943+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:39.352943+0000","last_peered":"2024-09-15T21:28:39.352943+0000","last_clean":"2024-09-15T21:28:39.352943+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:39.352943+0000","last_undegraded":"2024-09-15T21:28:39.352943+0000","last_fullsized":"2024-09-15T21:28:39.352943+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13329350,"num_objects":3256,"num_object_clones":0,"num_object_copies":6512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3256,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15701,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17814","reported_seq":21671,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:39.360148+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:39.360148+0000","last_peered":"2024-09-15T21:28:39.360148+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:39.360148+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13479960,"num_objects":3292,"num_object_clones":0,"num_object_copies":6584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2406,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3292,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15914,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1862,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":886}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17113","reported_seq":17189,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:39.327331+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:39.327331+0000","last_peered":"2024-09-15T21:28:39.327331+0000","last_clean":"2024-09-15T21:28:39.327331+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:39.327331+0000","last_undegraded":"2024-09-15T21:28:39.327331+0000","last_fullsized":"2024-09-15T21:28:39.327331+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13111296,"num_objects":3201,"num_object_clones":0,"num_object_copies":6402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17113,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17266","reported_seq":17404,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:38.756020+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:38.756020+0000","last_peered":"2024-09-15T21:28:38.756020+0000","last_clean":"2024-09-15T21:28:38.756020+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:38.756020+0000","last_undegraded":"2024-09-15T21:28:38.756020+0000","last_fullsized":"2024-09-15T21:28:38.756020+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13267427,"num_objects":3240,"num_object_clones":0,"num_object_copies":6480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13721,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17327","reported_seq":17469,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.632873+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:38.632873+0000","last_peered":"2024-09-15T21:28:38.632873+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:38.632873+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13550051,"num_objects":3309,"num_object_clones":0,"num_object_copies":6618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14003,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-767},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17451","reported_seq":17593,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.665379+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:38.665379+0000","last_peered":"2024-09-15T21:28:38.665379+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:38.665379+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13185990,"num_objects":3221,"num_object_clones":0,"num_object_copies":6442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13762,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-940},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17631","reported_seq":17765,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:38.718400+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:38.718400+0000","last_peered":"2024-09-15T21:28:38.718400+0000","last_clean":"2024-09-15T21:28:38.718400+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:38.718400+0000","last_undegraded":"2024-09-15T21:28:38.718400+0000","last_fullsized":"2024-09-15T21:28:38.718400+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13258752,"num_objects":3237,"num_object_clones":0,"num_object_copies":6474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14403,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17292","reported_seq":17424,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:39.186121+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:39.186121+0000","last_peered":"2024-09-15T21:28:39.186121+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:39.186121+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13193699,"num_objects":3222,"num_object_clones":0,"num_object_copies":6444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3222,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15224,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-865}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17160","reported_seq":17297,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.728972+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:38.728972+0000","last_peered":"2024-09-15T21:28:38.728972+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:38.728972+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13606912,"num_objects":3322,"num_object_clones":0,"num_object_copies":6644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13833,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-698}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17388","reported_seq":17512,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:39.339748+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:39.339748+0000","last_peered":"2024-09-15T21:28:39.339748+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:39.339748+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13332963,"num_objects":3256,"num_object_clones":0,"num_object_copies":6512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3256,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15247,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-834}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17405","reported_seq":25869,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:39.339088+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:39.339088+0000","last_peered":"2024-09-15T21:28:39.339088+0000","last_clean":"2024-09-15T21:28:39.339088+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:39.339088+0000","last_undegraded":"2024-09-15T21:28:39.339088+0000","last_fullsized":"2024-09-15T21:28:39.339088+0000","mapping_epoch":44,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12922880,"num_objects":3155,"num_object_clones":0,"num_object_copies":6310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15098,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17397","reported_seq":17536,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.725990+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:38.725990+0000","last_peered":"2024-09-15T21:28:38.725990+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:38.725990+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13471744,"num_objects":3289,"num_object_clones":0,"num_object_copies":6578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14341,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-805}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17357","reported_seq":17491,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.778691+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:38.778691+0000","last_peered":"2024-09-15T21:28:38.778691+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:38.778691+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13242368,"num_objects":3233,"num_object_clones":0,"num_object_copies":6466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14266,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17131","reported_seq":25446,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:39.942214+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:39.942214+0000","last_peered":"2024-09-15T21:28:39.942214+0000","last_clean":"2024-09-15T21:28:39.942214+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:39.942214+0000","last_undegraded":"2024-09-15T21:28:39.942214+0000","last_fullsized":"2024-09-15T21:28:39.942214+0000","mapping_epoch":46,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13340672,"num_objects":3257,"num_object_clones":0,"num_object_copies":6514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3257,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13615,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17454","reported_seq":17599,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:38.720777+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:38.720777+0000","last_peered":"2024-09-15T21:28:38.720777+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:38.720777+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13976518,"num_objects":3414,"num_object_clones":0,"num_object_copies":6828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3414,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14158,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-733},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":214008566,"num_objects":52258,"num_object_clones":0,"num_object_copies":104516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39363,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52258,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":238197,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10111,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":629334016,"data_stored":629188578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160804,"ondisk_log_size":160804,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582140,"kb_used_data":235308,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93789700,"statfs":{"total":96636764160,"available":96040652800,"internally_reserved":0,"allocated":240955392,"data_stored":240670134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,0,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":662096,"kb_used_data":225084,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93709744,"statfs":{"total":96636764160,"available":95958777856,"internally_reserved":0,"allocated":230486016,"data_stored":230200753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571696,"kb_used_data":156832,"kb_used_omap":4,"kb_used_meta":414843,"kb_avail":93800144,"statfs":{"total":96636764160,"available":96051347456,"internally_reserved":0,"allocated":160595968,"data_stored":160277729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":424799919},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":229384192,"data_stored":229348062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":159653888,"data_stored":159584782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":240295936,"data_stored":240255734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:47.199 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:47.742 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:47.743 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:47.763 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-15T21:28:46.842510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":204415753,"num_objects":49918,"num_object_clones":0,"num_object_copies":99836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49918,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":240541,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10233,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160749,"ondisk_log_size":160749,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1798452,"kb_used_data":598336,"kb_used_omap":12,"kb_used_meta":1200051,"kb_avail":281317068,"statfs":{"total":289910292480,"available":288068677632,"internally_reserved":0,"allocated":612696064,"data_stored":611807304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1228852227},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,8,9,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19738624,"num_objects":-4819,"num_object_clones":0,"num_object_copies":-9638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4819,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005912"},"pg_stats":[{"pgid":"2.f","version":"47'17956","reported_seq":18037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.723849+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:43.723849+0000","last_peered":"2024-09-15T21:28:43.723849+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:43.723849+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13090816,"num_objects":3196,"num_object_clones":0,"num_object_copies":6392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17956,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-936}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17565","reported_seq":17676,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:44.289199+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:44.289199+0000","last_peered":"2024-09-15T21:28:44.289199+0000","last_clean":"2024-09-15T21:28:44.289199+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:44.289199+0000","last_undegraded":"2024-09-15T21:28:44.289199+0000","last_fullsized":"2024-09-15T21:28:44.289199+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12735430,"num_objects":3111,"num_object_clones":0,"num_object_copies":6222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3111,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15846,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17946","reported_seq":22044,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:44.350919+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:44.350919+0000","last_peered":"2024-09-15T21:28:44.350919+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:44.350919+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12939288,"num_objects":3160,"num_object_clones":0,"num_object_copies":6320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16046,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1983,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":875}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17265","reported_seq":17341,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:44.317394+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:44.317394+0000","last_peered":"2024-09-15T21:28:44.317394+0000","last_clean":"2024-09-15T21:28:44.317394+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:44.317394+0000","last_undegraded":"2024-09-15T21:28:44.317394+0000","last_fullsized":"2024-09-15T21:28:44.317394+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12488704,"num_objects":3049,"num_object_clones":0,"num_object_copies":6098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17265,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17419","reported_seq":17557,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:43.779000+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:43.779000+0000","last_peered":"2024-09-15T21:28:43.779000+0000","last_clean":"2024-09-15T21:28:43.779000+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:43.779000+0000","last_undegraded":"2024-09-15T21:28:43.779000+0000","last_fullsized":"2024-09-15T21:28:43.779000+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12640739,"num_objects":3087,"num_object_clones":0,"num_object_copies":6174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13874,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17503","reported_seq":17645,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.747137+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:43.747137+0000","last_peered":"2024-09-15T21:28:43.747137+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:43.747137+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12829155,"num_objects":3133,"num_object_clones":0,"num_object_copies":6266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3133,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14179,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-943},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17577","reported_seq":17719,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.688662+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:43.688662+0000","last_peered":"2024-09-15T21:28:43.688662+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:43.688662+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12669894,"num_objects":3095,"num_object_clones":0,"num_object_copies":6190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13888,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1066},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17767","reported_seq":17901,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:43.698580+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:43.698580+0000","last_peered":"2024-09-15T21:28:43.698580+0000","last_clean":"2024-09-15T21:28:43.698580+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:43.698580+0000","last_undegraded":"2024-09-15T21:28:43.698580+0000","last_fullsized":"2024-09-15T21:28:43.698580+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12701696,"num_objects":3101,"num_object_clones":0,"num_object_copies":6202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14539,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17443","reported_seq":17575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:44.364079+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:44.364079+0000","last_peered":"2024-09-15T21:28:44.364079+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:44.364079+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12575203,"num_objects":3071,"num_object_clones":0,"num_object_copies":6142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15375,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1016}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17298","reported_seq":17434,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.764070+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:43.764070+0000","last_peered":"2024-09-15T21:28:43.764070+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:43.764070+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13041664,"num_objects":3184,"num_object_clones":0,"num_object_copies":6368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13971,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-836}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17531","reported_seq":17656,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:44.359440+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:44.359440+0000","last_peered":"2024-09-15T21:28:44.359440+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:44.359440+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12747235,"num_objects":3113,"num_object_clones":0,"num_object_copies":6226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3113,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15390,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17548","reported_seq":26012,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:44.337923+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:44.337923+0000","last_peered":"2024-09-15T21:28:44.337923+0000","last_clean":"2024-09-15T21:28:44.337923+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:44.337923+0000","last_undegraded":"2024-09-15T21:28:44.337923+0000","last_fullsized":"2024-09-15T21:28:44.337923+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12337152,"num_objects":3012,"num_object_clones":0,"num_object_copies":6024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15241,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17535","reported_seq":17674,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.722970+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:43.722970+0000","last_peered":"2024-09-15T21:28:43.722970+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:43.722970+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12906496,"num_objects":3151,"num_object_clones":0,"num_object_copies":6302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14479,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-943}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17507","reported_seq":17645,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.664764+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:43.664764+0000","last_peered":"2024-09-15T21:28:43.664764+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:43.664764+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12627968,"num_objects":3083,"num_object_clones":0,"num_object_copies":6166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14416,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1056}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17281","reported_seq":25596,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:44.984776+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:44.984776+0000","last_peered":"2024-09-15T21:28:44.984776+0000","last_clean":"2024-09-15T21:28:44.984776+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:44.984776+0000","last_undegraded":"2024-09-15T21:28:44.984776+0000","last_fullsized":"2024-09-15T21:28:44.984776+0000","mapping_epoch":46,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12726272,"num_objects":3107,"num_object_clones":0,"num_object_copies":6214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13765,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17605","reported_seq":17750,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.673963+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:43.673963+0000","last_peered":"2024-09-15T21:28:43.673963+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:43.673963+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13358022,"num_objects":3263,"num_object_clones":0,"num_object_copies":6526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14309,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-884},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":204415734,"num_objects":49916,"num_object_clones":0,"num_object_copies":99832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49916,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":240539,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10232,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":610131968,"data_stored":609986530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565096,"kb_used_data":230744,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93806744,"statfs":{"total":96636764160,"available":96058105856,"internally_reserved":0,"allocated":236281856,"data_stored":235996598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646472,"kb_used_data":217784,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93725368,"statfs":{"total":96636764160,"available":95974776832,"internally_reserved":0,"allocated":223010816,"data_stored":222725553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586884,"kb_used_data":149808,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93784956,"statfs":{"total":96636764160,"available":96035794944,"internally_reserved":0,"allocated":153403392,"data_stored":153085153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":447540911},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,1,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":222052352,"data_stored":222016222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":152477696,"data_stored":152408590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":235601920,"data_stored":235561718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:47.765 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:48.314 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:48.315 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:48.340 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-15T21:28:46.842510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":204415753,"num_objects":49918,"num_object_clones":0,"num_object_copies":99836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49918,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":240541,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10233,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160749,"ondisk_log_size":160749,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1798452,"kb_used_data":598336,"kb_used_omap":12,"kb_used_meta":1200051,"kb_avail":281317068,"statfs":{"total":289910292480,"available":288068677632,"internally_reserved":0,"allocated":612696064,"data_stored":611807304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1228852227},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,8,9,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19738624,"num_objects":-4819,"num_object_clones":0,"num_object_copies":-9638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4819,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005912"},"pg_stats":[{"pgid":"2.f","version":"47'17956","reported_seq":18037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.723849+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:43.723849+0000","last_peered":"2024-09-15T21:28:43.723849+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:43.723849+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13090816,"num_objects":3196,"num_object_clones":0,"num_object_copies":6392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17956,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-936}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17565","reported_seq":17676,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:44.289199+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:44.289199+0000","last_peered":"2024-09-15T21:28:44.289199+0000","last_clean":"2024-09-15T21:28:44.289199+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:44.289199+0000","last_undegraded":"2024-09-15T21:28:44.289199+0000","last_fullsized":"2024-09-15T21:28:44.289199+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12735430,"num_objects":3111,"num_object_clones":0,"num_object_copies":6222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3111,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15846,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17946","reported_seq":22044,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:44.350919+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:44.350919+0000","last_peered":"2024-09-15T21:28:44.350919+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:44.350919+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12939288,"num_objects":3160,"num_object_clones":0,"num_object_copies":6320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16046,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1983,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":875}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17265","reported_seq":17341,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:44.317394+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:44.317394+0000","last_peered":"2024-09-15T21:28:44.317394+0000","last_clean":"2024-09-15T21:28:44.317394+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:44.317394+0000","last_undegraded":"2024-09-15T21:28:44.317394+0000","last_fullsized":"2024-09-15T21:28:44.317394+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12488704,"num_objects":3049,"num_object_clones":0,"num_object_copies":6098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17265,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17419","reported_seq":17557,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:43.779000+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:43.779000+0000","last_peered":"2024-09-15T21:28:43.779000+0000","last_clean":"2024-09-15T21:28:43.779000+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:43.779000+0000","last_undegraded":"2024-09-15T21:28:43.779000+0000","last_fullsized":"2024-09-15T21:28:43.779000+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12640739,"num_objects":3087,"num_object_clones":0,"num_object_copies":6174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13874,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17503","reported_seq":17645,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.747137+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:43.747137+0000","last_peered":"2024-09-15T21:28:43.747137+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:43.747137+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12829155,"num_objects":3133,"num_object_clones":0,"num_object_copies":6266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3133,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14179,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-943},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17577","reported_seq":17719,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.688662+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:43.688662+0000","last_peered":"2024-09-15T21:28:43.688662+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:43.688662+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12669894,"num_objects":3095,"num_object_clones":0,"num_object_copies":6190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13888,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1066},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17767","reported_seq":17901,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:43.698580+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:43.698580+0000","last_peered":"2024-09-15T21:28:43.698580+0000","last_clean":"2024-09-15T21:28:43.698580+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:43.698580+0000","last_undegraded":"2024-09-15T21:28:43.698580+0000","last_fullsized":"2024-09-15T21:28:43.698580+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12701696,"num_objects":3101,"num_object_clones":0,"num_object_copies":6202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14539,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17443","reported_seq":17575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:44.364079+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:44.364079+0000","last_peered":"2024-09-15T21:28:44.364079+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:44.364079+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12575203,"num_objects":3071,"num_object_clones":0,"num_object_copies":6142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15375,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1016}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17298","reported_seq":17434,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.764070+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:43.764070+0000","last_peered":"2024-09-15T21:28:43.764070+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:43.764070+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13041664,"num_objects":3184,"num_object_clones":0,"num_object_copies":6368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13971,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-836}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17531","reported_seq":17656,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:44.359440+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:44.359440+0000","last_peered":"2024-09-15T21:28:44.359440+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:44.359440+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12747235,"num_objects":3113,"num_object_clones":0,"num_object_copies":6226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3113,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15390,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17548","reported_seq":26012,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:44.337923+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:44.337923+0000","last_peered":"2024-09-15T21:28:44.337923+0000","last_clean":"2024-09-15T21:28:44.337923+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:44.337923+0000","last_undegraded":"2024-09-15T21:28:44.337923+0000","last_fullsized":"2024-09-15T21:28:44.337923+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12337152,"num_objects":3012,"num_object_clones":0,"num_object_copies":6024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15241,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17535","reported_seq":17674,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.722970+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:43.722970+0000","last_peered":"2024-09-15T21:28:43.722970+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:43.722970+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12906496,"num_objects":3151,"num_object_clones":0,"num_object_copies":6302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14479,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-943}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17507","reported_seq":17645,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.664764+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:43.664764+0000","last_peered":"2024-09-15T21:28:43.664764+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:43.664764+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12627968,"num_objects":3083,"num_object_clones":0,"num_object_copies":6166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14416,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1056}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17281","reported_seq":25596,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:44.984776+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:44.984776+0000","last_peered":"2024-09-15T21:28:44.984776+0000","last_clean":"2024-09-15T21:28:44.984776+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:44.984776+0000","last_undegraded":"2024-09-15T21:28:44.984776+0000","last_fullsized":"2024-09-15T21:28:44.984776+0000","mapping_epoch":46,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12726272,"num_objects":3107,"num_object_clones":0,"num_object_copies":6214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13765,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17605","reported_seq":17750,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.673963+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:43.673963+0000","last_peered":"2024-09-15T21:28:43.673963+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:43.673963+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13358022,"num_objects":3263,"num_object_clones":0,"num_object_copies":6526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14309,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-884},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":204415734,"num_objects":49916,"num_object_clones":0,"num_object_copies":99832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49916,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":240539,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10232,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":610131968,"data_stored":609986530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565096,"kb_used_data":230744,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93806744,"statfs":{"total":96636764160,"available":96058105856,"internally_reserved":0,"allocated":236281856,"data_stored":235996598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646472,"kb_used_data":217784,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93725368,"statfs":{"total":96636764160,"available":95974776832,"internally_reserved":0,"allocated":223010816,"data_stored":222725553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586884,"kb_used_data":149808,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93784956,"statfs":{"total":96636764160,"available":96035794944,"internally_reserved":0,"allocated":153403392,"data_stored":153085153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":447540911},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,1,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":222052352,"data_stored":222016222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":152477696,"data_stored":152408590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":235601920,"data_stored":235561718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:48.341 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:28:48.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'17956", 'reported_seq': 18037, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:43.723849+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:28:43.723849+0000', 'last_peered': '2024-09-15T21:28:43.723849+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:28:43.723849+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13090816, 'num_objects': 3196, 'num_object_clones': 0, 'num_object_copies': 6392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3196, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17956, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -936}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:48.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:28:48.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'17946", 'reported_seq': 22044, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:44.350919+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:28:44.350919+0000', 'last_peered': '2024-09-15T21:28:44.350919+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:28:44.350919+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12939288, 'num_objects': 3160, 'num_object_clones': 0, 'num_object_copies': 6320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2285, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3160, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16046, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1983, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 875}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:48.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:48.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17503", 'reported_seq': 17645, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:43.747137+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:28:43.747137+0000', 'last_peered': '2024-09-15T21:28:43.747137+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:28:43.747137+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12829155, 'num_objects': 3133, 'num_object_clones': 0, 'num_object_copies': 6266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3133, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14179, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -943}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:48.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:48.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17577", 'reported_seq': 17719, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:43.688662+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:28:43.688662+0000', 'last_peered': '2024-09-15T21:28:43.688662+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:28:43.688662+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12669894, 'num_objects': 3095, 'num_object_clones': 0, 'num_object_copies': 6190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3095, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13888, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1066}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:48.342 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:28:48.342 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'17443", 'reported_seq': 17575, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:44.364079+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:28:44.364079+0000', 'last_peered': '2024-09-15T21:28:44.364079+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:28:44.364079+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12575203, 'num_objects': 3071, 'num_object_clones': 0, 'num_object_copies': 6142, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3071, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15375, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1016}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:48.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:48.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17298", 'reported_seq': 17434, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:43.764070+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:28:43.764070+0000', 'last_peered': '2024-09-15T21:28:43.764070+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:28:43.764070+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13041664, 'num_objects': 3184, 'num_object_clones': 0, 'num_object_copies': 6368, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3184, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13971, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -836}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:48.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:28:48.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17531", 'reported_seq': 17656, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:44.359440+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:28:44.359440+0000', 'last_peered': '2024-09-15T21:28:44.359440+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:28:44.359440+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12747235, 'num_objects': 3113, 'num_object_clones': 0, 'num_object_copies': 6226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3113, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15390, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -977}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:48.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:48.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17535", 'reported_seq': 17674, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:43.722970+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:28:43.722970+0000', 'last_peered': '2024-09-15T21:28:43.722970+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:28:43.722970+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12906496, 'num_objects': 3151, 'num_object_clones': 0, 'num_object_copies': 6302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3151, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14479, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -943}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:48.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:28:48.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'17507", 'reported_seq': 17645, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:43.664764+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:28:43.664764+0000', 'last_peered': '2024-09-15T21:28:43.664764+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:28:43.664764+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12627968, 'num_objects': 3083, 'num_object_clones': 0, 'num_object_copies': 6166, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3083, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14416, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1056}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:48.343 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:28:48.343 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'17605", 'reported_seq': 17750, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:43.673963+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:28:43.673963+0000', 'last_peered': '2024-09-15T21:28:43.673963+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:28:43.673963+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13358022, 'num_objects': 3263, 'num_object_clones': 0, 'num_object_copies': 6526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3263, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14309, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -884}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:48.344 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:49.048 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:49.048 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39242/99836 objects degraded (39.306%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":580,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":49918,"data_bytes":204415753,"bytes_used":1841614848,"bytes_avail":288068677632,"bytes_total":289910292480,"degraded_objects":39242,"degraded_total":99836,"degraded_ratio":0.39306462598661812,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":390},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:49.074 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:49.074 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:49.651 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:49.654 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:49.676 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-15T21:28:48.843217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":204415753,"num_objects":49918,"num_object_clones":0,"num_object_copies":99836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49918,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":240541,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10233,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160749,"ondisk_log_size":160749,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1798452,"kb_used_data":598336,"kb_used_omap":12,"kb_used_meta":1200051,"kb_avail":281317068,"statfs":{"total":289910292480,"available":288068677632,"internally_reserved":0,"allocated":612696064,"data_stored":611807304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1228852227},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,5,8,9,12],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19214336,"num_objects":-4691,"num_object_clones":0,"num_object_copies":-9382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4691,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005908"},"pg_stats":[{"pgid":"2.f","version":"47'17956","reported_seq":18037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.723849+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:43.723849+0000","last_peered":"2024-09-15T21:28:43.723849+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:43.723849+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13090816,"num_objects":3196,"num_object_clones":0,"num_object_copies":6392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17956,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-936}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17565","reported_seq":17676,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:44.289199+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:44.289199+0000","last_peered":"2024-09-15T21:28:44.289199+0000","last_clean":"2024-09-15T21:28:44.289199+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:44.289199+0000","last_undegraded":"2024-09-15T21:28:44.289199+0000","last_fullsized":"2024-09-15T21:28:44.289199+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12735430,"num_objects":3111,"num_object_clones":0,"num_object_copies":6222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3111,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15846,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'17946","reported_seq":22044,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:44.350919+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:44.350919+0000","last_peered":"2024-09-15T21:28:44.350919+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:44.350919+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12939288,"num_objects":3160,"num_object_clones":0,"num_object_copies":6320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3160,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16046,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1983,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":875}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17265","reported_seq":17341,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:44.317394+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:44.317394+0000","last_peered":"2024-09-15T21:28:44.317394+0000","last_clean":"2024-09-15T21:28:44.317394+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:44.317394+0000","last_undegraded":"2024-09-15T21:28:44.317394+0000","last_fullsized":"2024-09-15T21:28:44.317394+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12488704,"num_objects":3049,"num_object_clones":0,"num_object_copies":6098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17265,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17419","reported_seq":17557,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:43.779000+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:43.779000+0000","last_peered":"2024-09-15T21:28:43.779000+0000","last_clean":"2024-09-15T21:28:43.779000+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:43.779000+0000","last_undegraded":"2024-09-15T21:28:43.779000+0000","last_fullsized":"2024-09-15T21:28:43.779000+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12640739,"num_objects":3087,"num_object_clones":0,"num_object_copies":6174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3087,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13874,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17503","reported_seq":17645,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.747137+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:43.747137+0000","last_peered":"2024-09-15T21:28:43.747137+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:43.747137+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12829155,"num_objects":3133,"num_object_clones":0,"num_object_copies":6266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3133,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14179,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-943},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17577","reported_seq":17719,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.688662+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:43.688662+0000","last_peered":"2024-09-15T21:28:43.688662+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:43.688662+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12669894,"num_objects":3095,"num_object_clones":0,"num_object_copies":6190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13888,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1066},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17767","reported_seq":17901,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:43.698580+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:43.698580+0000","last_peered":"2024-09-15T21:28:43.698580+0000","last_clean":"2024-09-15T21:28:43.698580+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:43.698580+0000","last_undegraded":"2024-09-15T21:28:43.698580+0000","last_fullsized":"2024-09-15T21:28:43.698580+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12701696,"num_objects":3101,"num_object_clones":0,"num_object_copies":6202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14539,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17443","reported_seq":17575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:44.364079+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:44.364079+0000","last_peered":"2024-09-15T21:28:44.364079+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:44.364079+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12575203,"num_objects":3071,"num_object_clones":0,"num_object_copies":6142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3071,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15375,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1016}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17298","reported_seq":17434,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.764070+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:43.764070+0000","last_peered":"2024-09-15T21:28:43.764070+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:43.764070+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13041664,"num_objects":3184,"num_object_clones":0,"num_object_copies":6368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13971,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-836}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17531","reported_seq":17656,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:44.359440+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:44.359440+0000","last_peered":"2024-09-15T21:28:44.359440+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:44.359440+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12747235,"num_objects":3113,"num_object_clones":0,"num_object_copies":6226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3113,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15390,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17548","reported_seq":26012,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:44.337923+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:44.337923+0000","last_peered":"2024-09-15T21:28:44.337923+0000","last_clean":"2024-09-15T21:28:44.337923+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:44.337923+0000","last_undegraded":"2024-09-15T21:28:44.337923+0000","last_fullsized":"2024-09-15T21:28:44.337923+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12337152,"num_objects":3012,"num_object_clones":0,"num_object_copies":6024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15241,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17535","reported_seq":17674,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.722970+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:43.722970+0000","last_peered":"2024-09-15T21:28:43.722970+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:43.722970+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12906496,"num_objects":3151,"num_object_clones":0,"num_object_copies":6302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14479,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-943}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17507","reported_seq":17645,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.664764+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:43.664764+0000","last_peered":"2024-09-15T21:28:43.664764+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:43.664764+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12627968,"num_objects":3083,"num_object_clones":0,"num_object_copies":6166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14416,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1056}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17281","reported_seq":25596,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:44.984776+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:44.984776+0000","last_peered":"2024-09-15T21:28:44.984776+0000","last_clean":"2024-09-15T21:28:44.984776+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:44.984776+0000","last_undegraded":"2024-09-15T21:28:44.984776+0000","last_fullsized":"2024-09-15T21:28:44.984776+0000","mapping_epoch":46,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12726272,"num_objects":3107,"num_object_clones":0,"num_object_copies":6214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13765,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17605","reported_seq":17750,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:43.673963+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:43.673963+0000","last_peered":"2024-09-15T21:28:43.673963+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:43.673963+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13358022,"num_objects":3263,"num_object_clones":0,"num_object_copies":6526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14309,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-884},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":204415734,"num_objects":49916,"num_object_clones":0,"num_object_copies":99832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49916,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":240539,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10232,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":610131968,"data_stored":609986530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":565096,"kb_used_data":230744,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93806744,"statfs":{"total":96636764160,"available":96058105856,"internally_reserved":0,"allocated":236281856,"data_stored":235996598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646472,"kb_used_data":217784,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93725368,"statfs":{"total":96636764160,"available":95974776832,"internally_reserved":0,"allocated":223010816,"data_stored":222725553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586884,"kb_used_data":149808,"kb_used_omap":4,"kb_used_meta":437051,"kb_avail":93784956,"statfs":{"total":96636764160,"available":96035794944,"internally_reserved":0,"allocated":153403392,"data_stored":153085153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":447540911},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,1,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":222052352,"data_stored":222016222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":152477696,"data_stored":152408590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":235601920,"data_stored":235561718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:52.679 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:53.214 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:53.217 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:53.235 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-15T21:28:52.844920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":194528009,"num_objects":47504,"num_object_clones":0,"num_object_copies":95008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47504,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":242955,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10344,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160763,"ondisk_log_size":160763,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1815768,"kb_used_data":579540,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281299752,"statfs":{"total":289910292480,"available":288050946048,"internally_reserved":0,"allocated":593448960,"data_stored":592560200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1265814531},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,3,1,6,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19480576,"num_objects":-4756,"num_object_clones":0,"num_object_copies":-9512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4756,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005093"},"pg_stats":[{"pgid":"2.f","version":"47'18087","reported_seq":18168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.605510+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:48.605510+0000","last_peered":"2024-09-15T21:28:48.605510+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:48.605510+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12554240,"num_objects":3065,"num_object_clones":0,"num_object_copies":6130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18087,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1067}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17710","reported_seq":17821,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:49.321978+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:49.321978+0000","last_peered":"2024-09-15T21:28:49.321978+0000","last_clean":"2024-09-15T21:28:49.321978+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:49.321978+0000","last_undegraded":"2024-09-15T21:28:49.321978+0000","last_fullsized":"2024-09-15T21:28:49.321978+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12141510,"num_objects":2966,"num_object_clones":0,"num_object_copies":5932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2966,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15991,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18100","reported_seq":22421,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:49.353834+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:49.353834+0000","last_peered":"2024-09-15T21:28:49.353834+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:49.353834+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12308504,"num_objects":3006,"num_object_clones":0,"num_object_copies":6012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3006,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16200,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2094,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":832}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17404","reported_seq":17480,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:49.367037+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:49.367037+0000","last_peered":"2024-09-15T21:28:49.367037+0000","last_clean":"2024-09-15T21:28:49.367037+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:49.367037+0000","last_undegraded":"2024-09-15T21:28:49.367037+0000","last_fullsized":"2024-09-15T21:28:49.367037+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11919360,"num_objects":2910,"num_object_clones":0,"num_object_copies":5820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17404,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17582","reported_seq":17720,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.801832+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:48.801832+0000","last_peered":"2024-09-15T21:28:48.801832+0000","last_clean":"2024-09-15T21:28:48.801832+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:48.801832+0000","last_undegraded":"2024-09-15T21:28:48.801832+0000","last_fullsized":"2024-09-15T21:28:48.801832+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11973091,"num_objects":2924,"num_object_clones":0,"num_object_copies":5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14037,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17641","reported_seq":17783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.757803+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:48.757803+0000","last_peered":"2024-09-15T21:28:48.757803+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:48.757803+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12263907,"num_objects":2995,"num_object_clones":0,"num_object_copies":5990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2995,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14317,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1081},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17721","reported_seq":17863,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.777064+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:48.777064+0000","last_peered":"2024-09-15T21:28:48.777064+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:48.777064+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12080070,"num_objects":2951,"num_object_clones":0,"num_object_copies":5902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14032,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1210},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17923","reported_seq":18057,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.810448+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:48.810448+0000","last_peered":"2024-09-15T21:28:48.810448+0000","last_clean":"2024-09-15T21:28:48.810448+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:48.810448+0000","last_undegraded":"2024-09-15T21:28:48.810448+0000","last_fullsized":"2024-09-15T21:28:48.810448+0000","mapping_epoch":34,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12062720,"num_objects":2945,"num_object_clones":0,"num_object_copies":5890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14695,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17601","reported_seq":17733,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:49.402096+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:49.402096+0000","last_peered":"2024-09-15T21:28:49.402096+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:49.402096+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11928035,"num_objects":2913,"num_object_clones":0,"num_object_copies":5826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2913,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15533,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1174}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17432","reported_seq":17569,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.827332+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:48.827332+0000","last_peered":"2024-09-15T21:28:48.827332+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:48.827332+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12492800,"num_objects":3050,"num_object_clones":0,"num_object_copies":6100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14105,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-970}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17690","reported_seq":17813,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:49.376116+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:49.376116+0000","last_peered":"2024-09-15T21:28:49.376116+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:49.376116+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12095971,"num_objects":2954,"num_object_clones":0,"num_object_copies":5908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15549,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1136}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17703","reported_seq":26166,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:49.401326+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:49.401326+0000","last_peered":"2024-09-15T21:28:49.401326+0000","last_clean":"2024-09-15T21:28:49.401326+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:49.401326+0000","last_undegraded":"2024-09-15T21:28:49.401326+0000","last_fullsized":"2024-09-15T21:28:49.401326+0000","mapping_epoch":44,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11702272,"num_objects":2857,"num_object_clones":0,"num_object_copies":5714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15396,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17683","reported_seq":17821,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.823520+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:48.823520+0000","last_peered":"2024-09-15T21:28:48.823520+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:48.823520+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12300288,"num_objects":3003,"num_object_clones":0,"num_object_copies":6006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14627,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1091}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17663","reported_seq":17800,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.863660+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:48.863660+0000","last_peered":"2024-09-15T21:28:48.863660+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:48.863660+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11988992,"num_objects":2927,"num_object_clones":0,"num_object_copies":5854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14572,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1212}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17446","reported_seq":25757,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:50.031951+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:50.031951+0000","last_peered":"2024-09-15T21:28:50.031951+0000","last_clean":"2024-09-15T21:28:50.031951+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:50.031951+0000","last_undegraded":"2024-09-15T21:28:50.031951+0000","last_fullsized":"2024-09-15T21:28:50.031951+0000","mapping_epoch":46,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12050432,"num_objects":2942,"num_object_clones":0,"num_object_copies":5884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13930,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17774","reported_seq":17919,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.850113+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:48.850113+0000","last_peered":"2024-09-15T21:28:48.850113+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:48.850113+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12665798,"num_objects":3094,"num_object_clones":0,"num_object_copies":6188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3094,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14478,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1053},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":194527990,"num_objects":47502,"num_object_clones":0,"num_object_copies":95004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47502,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":242953,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10343,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":590553088,"data_stored":590407650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160760,"ondisk_log_size":160760,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564380,"kb_used_data":225860,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93807460,"statfs":{"total":96636764160,"available":96058839040,"internally_reserved":0,"allocated":231280640,"data_stored":230995382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":661768,"kb_used_data":210868,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93710072,"statfs":{"total":96636764160,"available":95959113728,"internally_reserved":0,"allocated":215928832,"data_stored":215643569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589620,"kb_used_data":142812,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93782220,"statfs":{"total":96636764160,"available":96032993280,"internally_reserved":0,"allocated":146239488,"data_stored":145921249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,0,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":214798336,"data_stored":214762206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":145154048,"data_stored":145084942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":230600704,"data_stored":230560502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:53.237 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:53.821 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:53.822 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:53.844 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-15T21:28:52.844920+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":194528009,"num_objects":47504,"num_object_clones":0,"num_object_copies":95008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47504,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":242955,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10344,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160763,"ondisk_log_size":160763,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1815768,"kb_used_data":579540,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281299752,"statfs":{"total":289910292480,"available":288050946048,"internally_reserved":0,"allocated":593448960,"data_stored":592560200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1265814531},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,2,3,1,6,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19480576,"num_objects":-4756,"num_object_clones":0,"num_object_copies":-9512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-232,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4756,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005093"},"pg_stats":[{"pgid":"2.f","version":"47'18087","reported_seq":18168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.605510+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:48.605510+0000","last_peered":"2024-09-15T21:28:48.605510+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:48.605510+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12554240,"num_objects":3065,"num_object_clones":0,"num_object_copies":6130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18087,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1067}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17710","reported_seq":17821,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:49.321978+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:49.321978+0000","last_peered":"2024-09-15T21:28:49.321978+0000","last_clean":"2024-09-15T21:28:49.321978+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:49.321978+0000","last_undegraded":"2024-09-15T21:28:49.321978+0000","last_fullsized":"2024-09-15T21:28:49.321978+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12141510,"num_objects":2966,"num_object_clones":0,"num_object_copies":5932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2966,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":15991,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18100","reported_seq":22421,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:49.353834+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:49.353834+0000","last_peered":"2024-09-15T21:28:49.353834+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:49.353834+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12308504,"num_objects":3006,"num_object_clones":0,"num_object_copies":6012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3006,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16200,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2094,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":832}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17404","reported_seq":17480,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:49.367037+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:49.367037+0000","last_peered":"2024-09-15T21:28:49.367037+0000","last_clean":"2024-09-15T21:28:49.367037+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:49.367037+0000","last_undegraded":"2024-09-15T21:28:49.367037+0000","last_fullsized":"2024-09-15T21:28:49.367037+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11919360,"num_objects":2910,"num_object_clones":0,"num_object_copies":5820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17404,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17582","reported_seq":17720,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.801832+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:48.801832+0000","last_peered":"2024-09-15T21:28:48.801832+0000","last_clean":"2024-09-15T21:28:48.801832+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:48.801832+0000","last_undegraded":"2024-09-15T21:28:48.801832+0000","last_fullsized":"2024-09-15T21:28:48.801832+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11973091,"num_objects":2924,"num_object_clones":0,"num_object_copies":5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14037,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17641","reported_seq":17783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.757803+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:48.757803+0000","last_peered":"2024-09-15T21:28:48.757803+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:48.757803+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12263907,"num_objects":2995,"num_object_clones":0,"num_object_copies":5990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2995,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14317,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1081},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17721","reported_seq":17863,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.777064+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:48.777064+0000","last_peered":"2024-09-15T21:28:48.777064+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:48.777064+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12080070,"num_objects":2951,"num_object_clones":0,"num_object_copies":5902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14032,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1210},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'17923","reported_seq":18057,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:48.810448+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:48.810448+0000","last_peered":"2024-09-15T21:28:48.810448+0000","last_clean":"2024-09-15T21:28:48.810448+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:48.810448+0000","last_undegraded":"2024-09-15T21:28:48.810448+0000","last_fullsized":"2024-09-15T21:28:48.810448+0000","mapping_epoch":34,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12062720,"num_objects":2945,"num_object_clones":0,"num_object_copies":5890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14695,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17601","reported_seq":17733,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:49.402096+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:49.402096+0000","last_peered":"2024-09-15T21:28:49.402096+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:49.402096+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11928035,"num_objects":2913,"num_object_clones":0,"num_object_copies":5826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2913,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15533,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1174}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17432","reported_seq":17569,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.827332+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:48.827332+0000","last_peered":"2024-09-15T21:28:48.827332+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:48.827332+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12492800,"num_objects":3050,"num_object_clones":0,"num_object_copies":6100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14105,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-970}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17690","reported_seq":17813,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:49.376116+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:49.376116+0000","last_peered":"2024-09-15T21:28:49.376116+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:49.376116+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12095971,"num_objects":2954,"num_object_clones":0,"num_object_copies":5908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15549,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1136}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17703","reported_seq":26166,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:49.401326+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:49.401326+0000","last_peered":"2024-09-15T21:28:49.401326+0000","last_clean":"2024-09-15T21:28:49.401326+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:49.401326+0000","last_undegraded":"2024-09-15T21:28:49.401326+0000","last_fullsized":"2024-09-15T21:28:49.401326+0000","mapping_epoch":44,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11702272,"num_objects":2857,"num_object_clones":0,"num_object_copies":5714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15396,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17683","reported_seq":17821,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.823520+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:48.823520+0000","last_peered":"2024-09-15T21:28:48.823520+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:48.823520+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12300288,"num_objects":3003,"num_object_clones":0,"num_object_copies":6006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3003,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14627,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1091}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17663","reported_seq":17800,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.863660+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:48.863660+0000","last_peered":"2024-09-15T21:28:48.863660+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:48.863660+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11988992,"num_objects":2927,"num_object_clones":0,"num_object_copies":5854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14572,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1212}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17446","reported_seq":25757,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:50.031951+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:50.031951+0000","last_peered":"2024-09-15T21:28:50.031951+0000","last_clean":"2024-09-15T21:28:50.031951+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:50.031951+0000","last_undegraded":"2024-09-15T21:28:50.031951+0000","last_fullsized":"2024-09-15T21:28:50.031951+0000","mapping_epoch":46,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12050432,"num_objects":2942,"num_object_clones":0,"num_object_copies":5884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13930,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17774","reported_seq":17919,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:48.850113+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:48.850113+0000","last_peered":"2024-09-15T21:28:48.850113+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:48.850113+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12665798,"num_objects":3094,"num_object_clones":0,"num_object_copies":6188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3094,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14478,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1053},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":194527990,"num_objects":47502,"num_object_clones":0,"num_object_copies":95004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39131,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47502,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":242953,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10343,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":590553088,"data_stored":590407650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160760,"ondisk_log_size":160760,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":564380,"kb_used_data":225860,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93807460,"statfs":{"total":96636764160,"available":96058839040,"internally_reserved":0,"allocated":231280640,"data_stored":230995382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":661768,"kb_used_data":210868,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93710072,"statfs":{"total":96636764160,"available":95959113728,"internally_reserved":0,"allocated":215928832,"data_stored":215643569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589620,"kb_used_data":142812,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93782220,"statfs":{"total":96636764160,"available":96032993280,"internally_reserved":0,"allocated":146239488,"data_stored":145921249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,0,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":214798336,"data_stored":214762206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":145154048,"data_stored":145084942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":230600704,"data_stored":230560502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:53.846 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:28:53.846 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18087", 'reported_seq': 18168, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:48.605510+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:28:48.605510+0000', 'last_peered': '2024-09-15T21:28:48.605510+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:28:48.605510+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12554240, 'num_objects': 3065, 'num_object_clones': 0, 'num_object_copies': 6130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3065, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18087, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1067}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:53.846 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:28:53.846 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'18100", 'reported_seq': 22421, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:49.353834+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:28:49.353834+0000', 'last_peered': '2024-09-15T21:28:49.353834+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:28:49.353834+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12308504, 'num_objects': 3006, 'num_object_clones': 0, 'num_object_copies': 6012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3006, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16200, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2094, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 832}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:53.846 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:53.847 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17641", 'reported_seq': 17783, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:48.757803+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:28:48.757803+0000', 'last_peered': '2024-09-15T21:28:48.757803+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:28:48.757803+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12263907, 'num_objects': 2995, 'num_object_clones': 0, 'num_object_copies': 5990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2995, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14317, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1081}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:53.847 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:53.847 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17721", 'reported_seq': 17863, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:48.777064+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:28:48.777064+0000', 'last_peered': '2024-09-15T21:28:48.777064+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:28:48.777064+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12080070, 'num_objects': 2951, 'num_object_clones': 0, 'num_object_copies': 5902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14032, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1210}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:53.847 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:28:53.847 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'17601", 'reported_seq': 17733, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:49.402096+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:28:49.402096+0000', 'last_peered': '2024-09-15T21:28:49.402096+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:28:49.402096+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11928035, 'num_objects': 2913, 'num_object_clones': 0, 'num_object_copies': 5826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2913, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15533, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1174}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:53.847 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:53.847 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17432", 'reported_seq': 17569, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:48.827332+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:28:48.827332+0000', 'last_peered': '2024-09-15T21:28:48.827332+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:28:48.827332+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12492800, 'num_objects': 3050, 'num_object_clones': 0, 'num_object_copies': 6100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3050, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14105, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -970}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:53.847 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:28:53.847 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17690", 'reported_seq': 17813, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:49.376116+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:28:49.376116+0000', 'last_peered': '2024-09-15T21:28:49.376116+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:28:49.376116+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12095971, 'num_objects': 2954, 'num_object_clones': 0, 'num_object_copies': 5908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2954, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15549, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1136}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:53.847 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:53.847 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17683", 'reported_seq': 17821, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:48.823520+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:28:48.823520+0000', 'last_peered': '2024-09-15T21:28:48.823520+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:28:48.823520+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12300288, 'num_objects': 3003, 'num_object_clones': 0, 'num_object_copies': 6006, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3003, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14627, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1091}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:53.848 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:28:53.848 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'17663", 'reported_seq': 17800, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:48.863660+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:28:48.863660+0000', 'last_peered': '2024-09-15T21:28:48.863660+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:28:48.863660+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11988992, 'num_objects': 2927, 'num_object_clones': 0, 'num_object_copies': 5854, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2927, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14572, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1212}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:53.848 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:28:53.848 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'17774", 'reported_seq': 17919, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:48.850113+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:28:48.850113+0000', 'last_peered': '2024-09-15T21:28:48.850113+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:28:48.850113+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12665798, 'num_objects': 3094, 'num_object_clones': 0, 'num_object_copies': 6188, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3094, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14478, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1053}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:53.848 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:28:54.545 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:54.546 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39131/95008 objects degraded (41.187%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":586,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":47504,"data_bytes":194528009,"bytes_used":1859346432,"bytes_avail":288050946048,"bytes_total":289910292480,"degraded_objects":39131,"degraded_total":95008,"degraded_ratio":0.41187057931963622,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":232,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":396},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:28:54.571 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:28:54.572 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:55.152 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:55.154 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:55.181 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-15T21:28:54.845684+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":185467657,"num_objects":45292,"num_object_clones":0,"num_object_copies":90584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45292,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":245167,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10461,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1816588,"kb_used_data":567880,"kb_used_omap":12,"kb_used_meta":1248627,"kb_avail":281298932,"statfs":{"total":289910292480,"available":288050106368,"internally_reserved":0,"allocated":581509120,"data_stored":580620360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1278594051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,3,1,2,8,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28540928,"num_objects":-6968,"num_object_clones":0,"num_object_copies":-13936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-349,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6968,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005107"},"pg_stats":[{"pgid":"2.f","version":"47'18246","reported_seq":18326,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.823108+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:53.823108+0000","last_peered":"2024-09-15T21:28:53.823108+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:53.823108+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11902976,"num_objects":2906,"num_object_clones":0,"num_object_copies":5812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18246,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1226}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17847","reported_seq":17958,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:54.357928+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:54.357928+0000","last_peered":"2024-09-15T21:28:54.357928+0000","last_clean":"2024-09-15T21:28:54.357928+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:54.357928+0000","last_undegraded":"2024-09-15T21:28:54.357928+0000","last_fullsized":"2024-09-15T21:28:54.357928+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11580358,"num_objects":2829,"num_object_clones":0,"num_object_copies":5658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2829,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16128,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18247","reported_seq":22802,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:54.439083+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:54.439083+0000","last_peered":"2024-09-15T21:28:54.439083+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:54.439083+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11706392,"num_objects":2859,"num_object_clones":0,"num_object_copies":5718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2057,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16347,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":802}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17549","reported_seq":17625,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:54.386358+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:54.386358+0000","last_peered":"2024-09-15T21:28:54.386358+0000","last_clean":"2024-09-15T21:28:54.386358+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:54.386358+0000","last_undegraded":"2024-09-15T21:28:54.386358+0000","last_fullsized":"2024-09-15T21:28:54.386358+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11325440,"num_objects":2765,"num_object_clones":0,"num_object_copies":5530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17549,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17729","reported_seq":17866,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:53.875008+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:53.875008+0000","last_peered":"2024-09-15T21:28:53.875008+0000","last_clean":"2024-09-15T21:28:53.875008+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:53.875008+0000","last_undegraded":"2024-09-15T21:28:53.875008+0000","last_fullsized":"2024-09-15T21:28:53.875008+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11370979,"num_objects":2777,"num_object_clones":0,"num_object_copies":5554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14184,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17772","reported_seq":17914,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.689023+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:53.689023+0000","last_peered":"2024-09-15T21:28:53.689023+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:53.689023+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11727331,"num_objects":2864,"num_object_clones":0,"num_object_copies":5728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2864,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14448,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1212},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17867","reported_seq":18009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.801344+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:53.801344+0000","last_peered":"2024-09-15T21:28:53.801344+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:53.801344+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11482054,"num_objects":2805,"num_object_clones":0,"num_object_copies":5610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14178,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1356},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18077","reported_seq":18212,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:53.770698+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:53.770698+0000","last_peered":"2024-09-15T21:28:53.770698+0000","last_clean":"2024-09-15T21:28:53.770698+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:53.770698+0000","last_undegraded":"2024-09-15T21:28:53.770698+0000","last_fullsized":"2024-09-15T21:28:53.770698+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11431936,"num_objects":2791,"num_object_clones":0,"num_object_copies":5582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14849,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17754","reported_seq":17887,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:54.379878+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:54.379878+0000","last_peered":"2024-09-15T21:28:54.379878+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:54.379878+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11301347,"num_objects":2760,"num_object_clones":0,"num_object_copies":5520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2760,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15686,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1327}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17583","reported_seq":17720,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.731038+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:53.731038+0000","last_peered":"2024-09-15T21:28:53.731038+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:53.731038+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11874304,"num_objects":2899,"num_object_clones":0,"num_object_copies":5798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1121}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17836","reported_seq":17960,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:54.330674+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:54.330674+0000","last_peered":"2024-09-15T21:28:54.330674+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:54.330674+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11497955,"num_objects":2808,"num_object_clones":0,"num_object_copies":5616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2808,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15695,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1282}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17847","reported_seq":26312,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:54.391239+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:54.391239+0000","last_peered":"2024-09-15T21:28:54.391239+0000","last_clean":"2024-09-15T21:28:54.391239+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:54.391239+0000","last_undegraded":"2024-09-15T21:28:54.391239+0000","last_fullsized":"2024-09-15T21:28:54.391239+0000","mapping_epoch":44,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15540,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17864","reported_seq":18003,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.771282+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:53.771282+0000","last_peered":"2024-09-15T21:28:53.771282+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:53.771282+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11558912,"num_objects":2822,"num_object_clones":0,"num_object_copies":5644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14808,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17784","reported_seq":17922,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.867714+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:53.867714+0000","last_peered":"2024-09-15T21:28:53.867714+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:53.867714+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11493376,"num_objects":2806,"num_object_clones":0,"num_object_copies":5612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14693,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17446","reported_seq":25757,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:50.031951+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:50.031951+0000","last_peered":"2024-09-15T21:28:50.031951+0000","last_clean":"2024-09-15T21:28:50.031951+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:50.031951+0000","last_undegraded":"2024-09-15T21:28:50.031951+0000","last_fullsized":"2024-09-15T21:28:50.031951+0000","mapping_epoch":46,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12050432,"num_objects":2942,"num_object_clones":0,"num_object_copies":5884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13930,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17924","reported_seq":18069,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.823503+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:53.823503+0000","last_peered":"2024-09-15T21:28:53.823503+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:53.823503+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12051398,"num_objects":2944,"num_object_clones":0,"num_object_copies":5888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14628,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1203},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":185467638,"num_objects":45290,"num_object_clones":0,"num_object_copies":90580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45290,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":245165,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10460,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":578465792,"data_stored":578320354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160872,"ondisk_log_size":160872,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563928,"kb_used_data":221256,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93807912,"statfs":{"total":96636764160,"available":96059301888,"internally_reserved":0,"allocated":226566144,"data_stored":226280886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663040,"kb_used_data":203812,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93708800,"statfs":{"total":96636764160,"available":95957811200,"internally_reserved":0,"allocated":208703488,"data_stored":208418225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589620,"kb_used_data":142812,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93782220,"statfs":{"total":96636764160,"available":96032993280,"internally_reserved":0,"allocated":146239488,"data_stored":145921249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,0,2,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207536128,"data_stored":207499998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":145154048,"data_stored":145084942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":225775616,"data_stored":225735414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:58.182 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:58.774 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:58.776 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:58.797 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-15T21:28:56.846438+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":184898313,"num_objects":45153,"num_object_clones":0,"num_object_copies":90306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45153,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":245306,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10461,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160914,"ondisk_log_size":160914,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1813536,"kb_used_data":560668,"kb_used_omap":12,"kb_used_meta":1252787,"kb_avail":281301984,"statfs":{"total":289910292480,"available":288053231616,"internally_reserved":0,"allocated":574124032,"data_stored":573235272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1282853891},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,3,4,11,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20131840,"num_objects":-4915,"num_object_clones":0,"num_object_copies":-9830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4915,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004464"},"pg_stats":[{"pgid":"2.f","version":"47'18246","reported_seq":18326,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.823108+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:53.823108+0000","last_peered":"2024-09-15T21:28:53.823108+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:53.823108+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11902976,"num_objects":2906,"num_object_clones":0,"num_object_copies":5812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18246,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1226}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17847","reported_seq":17958,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:54.357928+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:54.357928+0000","last_peered":"2024-09-15T21:28:54.357928+0000","last_clean":"2024-09-15T21:28:54.357928+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:54.357928+0000","last_undegraded":"2024-09-15T21:28:54.357928+0000","last_fullsized":"2024-09-15T21:28:54.357928+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11580358,"num_objects":2829,"num_object_clones":0,"num_object_copies":5658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2829,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16128,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18247","reported_seq":22802,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:54.439083+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:54.439083+0000","last_peered":"2024-09-15T21:28:54.439083+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:54.439083+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11706392,"num_objects":2859,"num_object_clones":0,"num_object_copies":5718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2057,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16347,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":802}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17549","reported_seq":17625,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:54.386358+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:54.386358+0000","last_peered":"2024-09-15T21:28:54.386358+0000","last_clean":"2024-09-15T21:28:54.386358+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:54.386358+0000","last_undegraded":"2024-09-15T21:28:54.386358+0000","last_fullsized":"2024-09-15T21:28:54.386358+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11325440,"num_objects":2765,"num_object_clones":0,"num_object_copies":5530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17549,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17729","reported_seq":17866,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:53.875008+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:53.875008+0000","last_peered":"2024-09-15T21:28:53.875008+0000","last_clean":"2024-09-15T21:28:53.875008+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:53.875008+0000","last_undegraded":"2024-09-15T21:28:53.875008+0000","last_fullsized":"2024-09-15T21:28:53.875008+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11370979,"num_objects":2777,"num_object_clones":0,"num_object_copies":5554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14184,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17772","reported_seq":17914,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.689023+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:53.689023+0000","last_peered":"2024-09-15T21:28:53.689023+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:53.689023+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11727331,"num_objects":2864,"num_object_clones":0,"num_object_copies":5728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2864,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14448,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1212},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17867","reported_seq":18009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.801344+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:53.801344+0000","last_peered":"2024-09-15T21:28:53.801344+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:53.801344+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11482054,"num_objects":2805,"num_object_clones":0,"num_object_copies":5610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14178,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1356},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18077","reported_seq":18212,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:53.770698+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:53.770698+0000","last_peered":"2024-09-15T21:28:53.770698+0000","last_clean":"2024-09-15T21:28:53.770698+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:53.770698+0000","last_undegraded":"2024-09-15T21:28:53.770698+0000","last_fullsized":"2024-09-15T21:28:53.770698+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11431936,"num_objects":2791,"num_object_clones":0,"num_object_copies":5582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14849,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17754","reported_seq":17887,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:54.379878+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:54.379878+0000","last_peered":"2024-09-15T21:28:54.379878+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:54.379878+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11301347,"num_objects":2760,"num_object_clones":0,"num_object_copies":5520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2760,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15686,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1327}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17583","reported_seq":17720,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.731038+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:53.731038+0000","last_peered":"2024-09-15T21:28:53.731038+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:53.731038+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11874304,"num_objects":2899,"num_object_clones":0,"num_object_copies":5798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1121}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17836","reported_seq":17960,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:54.330674+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:54.330674+0000","last_peered":"2024-09-15T21:28:54.330674+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:54.330674+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11497955,"num_objects":2808,"num_object_clones":0,"num_object_copies":5616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2808,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15695,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1282}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17847","reported_seq":26312,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:54.391239+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:54.391239+0000","last_peered":"2024-09-15T21:28:54.391239+0000","last_clean":"2024-09-15T21:28:54.391239+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:54.391239+0000","last_undegraded":"2024-09-15T21:28:54.391239+0000","last_fullsized":"2024-09-15T21:28:54.391239+0000","mapping_epoch":44,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15540,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17864","reported_seq":18003,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.771282+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:53.771282+0000","last_peered":"2024-09-15T21:28:53.771282+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:53.771282+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11558912,"num_objects":2822,"num_object_clones":0,"num_object_copies":5644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14808,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17784","reported_seq":17922,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.867714+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:53.867714+0000","last_peered":"2024-09-15T21:28:53.867714+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:53.867714+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11493376,"num_objects":2806,"num_object_clones":0,"num_object_copies":5612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14693,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17585","reported_seq":25900,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:55.105301+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:55.105301+0000","last_peered":"2024-09-15T21:28:55.105301+0000","last_clean":"2024-09-15T21:28:55.105301+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:55.105301+0000","last_undegraded":"2024-09-15T21:28:55.105301+0000","last_fullsized":"2024-09-15T21:28:55.105301+0000","mapping_epoch":46,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11481088,"num_objects":2803,"num_object_clones":0,"num_object_copies":5606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14069,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17924","reported_seq":18069,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.823503+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:53.823503+0000","last_peered":"2024-09-15T21:28:53.823503+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:53.823503+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12051398,"num_objects":2944,"num_object_clones":0,"num_object_copies":5888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14628,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1203},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":184898294,"num_objects":45151,"num_object_clones":0,"num_object_copies":90302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45151,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":245304,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10460,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":571162624,"data_stored":571017186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563928,"kb_used_data":221256,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93807912,"statfs":{"total":96636764160,"available":96059301888,"internally_reserved":0,"allocated":226566144,"data_stored":226280886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663040,"kb_used_data":203812,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93708800,"statfs":{"total":96636764160,"available":95957811200,"internally_reserved":0,"allocated":208703488,"data_stored":208418225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586568,"kb_used_data":135600,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93785272,"statfs":{"total":96636764160,"available":96036118528,"internally_reserved":0,"allocated":138854400,"data_stored":138536161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207536128,"data_stored":207499998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":137850880,"data_stored":137781774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":225775616,"data_stored":225735414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:58.798 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:28:59.354 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:28:59.356 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:28:59.381 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-15T21:28:58.847209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":184898313,"num_objects":45153,"num_object_clones":0,"num_object_copies":90306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45153,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":245306,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10461,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160914,"ondisk_log_size":160914,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1813536,"kb_used_data":560668,"kb_used_omap":12,"kb_used_meta":1252787,"kb_avail":281301984,"statfs":{"total":289910292480,"available":288053231616,"internally_reserved":0,"allocated":574124032,"data_stored":573235272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1282853891},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,3,4,11,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19517440,"num_objects":-4765,"num_object_clones":0,"num_object_copies":-9530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4765,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004699"},"pg_stats":[{"pgid":"2.f","version":"47'18246","reported_seq":18326,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.823108+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:53.823108+0000","last_peered":"2024-09-15T21:28:53.823108+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:53.823108+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11902976,"num_objects":2906,"num_object_clones":0,"num_object_copies":5812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18246,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1226}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17847","reported_seq":17958,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:54.357928+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:54.357928+0000","last_peered":"2024-09-15T21:28:54.357928+0000","last_clean":"2024-09-15T21:28:54.357928+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:54.357928+0000","last_undegraded":"2024-09-15T21:28:54.357928+0000","last_fullsized":"2024-09-15T21:28:54.357928+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11580358,"num_objects":2829,"num_object_clones":0,"num_object_copies":5658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2829,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16128,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18247","reported_seq":22802,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:54.439083+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:54.439083+0000","last_peered":"2024-09-15T21:28:54.439083+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:54.439083+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11706392,"num_objects":2859,"num_object_clones":0,"num_object_copies":5718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2057,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16347,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":802}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17549","reported_seq":17625,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:54.386358+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:54.386358+0000","last_peered":"2024-09-15T21:28:54.386358+0000","last_clean":"2024-09-15T21:28:54.386358+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:54.386358+0000","last_undegraded":"2024-09-15T21:28:54.386358+0000","last_fullsized":"2024-09-15T21:28:54.386358+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11325440,"num_objects":2765,"num_object_clones":0,"num_object_copies":5530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17549,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17729","reported_seq":17866,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:53.875008+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:53.875008+0000","last_peered":"2024-09-15T21:28:53.875008+0000","last_clean":"2024-09-15T21:28:53.875008+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:53.875008+0000","last_undegraded":"2024-09-15T21:28:53.875008+0000","last_fullsized":"2024-09-15T21:28:53.875008+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11370979,"num_objects":2777,"num_object_clones":0,"num_object_copies":5554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14184,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17772","reported_seq":17914,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.689023+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:53.689023+0000","last_peered":"2024-09-15T21:28:53.689023+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:53.689023+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11727331,"num_objects":2864,"num_object_clones":0,"num_object_copies":5728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2864,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14448,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1212},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17867","reported_seq":18009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.801344+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:53.801344+0000","last_peered":"2024-09-15T21:28:53.801344+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:53.801344+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11482054,"num_objects":2805,"num_object_clones":0,"num_object_copies":5610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14178,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1356},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18077","reported_seq":18212,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:53.770698+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:53.770698+0000","last_peered":"2024-09-15T21:28:53.770698+0000","last_clean":"2024-09-15T21:28:53.770698+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:53.770698+0000","last_undegraded":"2024-09-15T21:28:53.770698+0000","last_fullsized":"2024-09-15T21:28:53.770698+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11431936,"num_objects":2791,"num_object_clones":0,"num_object_copies":5582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14849,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17754","reported_seq":17887,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:54.379878+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:54.379878+0000","last_peered":"2024-09-15T21:28:54.379878+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:54.379878+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11301347,"num_objects":2760,"num_object_clones":0,"num_object_copies":5520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2760,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15686,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1327}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17583","reported_seq":17720,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.731038+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:53.731038+0000","last_peered":"2024-09-15T21:28:53.731038+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:53.731038+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11874304,"num_objects":2899,"num_object_clones":0,"num_object_copies":5798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1121}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17836","reported_seq":17960,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:54.330674+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:54.330674+0000","last_peered":"2024-09-15T21:28:54.330674+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:54.330674+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11497955,"num_objects":2808,"num_object_clones":0,"num_object_copies":5616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2808,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15695,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1282}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17847","reported_seq":26312,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:54.391239+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:54.391239+0000","last_peered":"2024-09-15T21:28:54.391239+0000","last_clean":"2024-09-15T21:28:54.391239+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:54.391239+0000","last_undegraded":"2024-09-15T21:28:54.391239+0000","last_fullsized":"2024-09-15T21:28:54.391239+0000","mapping_epoch":44,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15540,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17864","reported_seq":18003,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.771282+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:53.771282+0000","last_peered":"2024-09-15T21:28:53.771282+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:53.771282+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11558912,"num_objects":2822,"num_object_clones":0,"num_object_copies":5644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14808,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17784","reported_seq":17922,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.867714+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:53.867714+0000","last_peered":"2024-09-15T21:28:53.867714+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:53.867714+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11493376,"num_objects":2806,"num_object_clones":0,"num_object_copies":5612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14693,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17585","reported_seq":25900,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:55.105301+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:55.105301+0000","last_peered":"2024-09-15T21:28:55.105301+0000","last_clean":"2024-09-15T21:28:55.105301+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:55.105301+0000","last_undegraded":"2024-09-15T21:28:55.105301+0000","last_fullsized":"2024-09-15T21:28:55.105301+0000","mapping_epoch":46,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11481088,"num_objects":2803,"num_object_clones":0,"num_object_copies":5606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14069,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17924","reported_seq":18069,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.823503+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:53.823503+0000","last_peered":"2024-09-15T21:28:53.823503+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:53.823503+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12051398,"num_objects":2944,"num_object_clones":0,"num_object_copies":5888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14628,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1203},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":184898294,"num_objects":45151,"num_object_clones":0,"num_object_copies":90302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45151,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":245304,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10460,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":571162624,"data_stored":571017186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563928,"kb_used_data":221256,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93807912,"statfs":{"total":96636764160,"available":96059301888,"internally_reserved":0,"allocated":226566144,"data_stored":226280886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663040,"kb_used_data":203812,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93708800,"statfs":{"total":96636764160,"available":95957811200,"internally_reserved":0,"allocated":208703488,"data_stored":208418225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586568,"kb_used_data":135600,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93785272,"statfs":{"total":96636764160,"available":96036118528,"internally_reserved":0,"allocated":138854400,"data_stored":138536161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207536128,"data_stored":207499998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":137850880,"data_stored":137781774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":225775616,"data_stored":225735414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:28:59.383 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:28:59.383 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18246", 'reported_seq': 18326, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:53.823108+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:28:53.823108+0000', 'last_peered': '2024-09-15T21:28:53.823108+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:28:53.823108+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11902976, 'num_objects': 2906, 'num_object_clones': 0, 'num_object_copies': 5812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2906, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18246, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1226}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:59.383 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:28:59.383 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'18247", 'reported_seq': 22802, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:54.439083+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:28:54.439083+0000', 'last_peered': '2024-09-15T21:28:54.439083+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:28:54.439083+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11706392, 'num_objects': 2859, 'num_object_clones': 0, 'num_object_copies': 5718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2057, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2859, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16347, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2211, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 802}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:59.383 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:28:59.383 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17772", 'reported_seq': 17914, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:53.689023+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:28:53.689023+0000', 'last_peered': '2024-09-15T21:28:53.689023+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:28:53.689023+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11727331, 'num_objects': 2864, 'num_object_clones': 0, 'num_object_copies': 5728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2864, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14448, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1212}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:59.383 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:28:59.383 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'17867", 'reported_seq': 18009, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:53.801344+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:28:53.801344+0000', 'last_peered': '2024-09-15T21:28:53.801344+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:28:53.801344+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11482054, 'num_objects': 2805, 'num_object_clones': 0, 'num_object_copies': 5610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2805, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14178, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1356}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:59.383 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:28:59.383 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'17754", 'reported_seq': 17887, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:54.379878+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:28:54.379878+0000', 'last_peered': '2024-09-15T21:28:54.379878+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:28:54.379878+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11301347, 'num_objects': 2760, 'num_object_clones': 0, 'num_object_copies': 5520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2760, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15686, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1327}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:59.384 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:28:59.384 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17583", 'reported_seq': 17720, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:53.731038+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:28:53.731038+0000', 'last_peered': '2024-09-15T21:28:53.731038+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:28:53.731038+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11874304, 'num_objects': 2899, 'num_object_clones': 0, 'num_object_copies': 5798, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2899, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14256, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1121}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:59.384 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:28:59.384 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17836", 'reported_seq': 17960, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:54.330674+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:28:54.330674+0000', 'last_peered': '2024-09-15T21:28:54.330674+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:28:54.330674+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11497955, 'num_objects': 2808, 'num_object_clones': 0, 'num_object_copies': 5616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2808, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15695, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1282}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:28:59.384 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:28:59.384 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'17864", 'reported_seq': 18003, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:53.771282+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:28:53.771282+0000', 'last_peered': '2024-09-15T21:28:53.771282+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:28:53.771282+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11558912, 'num_objects': 2822, 'num_object_clones': 0, 'num_object_copies': 5644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14808, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1272}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:59.384 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:28:59.384 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'17784", 'reported_seq': 17922, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:53.867714+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:28:53.867714+0000', 'last_peered': '2024-09-15T21:28:53.867714+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:28:53.867714+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11493376, 'num_objects': 2806, 'num_object_clones': 0, 'num_object_copies': 5612, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2806, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14693, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1333}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:59.384 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:28:59.385 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'17924", 'reported_seq': 18069, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:53.823503+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:28:53.823503+0000', 'last_peered': '2024-09-15T21:28:53.823503+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:28:53.823503+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12051398, 'num_objects': 2944, 'num_object_clones': 0, 'num_object_copies': 5888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2944, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14628, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1203}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:28:59.385 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:00.078 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:00.079 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39014/90306 objects degraded (43.202%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":591,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":45153,"data_bytes":184898313,"bytes_used":1857060864,"bytes_avail":288053231616,"bytes_total":289910292480,"degraded_objects":39014,"degraded_total":90306,"degraded_ratio":0.43202002081810731,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":396},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:00.107 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:00.107 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:00.646 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:00.648 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:00.670 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":335,"stamp":"2024-09-15T21:28:58.847209+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":184898313,"num_objects":45153,"num_object_clones":0,"num_object_copies":90306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45153,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":245306,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10461,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160914,"ondisk_log_size":160914,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1813536,"kb_used_data":560668,"kb_used_omap":12,"kb_used_meta":1252787,"kb_avail":281301984,"statfs":{"total":289910292480,"available":288053231616,"internally_reserved":0,"allocated":574124032,"data_stored":573235272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1282853891},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,3,4,11,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19517440,"num_objects":-4765,"num_object_clones":0,"num_object_copies":-9530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4765,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004699"},"pg_stats":[{"pgid":"2.f","version":"47'18246","reported_seq":18326,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.823108+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:53.823108+0000","last_peered":"2024-09-15T21:28:53.823108+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:53.823108+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11902976,"num_objects":2906,"num_object_clones":0,"num_object_copies":5812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18246,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1226}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'17847","reported_seq":17958,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:54.357928+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:54.357928+0000","last_peered":"2024-09-15T21:28:54.357928+0000","last_clean":"2024-09-15T21:28:54.357928+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:54.357928+0000","last_undegraded":"2024-09-15T21:28:54.357928+0000","last_fullsized":"2024-09-15T21:28:54.357928+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11580358,"num_objects":2829,"num_object_clones":0,"num_object_copies":5658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2829,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16128,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18247","reported_seq":22802,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:54.439083+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:54.439083+0000","last_peered":"2024-09-15T21:28:54.439083+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:54.439083+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11706392,"num_objects":2859,"num_object_clones":0,"num_object_copies":5718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2057,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2859,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16347,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":802}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17549","reported_seq":17625,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:54.386358+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:54.386358+0000","last_peered":"2024-09-15T21:28:54.386358+0000","last_clean":"2024-09-15T21:28:54.386358+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:54.386358+0000","last_undegraded":"2024-09-15T21:28:54.386358+0000","last_fullsized":"2024-09-15T21:28:54.386358+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11325440,"num_objects":2765,"num_object_clones":0,"num_object_copies":5530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17549,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17729","reported_seq":17866,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:53.875008+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:53.875008+0000","last_peered":"2024-09-15T21:28:53.875008+0000","last_clean":"2024-09-15T21:28:53.875008+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:53.875008+0000","last_undegraded":"2024-09-15T21:28:53.875008+0000","last_fullsized":"2024-09-15T21:28:53.875008+0000","mapping_epoch":34,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11370979,"num_objects":2777,"num_object_clones":0,"num_object_copies":5554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14184,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17772","reported_seq":17914,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.689023+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:53.689023+0000","last_peered":"2024-09-15T21:28:53.689023+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:53.689023+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11727331,"num_objects":2864,"num_object_clones":0,"num_object_copies":5728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2864,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14448,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1212},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'17867","reported_seq":18009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.801344+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:53.801344+0000","last_peered":"2024-09-15T21:28:53.801344+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:53.801344+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11482054,"num_objects":2805,"num_object_clones":0,"num_object_copies":5610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14178,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1356},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18077","reported_seq":18212,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:53.770698+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:53.770698+0000","last_peered":"2024-09-15T21:28:53.770698+0000","last_clean":"2024-09-15T21:28:53.770698+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:53.770698+0000","last_undegraded":"2024-09-15T21:28:53.770698+0000","last_fullsized":"2024-09-15T21:28:53.770698+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11431936,"num_objects":2791,"num_object_clones":0,"num_object_copies":5582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14849,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17754","reported_seq":17887,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:54.379878+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:54.379878+0000","last_peered":"2024-09-15T21:28:54.379878+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:54.379878+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11301347,"num_objects":2760,"num_object_clones":0,"num_object_copies":5520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2760,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15686,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1327}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17583","reported_seq":17720,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.731038+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:53.731038+0000","last_peered":"2024-09-15T21:28:53.731038+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:53.731038+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11874304,"num_objects":2899,"num_object_clones":0,"num_object_copies":5798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14256,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1121}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17836","reported_seq":17960,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:54.330674+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:54.330674+0000","last_peered":"2024-09-15T21:28:54.330674+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:54.330674+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11497955,"num_objects":2808,"num_object_clones":0,"num_object_copies":5616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2808,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15695,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1282}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17847","reported_seq":26312,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:54.391239+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:54.391239+0000","last_peered":"2024-09-15T21:28:54.391239+0000","last_clean":"2024-09-15T21:28:54.391239+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:54.391239+0000","last_undegraded":"2024-09-15T21:28:54.391239+0000","last_fullsized":"2024-09-15T21:28:54.391239+0000","mapping_epoch":44,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11112448,"num_objects":2713,"num_object_clones":0,"num_object_copies":5426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15540,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'17864","reported_seq":18003,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.771282+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:53.771282+0000","last_peered":"2024-09-15T21:28:53.771282+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:53.771282+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11558912,"num_objects":2822,"num_object_clones":0,"num_object_copies":5644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14808,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1272}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17784","reported_seq":17922,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.867714+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:53.867714+0000","last_peered":"2024-09-15T21:28:53.867714+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:53.867714+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11493376,"num_objects":2806,"num_object_clones":0,"num_object_copies":5612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14693,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1333}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17585","reported_seq":25900,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:55.105301+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:28:55.105301+0000","last_peered":"2024-09-15T21:28:55.105301+0000","last_clean":"2024-09-15T21:28:55.105301+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:28:55.105301+0000","last_undegraded":"2024-09-15T21:28:55.105301+0000","last_fullsized":"2024-09-15T21:28:55.105301+0000","mapping_epoch":46,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11481088,"num_objects":2803,"num_object_clones":0,"num_object_copies":5606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14069,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'17924","reported_seq":18069,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:53.823503+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:53.823503+0000","last_peered":"2024-09-15T21:28:53.823503+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:53.823503+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12051398,"num_objects":2944,"num_object_clones":0,"num_object_copies":5888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2944,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14628,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1203},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":184898294,"num_objects":45151,"num_object_clones":0,"num_object_copies":90302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39014,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45151,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":245304,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10460,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":571162624,"data_stored":571017186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563928,"kb_used_data":221256,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93807912,"statfs":{"total":96636764160,"available":96059301888,"internally_reserved":0,"allocated":226566144,"data_stored":226280886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":663040,"kb_used_data":203812,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93708800,"statfs":{"total":96636764160,"available":95957811200,"internally_reserved":0,"allocated":208703488,"data_stored":208418225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":586568,"kb_used_data":135600,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93785272,"statfs":{"total":96636764160,"available":96036118528,"internally_reserved":0,"allocated":138854400,"data_stored":138536161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,5,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":207536128,"data_stored":207499998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":137850880,"data_stored":137781774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":225775616,"data_stored":225735414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:03.673 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:04.237 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:04.238 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:04.260 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-15T21:29:02.848662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":175223561,"num_objects":42791,"num_object_clones":0,"num_object_copies":85582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42791,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":247668,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10578,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160676,"ondisk_log_size":160676,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1777872,"kb_used_data":541648,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281337648,"statfs":{"total":289910292480,"available":288089751552,"internally_reserved":0,"allocated":554647552,"data_stored":553758792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1265814531},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,8,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19304448,"num_objects":-4713,"num_object_clones":0,"num_object_copies":-9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004625"},"pg_stats":[{"pgid":"2.f","version":"47'18401","reported_seq":18482,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.849904+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:58.849904+0000","last_peered":"2024-09-15T21:28:58.849904+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:58.849904+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268096,"num_objects":2751,"num_object_clones":0,"num_object_copies":5502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18401,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1381}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18007","reported_seq":18118,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:59.365681+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:59.365681+0000","last_peered":"2024-09-15T21:28:59.365681+0000","last_clean":"2024-09-15T21:28:59.365681+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:59.365681+0000","last_undegraded":"2024-09-15T21:28:59.365681+0000","last_fullsized":"2024-09-15T21:28:59.365681+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10924998,"num_objects":2669,"num_object_clones":0,"num_object_copies":5338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2669,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16288,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18404","reported_seq":23193,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:59.435227+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:59.435227+0000","last_peered":"2024-09-15T21:28:59.435227+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:59.435227+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11063320,"num_objects":2702,"num_object_clones":0,"num_object_copies":5404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1940,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2702,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16504,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2328,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":762}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17675","reported_seq":17751,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:59.363999+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:59.363999+0000","last_peered":"2024-09-15T21:28:59.363999+0000","last_clean":"2024-09-15T21:28:59.363999+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:59.363999+0000","last_undegraded":"2024-09-15T21:28:59.363999+0000","last_fullsized":"2024-09-15T21:28:59.363999+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10809344,"num_objects":2639,"num_object_clones":0,"num_object_copies":5278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17675,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17857","reported_seq":17996,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:58.869995+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:58.869995+0000","last_peered":"2024-09-15T21:28:58.869995+0000","last_clean":"2024-09-15T21:28:58.869995+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:58.869995+0000","last_undegraded":"2024-09-15T21:28:58.869995+0000","last_fullsized":"2024-09-15T21:28:58.869995+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10846691,"num_objects":2649,"num_object_clones":0,"num_object_copies":5298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14312,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17932","reported_seq":18074,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.878189+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:58.878189+0000","last_peered":"2024-09-15T21:28:58.878189+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:58.878189+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11071971,"num_objects":2704,"num_object_clones":0,"num_object_copies":5408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14608,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1372},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18005","reported_seq":18146,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.898517+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:58.898517+0000","last_peered":"2024-09-15T21:28:58.898517+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:58.898517+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10916806,"num_objects":2667,"num_object_clones":0,"num_object_copies":5334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14316,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1494},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18243","reported_seq":18378,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:58.917119+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:58.917119+0000","last_peered":"2024-09-15T21:28:58.917119+0000","last_clean":"2024-09-15T21:28:58.917119+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:58.917119+0000","last_undegraded":"2024-09-15T21:28:58.917119+0000","last_fullsized":"2024-09-15T21:28:58.917119+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10752000,"num_objects":2625,"num_object_clones":0,"num_object_copies":5250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15015,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17892","reported_seq":18025,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:59.153133+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:59.153133+0000","last_peered":"2024-09-15T21:28:59.153133+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:59.153133+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10736099,"num_objects":2622,"num_object_clones":0,"num_object_copies":5244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15824,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1465}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17748","reported_seq":17884,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.894885+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:58.894885+0000","last_peered":"2024-09-15T21:28:58.894885+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:58.894885+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11198464,"num_objects":2734,"num_object_clones":0,"num_object_copies":5468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14421,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1286}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17989","reported_seq":18114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:59.403321+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:59.403321+0000","last_peered":"2024-09-15T21:28:59.403321+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:59.403321+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10871267,"num_objects":2655,"num_object_clones":0,"num_object_copies":5310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2655,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15848,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1435}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17979","reported_seq":26444,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:59.409519+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:59.409519+0000","last_peered":"2024-09-15T21:28:59.409519+0000","last_clean":"2024-09-15T21:28:59.409519+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:59.409519+0000","last_undegraded":"2024-09-15T21:28:59.409519+0000","last_fullsized":"2024-09-15T21:28:59.409519+0000","mapping_epoch":44,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10571776,"num_objects":2581,"num_object_clones":0,"num_object_copies":5162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15672,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18011","reported_seq":18151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.905164+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:58.905164+0000","last_peered":"2024-09-15T21:28:58.905164+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:58.905164+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10956800,"num_objects":2675,"num_object_clones":0,"num_object_copies":5350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14955,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17936","reported_seq":18075,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.882877+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:58.882877+0000","last_peered":"2024-09-15T21:28:58.882877+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:58.882877+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10870784,"num_objects":2654,"num_object_clones":0,"num_object_copies":5308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14845,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1485}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17717","reported_seq":26032,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:00.085877+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:00.085877+0000","last_peered":"2024-09-15T21:29:00.085877+0000","last_clean":"2024-09-15T21:29:00.085877+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:00.085877+0000","last_undegraded":"2024-09-15T21:29:00.085877+0000","last_fullsized":"2024-09-15T21:29:00.085877+0000","mapping_epoch":46,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10940416,"num_objects":2671,"num_object_clones":0,"num_object_copies":5342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14201,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18077","reported_seq":18221,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.890206+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:58.890206+0000","last_peered":"2024-09-15T21:28:58.890206+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:58.890206+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11424710,"num_objects":2791,"num_object_clones":0,"num_object_copies":5582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2791,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14781,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1356},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":175223542,"num_objects":42789,"num_object_clones":0,"num_object_copies":85578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42789,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":247666,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10577,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":551550976,"data_stored":551405538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160673,"ondisk_log_size":160673,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563284,"kb_used_data":216436,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93808556,"statfs":{"total":96636764160,"available":96059961344,"internally_reserved":0,"allocated":221630464,"data_stored":221345206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643496,"kb_used_data":196760,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93728344,"statfs":{"total":96636764160,"available":95977824256,"internally_reserved":0,"allocated":201482240,"data_stored":201196977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457436843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,6,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571092,"kb_used_data":128452,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93800748,"statfs":{"total":96636764160,"available":96051965952,"internally_reserved":0,"allocated":131534848,"data_stored":131216609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":200146944,"data_stored":200110814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":130379776,"data_stored":130310670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":221024256,"data_stored":220984054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:04.262 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:04.819 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:04.819 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:04.850 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-15T21:29:02.848662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":175223561,"num_objects":42791,"num_object_clones":0,"num_object_copies":85582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42791,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":247668,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10578,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160676,"ondisk_log_size":160676,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1777872,"kb_used_data":541648,"kb_used_omap":12,"kb_used_meta":1236147,"kb_avail":281337648,"statfs":{"total":289910292480,"available":288089751552,"internally_reserved":0,"allocated":554647552,"data_stored":553758792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1265814531},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,4,8,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19304448,"num_objects":-4713,"num_object_clones":0,"num_object_copies":-9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-234,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":234,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004625"},"pg_stats":[{"pgid":"2.f","version":"47'18401","reported_seq":18482,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.849904+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:28:58.849904+0000","last_peered":"2024-09-15T21:28:58.849904+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:28:58.849904+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11268096,"num_objects":2751,"num_object_clones":0,"num_object_copies":5502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18401,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1381}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18007","reported_seq":18118,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:59.365681+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:28:59.365681+0000","last_peered":"2024-09-15T21:28:59.365681+0000","last_clean":"2024-09-15T21:28:59.365681+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:28:59.365681+0000","last_undegraded":"2024-09-15T21:28:59.365681+0000","last_fullsized":"2024-09-15T21:28:59.365681+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10924998,"num_objects":2669,"num_object_clones":0,"num_object_copies":5338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2669,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16288,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18404","reported_seq":23193,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:59.435227+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:28:59.435227+0000","last_peered":"2024-09-15T21:28:59.435227+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:28:59.435227+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11063320,"num_objects":2702,"num_object_clones":0,"num_object_copies":5404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1940,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2702,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16504,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2328,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":762}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17675","reported_seq":17751,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:59.363999+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:28:59.363999+0000","last_peered":"2024-09-15T21:28:59.363999+0000","last_clean":"2024-09-15T21:28:59.363999+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:28:59.363999+0000","last_undegraded":"2024-09-15T21:28:59.363999+0000","last_fullsized":"2024-09-15T21:28:59.363999+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10809344,"num_objects":2639,"num_object_clones":0,"num_object_copies":5278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17675,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'17857","reported_seq":17996,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:58.869995+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:28:58.869995+0000","last_peered":"2024-09-15T21:28:58.869995+0000","last_clean":"2024-09-15T21:28:58.869995+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:28:58.869995+0000","last_undegraded":"2024-09-15T21:28:58.869995+0000","last_fullsized":"2024-09-15T21:28:58.869995+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10846691,"num_objects":2649,"num_object_clones":0,"num_object_copies":5298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14312,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'17932","reported_seq":18074,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.878189+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:28:58.878189+0000","last_peered":"2024-09-15T21:28:58.878189+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:28:58.878189+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11071971,"num_objects":2704,"num_object_clones":0,"num_object_copies":5408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2704,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14608,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1372},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18005","reported_seq":18146,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.898517+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:28:58.898517+0000","last_peered":"2024-09-15T21:28:58.898517+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:28:58.898517+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10916806,"num_objects":2667,"num_object_clones":0,"num_object_copies":5334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14316,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1494},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18243","reported_seq":18378,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:58.917119+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:28:58.917119+0000","last_peered":"2024-09-15T21:28:58.917119+0000","last_clean":"2024-09-15T21:28:58.917119+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:28:58.917119+0000","last_undegraded":"2024-09-15T21:28:58.917119+0000","last_fullsized":"2024-09-15T21:28:58.917119+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10752000,"num_objects":2625,"num_object_clones":0,"num_object_copies":5250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15015,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'17892","reported_seq":18025,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:59.153133+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:28:59.153133+0000","last_peered":"2024-09-15T21:28:59.153133+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:28:59.153133+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10736099,"num_objects":2622,"num_object_clones":0,"num_object_copies":5244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15824,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1465}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17748","reported_seq":17884,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.894885+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:28:58.894885+0000","last_peered":"2024-09-15T21:28:58.894885+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:28:58.894885+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11198464,"num_objects":2734,"num_object_clones":0,"num_object_copies":5468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2734,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14421,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1286}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'17989","reported_seq":18114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:59.403321+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:28:59.403321+0000","last_peered":"2024-09-15T21:28:59.403321+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:28:59.403321+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10871267,"num_objects":2655,"num_object_clones":0,"num_object_copies":5310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2655,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15848,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1435}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'17979","reported_seq":26444,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:28:59.409519+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:28:59.409519+0000","last_peered":"2024-09-15T21:28:59.409519+0000","last_clean":"2024-09-15T21:28:59.409519+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:28:59.409519+0000","last_undegraded":"2024-09-15T21:28:59.409519+0000","last_fullsized":"2024-09-15T21:28:59.409519+0000","mapping_epoch":44,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10571776,"num_objects":2581,"num_object_clones":0,"num_object_copies":5162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15672,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18011","reported_seq":18151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.905164+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:28:58.905164+0000","last_peered":"2024-09-15T21:28:58.905164+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:28:58.905164+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10956800,"num_objects":2675,"num_object_clones":0,"num_object_copies":5350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14955,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1419}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'17936","reported_seq":18075,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.882877+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:28:58.882877+0000","last_peered":"2024-09-15T21:28:58.882877+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:28:58.882877+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10870784,"num_objects":2654,"num_object_clones":0,"num_object_copies":5308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14845,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1485}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17717","reported_seq":26032,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:00.085877+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:00.085877+0000","last_peered":"2024-09-15T21:29:00.085877+0000","last_clean":"2024-09-15T21:29:00.085877+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:00.085877+0000","last_undegraded":"2024-09-15T21:29:00.085877+0000","last_fullsized":"2024-09-15T21:29:00.085877+0000","mapping_epoch":46,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10940416,"num_objects":2671,"num_object_clones":0,"num_object_copies":5342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14201,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18077","reported_seq":18221,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:28:58.890206+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:28:58.890206+0000","last_peered":"2024-09-15T21:28:58.890206+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:28:58.890206+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11424710,"num_objects":2791,"num_object_clones":0,"num_object_copies":5582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2791,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14781,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1356},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":175223542,"num_objects":42789,"num_object_clones":0,"num_object_copies":85578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42789,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":247666,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10577,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":551550976,"data_stored":551405538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160673,"ondisk_log_size":160673,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":563284,"kb_used_data":216436,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93808556,"statfs":{"total":96636764160,"available":96059961344,"internally_reserved":0,"allocated":221630464,"data_stored":221345206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643496,"kb_used_data":196760,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93728344,"statfs":{"total":96636764160,"available":95977824256,"internally_reserved":0,"allocated":201482240,"data_stored":201196977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457436843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,6,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571092,"kb_used_data":128452,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93800748,"statfs":{"total":96636764160,"available":96051965952,"internally_reserved":0,"allocated":131534848,"data_stored":131216609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":200146944,"data_stored":200110814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":130379776,"data_stored":130310670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":221024256,"data_stored":220984054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:04.852 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:29:04.852 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18401", 'reported_seq': 18482, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:58.849904+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:28:58.849904+0000', 'last_peered': '2024-09-15T21:28:58.849904+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:28:58.849904+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11268096, 'num_objects': 2751, 'num_object_clones': 0, 'num_object_copies': 5502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2751, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18401, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1381}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:04.852 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:29:04.852 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'18404", 'reported_seq': 23193, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:59.435227+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:28:59.435227+0000', 'last_peered': '2024-09-15T21:28:59.435227+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:28:59.435227+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11063320, 'num_objects': 2702, 'num_object_clones': 0, 'num_object_copies': 5404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1940, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2702, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16504, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2328, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 762}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:04.852 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:29:04.852 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'17932", 'reported_seq': 18074, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:58.878189+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:28:58.878189+0000', 'last_peered': '2024-09-15T21:28:58.878189+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:28:58.878189+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11071971, 'num_objects': 2704, 'num_object_clones': 0, 'num_object_copies': 5408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2704, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14608, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1372}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:04.853 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:04.853 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18005", 'reported_seq': 18146, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:58.898517+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:28:58.898517+0000', 'last_peered': '2024-09-15T21:28:58.898517+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:28:58.898517+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10916806, 'num_objects': 2667, 'num_object_clones': 0, 'num_object_copies': 5334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2667, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14316, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1494}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:04.853 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:29:04.853 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'17892", 'reported_seq': 18025, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:59.153133+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:28:59.153133+0000', 'last_peered': '2024-09-15T21:28:59.153133+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:28:59.153133+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10736099, 'num_objects': 2622, 'num_object_clones': 0, 'num_object_copies': 5244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2622, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15824, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1465}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:04.853 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:04.853 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17748", 'reported_seq': 17884, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:58.894885+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:28:58.894885+0000', 'last_peered': '2024-09-15T21:28:58.894885+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:28:58.894885+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11198464, 'num_objects': 2734, 'num_object_clones': 0, 'num_object_copies': 5468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2734, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14421, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1286}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:04.853 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:29:04.853 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'17989", 'reported_seq': 18114, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:59.403321+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:28:59.403321+0000', 'last_peered': '2024-09-15T21:28:59.403321+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:28:59.403321+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10871267, 'num_objects': 2655, 'num_object_clones': 0, 'num_object_copies': 5310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2655, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15848, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1435}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:04.854 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:29:04.854 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18011", 'reported_seq': 18151, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:58.905164+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:28:58.905164+0000', 'last_peered': '2024-09-15T21:28:58.905164+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:28:58.905164+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10956800, 'num_objects': 2675, 'num_object_clones': 0, 'num_object_copies': 5350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2675, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14955, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1419}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:04.854 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:29:04.854 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'17936", 'reported_seq': 18075, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:58.882877+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:28:58.882877+0000', 'last_peered': '2024-09-15T21:28:58.882877+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:28:58.882877+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10870784, 'num_objects': 2654, 'num_object_clones': 0, 'num_object_copies': 5308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2654, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14845, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1485}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:04.854 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:29:04.854 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'18077", 'reported_seq': 18221, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:28:58.890206+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:28:58.890206+0000', 'last_peered': '2024-09-15T21:28:58.890206+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:28:58.890206+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11424710, 'num_objects': 2791, 'num_object_clones': 0, 'num_object_copies': 5582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2791, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14781, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1356}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:04.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:05.549 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:05.549 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38767/81080 objects degraded (47.813%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":597,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":40540,"data_bytes":166003465,"bytes_used":1800155136,"bytes_avail":288110137344,"bytes_total":289910292480,"degraded_objects":38767,"degraded_total":81080,"degraded_ratio":0.4781327084361125,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":364,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":580},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:05.577 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:05.578 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:06.142 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:06.168 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-15T21:29:04.849085+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":166003465,"num_objects":40540,"num_object_clones":0,"num_object_copies":81080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40540,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":249919,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10708,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1757964,"kb_used_data":530068,"kb_used_omap":12,"kb_used_meta":1227827,"kb_avail":281357556,"statfs":{"total":289910292480,"available":288110137344,"internally_reserved":0,"allocated":542789632,"data_stored":541900872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1257294851},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,6,6,2,7,7,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28524544,"num_objects":-6964,"num_object_clones":0,"num_object_copies":-13928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6964,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004165"},"pg_stats":[{"pgid":"2.f","version":"47'18562","reported_seq":18642,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.941502+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:03.941502+0000","last_peered":"2024-09-15T21:29:03.941502+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:03.941502+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10608640,"num_objects":2590,"num_object_clones":0,"num_object_copies":5180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18562,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1542}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18163","reported_seq":18272,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:04.367015+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:04.367015+0000","last_peered":"2024-09-15T21:29:04.367015+0000","last_clean":"2024-09-15T21:29:04.367015+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:04.367015+0000","last_undegraded":"2024-09-15T21:29:04.367015+0000","last_fullsized":"2024-09-15T21:29:04.367015+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10286022,"num_objects":2513,"num_object_clones":0,"num_object_copies":5026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2513,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16444,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18561","reported_seq":23610,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:04.438265+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:04.438265+0000","last_peered":"2024-09-15T21:29:04.438265+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:04.438265+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10420248,"num_objects":2545,"num_object_clones":0,"num_object_copies":5090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1810,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16661,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":735}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17811","reported_seq":17887,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:04.444662+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:04.444662+0000","last_peered":"2024-09-15T21:29:04.444662+0000","last_clean":"2024-09-15T21:29:04.444662+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:04.444662+0000","last_undegraded":"2024-09-15T21:29:04.444662+0000","last_fullsized":"2024-09-15T21:29:04.444662+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10252288,"num_objects":2503,"num_object_clones":0,"num_object_copies":5006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17811,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18008","reported_seq":18147,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:03.841150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:03.841150+0000","last_peered":"2024-09-15T21:29:03.841150+0000","last_clean":"2024-09-15T21:29:03.841150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:03.841150+0000","last_undegraded":"2024-09-15T21:29:03.841150+0000","last_fullsized":"2024-09-15T21:29:03.841150+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10228195,"num_objects":2498,"num_object_clones":0,"num_object_copies":4996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14463,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18092","reported_seq":18229,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.900924+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:03.900924+0000","last_peered":"2024-09-15T21:29:03.900924+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:03.900924+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10416611,"num_objects":2544,"num_object_clones":0,"num_object_copies":5088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14768,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1532},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18155","reported_seq":18297,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.856286+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:03.856286+0000","last_peered":"2024-09-15T21:29:03.856286+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:03.856286+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10302406,"num_objects":2517,"num_object_clones":0,"num_object_copies":5034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14466,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1644},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18378","reported_seq":18513,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:03.866024+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:03.866024+0000","last_peered":"2024-09-15T21:29:03.866024+0000","last_clean":"2024-09-15T21:29:03.866024+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:03.866024+0000","last_undegraded":"2024-09-15T21:29:03.866024+0000","last_fullsized":"2024-09-15T21:29:03.866024+0000","mapping_epoch":34,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10199040,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15150,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18047","reported_seq":18180,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:04.405799+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:04.405799+0000","last_peered":"2024-09-15T21:29:04.405799+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:04.405799+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10101219,"num_objects":2467,"num_object_clones":0,"num_object_copies":4934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2467,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15979,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1620}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17877","reported_seq":18012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.789641+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:03.789641+0000","last_peered":"2024-09-15T21:29:03.789641+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:03.789641+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10670080,"num_objects":2605,"num_object_clones":0,"num_object_copies":5210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14550,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1415}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18136","reported_seq":18259,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:04.423767+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:04.423767+0000","last_peered":"2024-09-15T21:29:04.423767+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:04.423767+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10269155,"num_objects":2508,"num_object_clones":0,"num_object_copies":5016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15995,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1582}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18134","reported_seq":26599,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:04.419804+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:04.419804+0000","last_peered":"2024-09-15T21:29:04.419804+0000","last_clean":"2024-09-15T21:29:04.419804+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:04.419804+0000","last_undegraded":"2024-09-15T21:29:04.419804+0000","last_fullsized":"2024-09-15T21:29:04.419804+0000","mapping_epoch":44,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9936896,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15827,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18178","reported_seq":18318,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.918321+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:03.918321+0000","last_peered":"2024-09-15T21:29:03.918321+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:03.918321+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10272768,"num_objects":2508,"num_object_clones":0,"num_object_copies":5016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15122,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18078","reported_seq":18217,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.800705+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:03.800705+0000","last_peered":"2024-09-15T21:29:03.800705+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:03.800705+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10289152,"num_objects":2512,"num_object_clones":0,"num_object_copies":5024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14987,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1627}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17717","reported_seq":26032,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:00.085877+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:00.085877+0000","last_peered":"2024-09-15T21:29:00.085877+0000","last_clean":"2024-09-15T21:29:00.085877+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:00.085877+0000","last_undegraded":"2024-09-15T21:29:00.085877+0000","last_fullsized":"2024-09-15T21:29:00.085877+0000","mapping_epoch":46,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10940416,"num_objects":2671,"num_object_clones":0,"num_object_copies":5342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14201,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18227","reported_seq":18369,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.891255+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:03.891255+0000","last_peered":"2024-09-15T21:29:03.891255+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:03.891255+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10810310,"num_objects":2641,"num_object_clones":0,"num_object_copies":5282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2641,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14931,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1506},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":166003446,"num_objects":40538,"num_object_clones":0,"num_object_copies":81076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40538,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":249917,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10707,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":539516928,"data_stored":539371490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160824,"ondisk_log_size":160824,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546040,"kb_used_data":211688,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93825800,"statfs":{"total":96636764160,"available":96077619200,"internally_reserved":0,"allocated":216768512,"data_stored":216483254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,5,2,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640832,"kb_used_data":189928,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93731008,"statfs":{"total":96636764160,"available":95980552192,"internally_reserved":0,"allocated":194486272,"data_stored":194201009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,3,2,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":571092,"kb_used_data":128452,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93800748,"statfs":{"total":96636764160,"available":96051965952,"internally_reserved":0,"allocated":131534848,"data_stored":131216609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":192970752,"data_stored":192934622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":130379776,"data_stored":130310670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":216166400,"data_stored":216126198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:09.170 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:09.730 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:09.733 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:09.754 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-15T21:29:08.850240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":165405449,"num_objects":40394,"num_object_clones":0,"num_object_copies":80788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40394,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":250065,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10708,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160873,"ondisk_log_size":160873,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1754964,"kb_used_data":522900,"kb_used_omap":12,"kb_used_meta":1231987,"kb_avail":281360556,"statfs":{"total":289910292480,"available":288113209344,"internally_reserved":0,"allocated":535449600,"data_stored":534560840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1261554691},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,6,5,3,7,8,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19492864,"num_objects":-4759,"num_object_clones":0,"num_object_copies":-9518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4759,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003802"},"pg_stats":[{"pgid":"2.f","version":"47'18562","reported_seq":18642,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.941502+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:03.941502+0000","last_peered":"2024-09-15T21:29:03.941502+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:03.941502+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10608640,"num_objects":2590,"num_object_clones":0,"num_object_copies":5180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18562,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1542}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18163","reported_seq":18272,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:04.367015+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:04.367015+0000","last_peered":"2024-09-15T21:29:04.367015+0000","last_clean":"2024-09-15T21:29:04.367015+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:04.367015+0000","last_undegraded":"2024-09-15T21:29:04.367015+0000","last_fullsized":"2024-09-15T21:29:04.367015+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10286022,"num_objects":2513,"num_object_clones":0,"num_object_copies":5026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2513,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16444,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18561","reported_seq":23610,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:04.438265+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:04.438265+0000","last_peered":"2024-09-15T21:29:04.438265+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:04.438265+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10420248,"num_objects":2545,"num_object_clones":0,"num_object_copies":5090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1810,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16661,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":735}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17811","reported_seq":17887,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:04.444662+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:04.444662+0000","last_peered":"2024-09-15T21:29:04.444662+0000","last_clean":"2024-09-15T21:29:04.444662+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:04.444662+0000","last_undegraded":"2024-09-15T21:29:04.444662+0000","last_fullsized":"2024-09-15T21:29:04.444662+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10252288,"num_objects":2503,"num_object_clones":0,"num_object_copies":5006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17811,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18008","reported_seq":18147,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:03.841150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:03.841150+0000","last_peered":"2024-09-15T21:29:03.841150+0000","last_clean":"2024-09-15T21:29:03.841150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:03.841150+0000","last_undegraded":"2024-09-15T21:29:03.841150+0000","last_fullsized":"2024-09-15T21:29:03.841150+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10228195,"num_objects":2498,"num_object_clones":0,"num_object_copies":4996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14463,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18092","reported_seq":18229,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.900924+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:03.900924+0000","last_peered":"2024-09-15T21:29:03.900924+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:03.900924+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10416611,"num_objects":2544,"num_object_clones":0,"num_object_copies":5088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14768,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1532},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18155","reported_seq":18297,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.856286+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:03.856286+0000","last_peered":"2024-09-15T21:29:03.856286+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:03.856286+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10302406,"num_objects":2517,"num_object_clones":0,"num_object_copies":5034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14466,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1644},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18378","reported_seq":18513,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:03.866024+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:03.866024+0000","last_peered":"2024-09-15T21:29:03.866024+0000","last_clean":"2024-09-15T21:29:03.866024+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:03.866024+0000","last_undegraded":"2024-09-15T21:29:03.866024+0000","last_fullsized":"2024-09-15T21:29:03.866024+0000","mapping_epoch":34,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10199040,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15150,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18047","reported_seq":18180,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:04.405799+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:04.405799+0000","last_peered":"2024-09-15T21:29:04.405799+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:04.405799+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10101219,"num_objects":2467,"num_object_clones":0,"num_object_copies":4934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2467,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15979,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1620}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17877","reported_seq":18012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.789641+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:03.789641+0000","last_peered":"2024-09-15T21:29:03.789641+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:03.789641+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10670080,"num_objects":2605,"num_object_clones":0,"num_object_copies":5210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14550,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1415}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18136","reported_seq":18259,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:04.423767+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:04.423767+0000","last_peered":"2024-09-15T21:29:04.423767+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:04.423767+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10269155,"num_objects":2508,"num_object_clones":0,"num_object_copies":5016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15995,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1582}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18134","reported_seq":26599,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:04.419804+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:04.419804+0000","last_peered":"2024-09-15T21:29:04.419804+0000","last_clean":"2024-09-15T21:29:04.419804+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:04.419804+0000","last_undegraded":"2024-09-15T21:29:04.419804+0000","last_fullsized":"2024-09-15T21:29:04.419804+0000","mapping_epoch":44,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9936896,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15827,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18178","reported_seq":18318,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.918321+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:03.918321+0000","last_peered":"2024-09-15T21:29:03.918321+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:03.918321+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10272768,"num_objects":2508,"num_object_clones":0,"num_object_copies":5016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15122,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18078","reported_seq":18217,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.800705+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:03.800705+0000","last_peered":"2024-09-15T21:29:03.800705+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:03.800705+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10289152,"num_objects":2512,"num_object_clones":0,"num_object_copies":5024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14987,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1627}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17863","reported_seq":26178,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:05.347185+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:05.347185+0000","last_peered":"2024-09-15T21:29:05.347185+0000","last_clean":"2024-09-15T21:29:05.347185+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:05.347185+0000","last_undegraded":"2024-09-15T21:29:05.347185+0000","last_fullsized":"2024-09-15T21:29:05.347185+0000","mapping_epoch":46,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10342400,"num_objects":2525,"num_object_clones":0,"num_object_copies":5050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14347,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18227","reported_seq":18369,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.891255+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:03.891255+0000","last_peered":"2024-09-15T21:29:03.891255+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:03.891255+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10810310,"num_objects":2641,"num_object_clones":0,"num_object_copies":5282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2641,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14931,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1506},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":165405430,"num_objects":40392,"num_object_clones":0,"num_object_copies":80784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40392,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":250063,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10707,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532074496,"data_stored":531929058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160870,"ondisk_log_size":160870,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546040,"kb_used_data":211688,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93825800,"statfs":{"total":96636764160,"available":96077619200,"internally_reserved":0,"allocated":216768512,"data_stored":216483254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,5,2,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640832,"kb_used_data":189928,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93731008,"statfs":{"total":96636764160,"available":95980552192,"internally_reserved":0,"allocated":194486272,"data_stored":194201009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,3,2,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568092,"kb_used_data":121284,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93803748,"statfs":{"total":96636764160,"available":96055037952,"internally_reserved":0,"allocated":124194816,"data_stored":123876577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":192970752,"data_stored":192934622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122937344,"data_stored":122868238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":216166400,"data_stored":216126198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:09.756 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:10.332 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:10.334 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:10.355 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-15T21:29:08.850240+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":165405449,"num_objects":40394,"num_object_clones":0,"num_object_copies":80788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40394,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":250065,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10708,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160873,"ondisk_log_size":160873,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1754964,"kb_used_data":522900,"kb_used_omap":12,"kb_used_meta":1231987,"kb_avail":281360556,"statfs":{"total":289910292480,"available":288113209344,"internally_reserved":0,"allocated":535449600,"data_stored":534560840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1261554691},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,6,5,3,7,8,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19492864,"num_objects":-4759,"num_object_clones":0,"num_object_copies":-9518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4759,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003802"},"pg_stats":[{"pgid":"2.f","version":"47'18562","reported_seq":18642,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.941502+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:03.941502+0000","last_peered":"2024-09-15T21:29:03.941502+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:03.941502+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10608640,"num_objects":2590,"num_object_clones":0,"num_object_copies":5180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18562,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1542}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18163","reported_seq":18272,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:04.367015+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:04.367015+0000","last_peered":"2024-09-15T21:29:04.367015+0000","last_clean":"2024-09-15T21:29:04.367015+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:04.367015+0000","last_undegraded":"2024-09-15T21:29:04.367015+0000","last_fullsized":"2024-09-15T21:29:04.367015+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10286022,"num_objects":2513,"num_object_clones":0,"num_object_copies":5026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2513,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16444,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18561","reported_seq":23610,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:04.438265+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:04.438265+0000","last_peered":"2024-09-15T21:29:04.438265+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:04.438265+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10420248,"num_objects":2545,"num_object_clones":0,"num_object_copies":5090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1810,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16661,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":735}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17811","reported_seq":17887,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:04.444662+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:04.444662+0000","last_peered":"2024-09-15T21:29:04.444662+0000","last_clean":"2024-09-15T21:29:04.444662+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:04.444662+0000","last_undegraded":"2024-09-15T21:29:04.444662+0000","last_fullsized":"2024-09-15T21:29:04.444662+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10252288,"num_objects":2503,"num_object_clones":0,"num_object_copies":5006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2503,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17811,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18008","reported_seq":18147,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:03.841150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:03.841150+0000","last_peered":"2024-09-15T21:29:03.841150+0000","last_clean":"2024-09-15T21:29:03.841150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:03.841150+0000","last_undegraded":"2024-09-15T21:29:03.841150+0000","last_fullsized":"2024-09-15T21:29:03.841150+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10228195,"num_objects":2498,"num_object_clones":0,"num_object_copies":4996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14463,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18092","reported_seq":18229,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.900924+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:03.900924+0000","last_peered":"2024-09-15T21:29:03.900924+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:03.900924+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10416611,"num_objects":2544,"num_object_clones":0,"num_object_copies":5088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14768,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1532},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18155","reported_seq":18297,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.856286+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:03.856286+0000","last_peered":"2024-09-15T21:29:03.856286+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:03.856286+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10302406,"num_objects":2517,"num_object_clones":0,"num_object_copies":5034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14466,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1644},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18378","reported_seq":18513,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:03.866024+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:03.866024+0000","last_peered":"2024-09-15T21:29:03.866024+0000","last_clean":"2024-09-15T21:29:03.866024+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:03.866024+0000","last_undegraded":"2024-09-15T21:29:03.866024+0000","last_fullsized":"2024-09-15T21:29:03.866024+0000","mapping_epoch":34,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10199040,"num_objects":2490,"num_object_clones":0,"num_object_copies":4980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15150,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18047","reported_seq":18180,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:04.405799+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:04.405799+0000","last_peered":"2024-09-15T21:29:04.405799+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:04.405799+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10101219,"num_objects":2467,"num_object_clones":0,"num_object_copies":4934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2467,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15979,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1620}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'17877","reported_seq":18012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.789641+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:03.789641+0000","last_peered":"2024-09-15T21:29:03.789641+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:03.789641+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10670080,"num_objects":2605,"num_object_clones":0,"num_object_copies":5210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14550,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1415}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18136","reported_seq":18259,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:04.423767+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:04.423767+0000","last_peered":"2024-09-15T21:29:04.423767+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:04.423767+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10269155,"num_objects":2508,"num_object_clones":0,"num_object_copies":5016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15995,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1582}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18134","reported_seq":26599,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:04.419804+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:04.419804+0000","last_peered":"2024-09-15T21:29:04.419804+0000","last_clean":"2024-09-15T21:29:04.419804+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:04.419804+0000","last_undegraded":"2024-09-15T21:29:04.419804+0000","last_fullsized":"2024-09-15T21:29:04.419804+0000","mapping_epoch":44,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9936896,"num_objects":2426,"num_object_clones":0,"num_object_copies":4852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15827,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18178","reported_seq":18318,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.918321+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:03.918321+0000","last_peered":"2024-09-15T21:29:03.918321+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:03.918321+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10272768,"num_objects":2508,"num_object_clones":0,"num_object_copies":5016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15122,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1586}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18078","reported_seq":18217,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.800705+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:03.800705+0000","last_peered":"2024-09-15T21:29:03.800705+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:03.800705+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10289152,"num_objects":2512,"num_object_clones":0,"num_object_copies":5024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14987,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1627}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'17863","reported_seq":26178,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:05.347185+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:05.347185+0000","last_peered":"2024-09-15T21:29:05.347185+0000","last_clean":"2024-09-15T21:29:05.347185+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:05.347185+0000","last_undegraded":"2024-09-15T21:29:05.347185+0000","last_fullsized":"2024-09-15T21:29:05.347185+0000","mapping_epoch":46,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10342400,"num_objects":2525,"num_object_clones":0,"num_object_copies":5050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14347,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18227","reported_seq":18369,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:03.891255+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:03.891255+0000","last_peered":"2024-09-15T21:29:03.891255+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:03.891255+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10810310,"num_objects":2641,"num_object_clones":0,"num_object_copies":5282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2641,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14931,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1506},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":165405430,"num_objects":40392,"num_object_clones":0,"num_object_copies":80784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38767,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40392,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":250063,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10707,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":532074496,"data_stored":531929058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160870,"ondisk_log_size":160870,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546040,"kb_used_data":211688,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93825800,"statfs":{"total":96636764160,"available":96077619200,"internally_reserved":0,"allocated":216768512,"data_stored":216483254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,5,2,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":640832,"kb_used_data":189928,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93731008,"statfs":{"total":96636764160,"available":95980552192,"internally_reserved":0,"allocated":194486272,"data_stored":194201009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,3,2,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568092,"kb_used_data":121284,"kb_used_omap":4,"kb_used_meta":446779,"kb_avail":93803748,"statfs":{"total":96636764160,"available":96055037952,"internally_reserved":0,"allocated":124194816,"data_stored":123876577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457502383},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":192970752,"data_stored":192934622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122937344,"data_stored":122868238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":216166400,"data_stored":216126198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:10.356 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:29:10.356 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18562", 'reported_seq': 18642, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:03.941502+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:29:03.941502+0000', 'last_peered': '2024-09-15T21:29:03.941502+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:29:03.941502+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10608640, 'num_objects': 2590, 'num_object_clones': 0, 'num_object_copies': 5180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2590, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18562, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1542}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:10.357 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:29:10.357 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'18561", 'reported_seq': 23610, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:04.438265+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:29:04.438265+0000', 'last_peered': '2024-09-15T21:29:04.438265+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:29:04.438265+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10420248, 'num_objects': 2545, 'num_object_clones': 0, 'num_object_copies': 5090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1810, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2545, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16661, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2458, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 735}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:10.357 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:29:10.357 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18092", 'reported_seq': 18229, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:03.900924+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:29:03.900924+0000', 'last_peered': '2024-09-15T21:29:03.900924+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:29:03.900924+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10416611, 'num_objects': 2544, 'num_object_clones': 0, 'num_object_copies': 5088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2544, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14768, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1532}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:10.357 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:10.357 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18155", 'reported_seq': 18297, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:03.856286+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:29:03.856286+0000', 'last_peered': '2024-09-15T21:29:03.856286+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:29:03.856286+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10302406, 'num_objects': 2517, 'num_object_clones': 0, 'num_object_copies': 5034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2517, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14466, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1644}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:10.357 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:29:10.357 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'18047", 'reported_seq': 18180, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:04.405799+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:29:04.405799+0000', 'last_peered': '2024-09-15T21:29:04.405799+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:29:04.405799+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10101219, 'num_objects': 2467, 'num_object_clones': 0, 'num_object_copies': 4934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2467, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15979, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1620}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:10.357 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:10.357 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'17877", 'reported_seq': 18012, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:03.789641+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:29:03.789641+0000', 'last_peered': '2024-09-15T21:29:03.789641+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:29:03.789641+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10670080, 'num_objects': 2605, 'num_object_clones': 0, 'num_object_copies': 5210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2605, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14550, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1415}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:10.358 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:29:10.358 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18136", 'reported_seq': 18259, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:04.423767+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:29:04.423767+0000', 'last_peered': '2024-09-15T21:29:04.423767+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:29:04.423767+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10269155, 'num_objects': 2508, 'num_object_clones': 0, 'num_object_copies': 5016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2508, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15995, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1582}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:10.358 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:29:10.358 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18178", 'reported_seq': 18318, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:03.918321+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:29:03.918321+0000', 'last_peered': '2024-09-15T21:29:03.918321+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:29:03.918321+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10272768, 'num_objects': 2508, 'num_object_clones': 0, 'num_object_copies': 5016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2508, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15122, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1586}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:10.358 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:29:10.358 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'18078", 'reported_seq': 18217, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:03.800705+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:29:03.800705+0000', 'last_peered': '2024-09-15T21:29:03.800705+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:29:03.800705+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10289152, 'num_objects': 2512, 'num_object_clones': 0, 'num_object_copies': 5024, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2512, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14987, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1627}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:10.358 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:29:10.358 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'18227", 'reported_seq': 18369, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:03.891255+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:29:03.891255+0000', 'last_peered': '2024-09-15T21:29:03.891255+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:29:03.891255+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10810310, 'num_objects': 2641, 'num_object_clones': 0, 'num_object_copies': 5282, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2641, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14931, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1506}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:10.359 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:11.025 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:11.025 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38767/80788 objects degraded (47.986%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":602,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":40394,"data_bytes":165405449,"bytes_used":1797083136,"bytes_avail":288113209344,"bytes_total":289910292480,"degraded_objects":38767,"degraded_total":80788,"degraded_ratio":0.47986087042630099,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":396},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:11.049 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:11.049 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:11.591 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:11.593 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:11.631 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-09-15T21:29:10.850809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":155673353,"num_objects":38018,"num_object_clones":0,"num_object_copies":76036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38018,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":252441,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10816,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160649,"ondisk_log_size":160649,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1756752,"kb_used_data":503888,"kb_used_omap":12,"kb_used_meta":1252787,"kb_avail":281358768,"statfs":{"total":289910292480,"available":288111378432,"internally_reserved":0,"allocated":515981312,"data_stored":515092552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1282853891},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,3,8,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29224960,"num_objects":-7135,"num_object_clones":0,"num_object_copies":-14270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7135,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":355,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003599"},"pg_stats":[{"pgid":"2.f","version":"47'18709","reported_seq":18791,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:08.904219+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:08.904219+0000","last_peered":"2024-09-15T21:29:08.904219+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:08.904219+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10006528,"num_objects":2443,"num_object_clones":0,"num_object_copies":4886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18709,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1689}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18318","reported_seq":18429,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:09.375367+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:09.375367+0000","last_peered":"2024-09-15T21:29:09.375367+0000","last_clean":"2024-09-15T21:29:09.375367+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:09.375367+0000","last_undegraded":"2024-09-15T21:29:09.375367+0000","last_fullsized":"2024-09-15T21:29:09.375367+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9651142,"num_objects":2358,"num_object_clones":0,"num_object_copies":4716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2358,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16599,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18724","reported_seq":23989,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:09.451427+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:09.451427+0000","last_peered":"2024-09-15T21:29:09.451427+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:09.451427+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9752600,"num_objects":2382,"num_object_clones":0,"num_object_copies":4764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2382,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16824,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2566,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":680}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'17966","reported_seq":18042,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:09.426163+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:09.426163+0000","last_peered":"2024-09-15T21:29:09.426163+0000","last_clean":"2024-09-15T21:29:09.426163+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:09.426163+0000","last_undegraded":"2024-09-15T21:29:09.426163+0000","last_fullsized":"2024-09-15T21:29:09.426163+0000","mapping_epoch":35,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9617408,"num_objects":2348,"num_object_clones":0,"num_object_copies":4696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17966,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18152","reported_seq":18291,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:08.945294+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:08.945294+0000","last_peered":"2024-09-15T21:29:08.945294+0000","last_clean":"2024-09-15T21:29:08.945294+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:08.945294+0000","last_undegraded":"2024-09-15T21:29:08.945294+0000","last_fullsized":"2024-09-15T21:29:08.945294+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9638371,"num_objects":2354,"num_object_clones":0,"num_object_copies":4708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2354,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14607,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18241","reported_seq":18380,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:08.914600+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:08.914600+0000","last_peered":"2024-09-15T21:29:08.914600+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:08.914600+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9806307,"num_objects":2395,"num_object_clones":0,"num_object_copies":4790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2395,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14917,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1681},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18302","reported_seq":18442,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:08.918636+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:08.918636+0000","last_peered":"2024-09-15T21:29:08.918636+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:08.918636+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9700294,"num_objects":2370,"num_object_clones":0,"num_object_copies":4740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14613,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1791},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18516","reported_seq":18651,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:08.863972+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:08.863972+0000","last_peered":"2024-09-15T21:29:08.863972+0000","last_clean":"2024-09-15T21:29:08.863972+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:08.863972+0000","last_undegraded":"2024-09-15T21:29:08.863972+0000","last_fullsized":"2024-09-15T21:29:08.863972+0000","mapping_epoch":34,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9633792,"num_objects":2352,"num_object_clones":0,"num_object_copies":4704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15288,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18203","reported_seq":18332,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:09.319981+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:09.319981+0000","last_peered":"2024-09-15T21:29:09.319981+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:09.319981+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9462243,"num_objects":2311,"num_object_clones":0,"num_object_copies":4622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2311,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16135,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1776}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18027","reported_seq":18164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:08.940704+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:08.940704+0000","last_peered":"2024-09-15T21:29:08.940704+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:08.940704+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10055680,"num_objects":2455,"num_object_clones":0,"num_object_copies":4910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14700,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1565}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18291","reported_seq":18416,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:09.461255+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:09.461255+0000","last_peered":"2024-09-15T21:29:09.461255+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:09.461255+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9634275,"num_objects":2353,"num_object_clones":0,"num_object_copies":4706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2353,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16150,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1737}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18288","reported_seq":26753,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:09.330948+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:09.330948+0000","last_peered":"2024-09-15T21:29:09.330948+0000","last_clean":"2024-09-15T21:29:09.330948+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:09.330948+0000","last_undegraded":"2024-09-15T21:29:09.330948+0000","last_fullsized":"2024-09-15T21:29:09.330948+0000","mapping_epoch":44,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9306112,"num_objects":2272,"num_object_clones":0,"num_object_copies":4544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15981,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18281","reported_seq":18421,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:08.833846+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:08.833846+0000","last_peered":"2024-09-15T21:29:08.833846+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:08.833846+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9850880,"num_objects":2405,"num_object_clones":0,"num_object_copies":4810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15225,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1689}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18224","reported_seq":18363,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:08.895308+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:08.895308+0000","last_peered":"2024-09-15T21:29:08.895308+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:08.895308+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9691136,"num_objects":2366,"num_object_clones":0,"num_object_copies":4732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2366,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15133,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1773}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18028","reported_seq":26343,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:10.401198+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:10.401198+0000","last_peered":"2024-09-15T21:29:10.401198+0000","last_clean":"2024-09-15T21:29:10.401198+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:10.401198+0000","last_undegraded":"2024-09-15T21:29:10.401198+0000","last_fullsized":"2024-09-15T21:29:10.401198+0000","mapping_epoch":46,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9666560,"num_objects":2360,"num_object_clones":0,"num_object_copies":4720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14512,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18376","reported_seq":18521,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:08.918275+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:08.918275+0000","last_peered":"2024-09-15T21:29:08.918275+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:08.918275+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10200006,"num_objects":2492,"num_object_clones":0,"num_object_copies":4984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2492,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15080,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1655},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":155673334,"num_objects":38016,"num_object_clones":0,"num_object_copies":76032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38016,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":252439,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10815,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":512565248,"data_stored":512419810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160646,"ondisk_log_size":160646,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545332,"kb_used_data":206816,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93826508,"statfs":{"total":96636764160,"available":96078344192,"internally_reserved":0,"allocated":211779584,"data_stored":211494326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,0,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":642160,"kb_used_data":182936,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93729680,"statfs":{"total":96636764160,"available":95979192320,"internally_reserved":0,"allocated":187326464,"data_stored":187041201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659232,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569260,"kb_used_data":114136,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93802580,"statfs":{"total":96636764160,"available":96053841920,"internally_reserved":0,"allocated":116875264,"data_stored":116557025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":185802752,"data_stored":185766622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115625984,"data_stored":115556878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":211136512,"data_stored":211096310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:14.632 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:15.694 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:15.700 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:15.722 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-15T21:29:14.852008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":146600713,"num_objects":35803,"num_object_clones":0,"num_object_copies":71606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35803,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":254656,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10936,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160864,"ondisk_log_size":160864,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1736676,"kb_used_data":492136,"kb_used_omap":12,"kb_used_meta":1244467,"kb_avail":281378844,"statfs":{"total":289910292480,"available":288131936256,"internally_reserved":0,"allocated":503947264,"data_stored":503058504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1274334211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,6,7,11,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28622848,"num_objects":-6988,"num_object_clones":0,"num_object_copies":-13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003345"},"pg_stats":[{"pgid":"2.f","version":"47'18863","reported_seq":18945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.942540+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:13.942540+0000","last_peered":"2024-09-15T21:29:13.942540+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:13.942540+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9375744,"num_objects":2289,"num_object_clones":0,"num_object_copies":4578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18863,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1843}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18464","reported_seq":18571,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.436773+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:14.436773+0000","last_peered":"2024-09-15T21:29:14.436773+0000","last_clean":"2024-09-15T21:29:14.436773+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:14.436773+0000","last_undegraded":"2024-09-15T21:29:14.436773+0000","last_fullsized":"2024-09-15T21:29:14.436773+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9053126,"num_objects":2212,"num_object_clones":0,"num_object_copies":4424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2212,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16745,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18873","reported_seq":24378,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.458042+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:14.458042+0000","last_peered":"2024-09-15T21:29:14.458042+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:14.458042+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9142296,"num_objects":2233,"num_object_clones":0,"num_object_copies":4466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1582,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2233,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16973,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":651}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18105","reported_seq":18181,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.446511+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:14.446511+0000","last_peered":"2024-09-15T21:29:14.446511+0000","last_clean":"2024-09-15T21:29:14.446511+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:14.446511+0000","last_undegraded":"2024-09-15T21:29:14.446511+0000","last_fullsized":"2024-09-15T21:29:14.446511+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9048064,"num_objects":2209,"num_object_clones":0,"num_object_copies":4418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18105,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18290","reported_seq":18429,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:13.996123+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:13.996123+0000","last_peered":"2024-09-15T21:29:13.996123+0000","last_clean":"2024-09-15T21:29:13.996123+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:13.996123+0000","last_undegraded":"2024-09-15T21:29:13.996123+0000","last_fullsized":"2024-09-15T21:29:13.996123+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9073123,"num_objects":2216,"num_object_clones":0,"num_object_copies":4432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14745,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18394","reported_seq":18536,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.809600+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:13.809600+0000","last_peered":"2024-09-15T21:29:13.809600+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:13.809600+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9179619,"num_objects":2242,"num_object_clones":0,"num_object_copies":4484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15070,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1834},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18448","reported_seq":18590,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.984381+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:13.984381+0000","last_peered":"2024-09-15T21:29:13.984381+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:13.984381+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9102278,"num_objects":2224,"num_object_clones":0,"num_object_copies":4448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14759,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1937},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18660","reported_seq":18795,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:13.920603+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:13.920603+0000","last_peered":"2024-09-15T21:29:13.920603+0000","last_clean":"2024-09-15T21:29:13.920603+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:13.920603+0000","last_undegraded":"2024-09-15T21:29:13.920603+0000","last_fullsized":"2024-09-15T21:29:13.920603+0000","mapping_epoch":34,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9043968,"num_objects":2208,"num_object_clones":0,"num_object_copies":4416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15432,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18334","reported_seq":18467,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.468144+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:14.468144+0000","last_peered":"2024-09-15T21:29:14.468144+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:14.468144+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8925667,"num_objects":2180,"num_object_clones":0,"num_object_copies":4360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16266,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1907}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18197","reported_seq":18331,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.924910+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:13.924910+0000","last_peered":"2024-09-15T21:29:13.924910+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:13.924910+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9359360,"num_objects":2285,"num_object_clones":0,"num_object_copies":4570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14870,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1735}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18442","reported_seq":18567,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.402172+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:14.402172+0000","last_peered":"2024-09-15T21:29:14.402172+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:14.402172+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9015779,"num_objects":2202,"num_object_clones":0,"num_object_copies":4404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16301,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1888}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18434","reported_seq":26899,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.449503+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:14.449503+0000","last_peered":"2024-09-15T21:29:14.449503+0000","last_clean":"2024-09-15T21:29:14.449503+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:14.449503+0000","last_undegraded":"2024-09-15T21:29:14.449503+0000","last_fullsized":"2024-09-15T21:29:14.449503+0000","mapping_epoch":44,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8708096,"num_objects":2126,"num_object_clones":0,"num_object_copies":4252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16127,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18432","reported_seq":18572,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.842323+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:13.842323+0000","last_peered":"2024-09-15T21:29:13.842323+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:13.842323+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9232384,"num_objects":2254,"num_object_clones":0,"num_object_copies":4508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15376,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18371","reported_seq":18510,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.970598+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:13.970598+0000","last_peered":"2024-09-15T21:29:13.970598+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:13.970598+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9089024,"num_objects":2219,"num_object_clones":0,"num_object_copies":4438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15280,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18028","reported_seq":26343,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:10.401198+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:10.401198+0000","last_peered":"2024-09-15T21:29:10.401198+0000","last_clean":"2024-09-15T21:29:10.401198+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:10.401198+0000","last_undegraded":"2024-09-15T21:29:10.401198+0000","last_fullsized":"2024-09-15T21:29:10.401198+0000","mapping_epoch":46,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9666560,"num_objects":2360,"num_object_clones":0,"num_object_copies":4720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14512,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18526","reported_seq":18670,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.965039+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:13.965039+0000","last_peered":"2024-09-15T21:29:13.965039+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:13.965039+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9585606,"num_objects":2342,"num_object_clones":0,"num_object_copies":4684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2342,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15230,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1805},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":146600694,"num_objects":35801,"num_object_clones":0,"num_object_copies":71602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35801,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":254654,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10935,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":500637696,"data_stored":500492258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160861,"ondisk_log_size":160861,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544968,"kb_used_data":202296,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93826872,"statfs":{"total":96636764160,"available":96078716928,"internally_reserved":0,"allocated":207151104,"data_stored":206865846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622448,"kb_used_data":175704,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93749392,"statfs":{"total":96636764160,"available":95999377408,"internally_reserved":0,"allocated":179920896,"data_stored":179635633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457436843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,4,2,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659232,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569260,"kb_used_data":114136,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93802580,"statfs":{"total":96636764160,"available":96053841920,"internally_reserved":0,"allocated":116875264,"data_stored":116557025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":178540544,"data_stored":178504414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115625984,"data_stored":115556878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":206471168,"data_stored":206430966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:15.724 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:16.251 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:16.254 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:16.285 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-15T21:29:14.852008+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":146600713,"num_objects":35803,"num_object_clones":0,"num_object_copies":71606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35803,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":254656,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10936,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160864,"ondisk_log_size":160864,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1736676,"kb_used_data":492136,"kb_used_omap":12,"kb_used_meta":1244467,"kb_avail":281378844,"statfs":{"total":289910292480,"available":288131936256,"internally_reserved":0,"allocated":503947264,"data_stored":503058504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1274334211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,6,7,11,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28622848,"num_objects":-6988,"num_object_clones":0,"num_object_copies":-13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-358,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003345"},"pg_stats":[{"pgid":"2.f","version":"47'18863","reported_seq":18945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.942540+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:13.942540+0000","last_peered":"2024-09-15T21:29:13.942540+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:13.942540+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9375744,"num_objects":2289,"num_object_clones":0,"num_object_copies":4578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18863,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1843}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18464","reported_seq":18571,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.436773+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:14.436773+0000","last_peered":"2024-09-15T21:29:14.436773+0000","last_clean":"2024-09-15T21:29:14.436773+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:14.436773+0000","last_undegraded":"2024-09-15T21:29:14.436773+0000","last_fullsized":"2024-09-15T21:29:14.436773+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9053126,"num_objects":2212,"num_object_clones":0,"num_object_copies":4424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2212,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16745,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18873","reported_seq":24378,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.458042+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:14.458042+0000","last_peered":"2024-09-15T21:29:14.458042+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:14.458042+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9142296,"num_objects":2233,"num_object_clones":0,"num_object_copies":4466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1582,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2233,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16973,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":651}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18105","reported_seq":18181,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.446511+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:14.446511+0000","last_peered":"2024-09-15T21:29:14.446511+0000","last_clean":"2024-09-15T21:29:14.446511+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:14.446511+0000","last_undegraded":"2024-09-15T21:29:14.446511+0000","last_fullsized":"2024-09-15T21:29:14.446511+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9048064,"num_objects":2209,"num_object_clones":0,"num_object_copies":4418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18105,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18290","reported_seq":18429,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:13.996123+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:13.996123+0000","last_peered":"2024-09-15T21:29:13.996123+0000","last_clean":"2024-09-15T21:29:13.996123+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:13.996123+0000","last_undegraded":"2024-09-15T21:29:13.996123+0000","last_fullsized":"2024-09-15T21:29:13.996123+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9073123,"num_objects":2216,"num_object_clones":0,"num_object_copies":4432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14745,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18394","reported_seq":18536,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.809600+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:13.809600+0000","last_peered":"2024-09-15T21:29:13.809600+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:13.809600+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9179619,"num_objects":2242,"num_object_clones":0,"num_object_copies":4484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15070,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1834},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18448","reported_seq":18590,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.984381+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:13.984381+0000","last_peered":"2024-09-15T21:29:13.984381+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:13.984381+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9102278,"num_objects":2224,"num_object_clones":0,"num_object_copies":4448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14759,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1937},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18660","reported_seq":18795,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:13.920603+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:13.920603+0000","last_peered":"2024-09-15T21:29:13.920603+0000","last_clean":"2024-09-15T21:29:13.920603+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:13.920603+0000","last_undegraded":"2024-09-15T21:29:13.920603+0000","last_fullsized":"2024-09-15T21:29:13.920603+0000","mapping_epoch":34,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9043968,"num_objects":2208,"num_object_clones":0,"num_object_copies":4416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15432,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317138+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:09.317138+0000","last_peered":"2024-09-15T21:27:09.317138+0000","last_clean":"2024-09-15T21:27:09.317138+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:09.317138+0000","last_undegraded":"2024-09-15T21:27:09.317138+0000","last_fullsized":"2024-09-15T21:27:09.317138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18334","reported_seq":18467,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.468144+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:14.468144+0000","last_peered":"2024-09-15T21:29:14.468144+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:14.468144+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8925667,"num_objects":2180,"num_object_clones":0,"num_object_copies":4360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16266,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1907}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18197","reported_seq":18331,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.924910+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:13.924910+0000","last_peered":"2024-09-15T21:29:13.924910+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:13.924910+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9359360,"num_objects":2285,"num_object_clones":0,"num_object_copies":4570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14870,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1735}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18442","reported_seq":18567,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.402172+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:14.402172+0000","last_peered":"2024-09-15T21:29:14.402172+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:14.402172+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9015779,"num_objects":2202,"num_object_clones":0,"num_object_copies":4404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16301,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1888}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18434","reported_seq":26899,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.449503+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:14.449503+0000","last_peered":"2024-09-15T21:29:14.449503+0000","last_clean":"2024-09-15T21:29:14.449503+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:14.449503+0000","last_undegraded":"2024-09-15T21:29:14.449503+0000","last_fullsized":"2024-09-15T21:29:14.449503+0000","mapping_epoch":44,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8708096,"num_objects":2126,"num_object_clones":0,"num_object_copies":4252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16127,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18432","reported_seq":18572,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.842323+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:13.842323+0000","last_peered":"2024-09-15T21:29:13.842323+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:13.842323+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9232384,"num_objects":2254,"num_object_clones":0,"num_object_copies":4508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15376,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18371","reported_seq":18510,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.970598+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:13.970598+0000","last_peered":"2024-09-15T21:29:13.970598+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:13.970598+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9089024,"num_objects":2219,"num_object_clones":0,"num_object_copies":4438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15280,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18028","reported_seq":26343,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:10.401198+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:10.401198+0000","last_peered":"2024-09-15T21:29:10.401198+0000","last_clean":"2024-09-15T21:29:10.401198+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:10.401198+0000","last_undegraded":"2024-09-15T21:29:10.401198+0000","last_fullsized":"2024-09-15T21:29:10.401198+0000","mapping_epoch":46,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9666560,"num_objects":2360,"num_object_clones":0,"num_object_copies":4720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14512,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18526","reported_seq":18670,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.965039+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:13.965039+0000","last_peered":"2024-09-15T21:29:13.965039+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:13.965039+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9585606,"num_objects":2342,"num_object_clones":0,"num_object_copies":4684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2342,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15230,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1805},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":146600694,"num_objects":35801,"num_object_clones":0,"num_object_copies":71602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35801,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":254654,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10935,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":500637696,"data_stored":500492258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160861,"ondisk_log_size":160861,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544968,"kb_used_data":202296,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93826872,"statfs":{"total":96636764160,"available":96078716928,"internally_reserved":0,"allocated":207151104,"data_stored":206865846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622448,"kb_used_data":175704,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93749392,"statfs":{"total":96636764160,"available":95999377408,"internally_reserved":0,"allocated":179920896,"data_stored":179635633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457436843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,4,2,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659232,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569260,"kb_used_data":114136,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93802580,"statfs":{"total":96636764160,"available":96053841920,"internally_reserved":0,"allocated":116875264,"data_stored":116557025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":178540544,"data_stored":178504414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":115625984,"data_stored":115556878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":206471168,"data_stored":206430966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:16.286 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:29:16.287 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'18863", 'reported_seq': 18945, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:13.942540+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:29:13.942540+0000', 'last_peered': '2024-09-15T21:29:13.942540+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:29:13.942540+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9375744, 'num_objects': 2289, 'num_object_clones': 0, 'num_object_copies': 4578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2289, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18863, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1843}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:16.287 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:29:16.287 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'18873", 'reported_seq': 24378, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.458042+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:29:14.458042+0000', 'last_peered': '2024-09-15T21:29:14.458042+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:29:14.458042+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9142296, 'num_objects': 2233, 'num_object_clones': 0, 'num_object_copies': 4466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1582, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2233, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16973, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2686, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 651}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:16.287 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:29:16.287 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18394", 'reported_seq': 18536, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:13.809600+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:29:13.809600+0000', 'last_peered': '2024-09-15T21:29:13.809600+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:29:13.809600+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9179619, 'num_objects': 2242, 'num_object_clones': 0, 'num_object_copies': 4484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2242, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15070, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1834}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:16.287 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:16.287 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18448", 'reported_seq': 18590, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:13.984381+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:29:13.984381+0000', 'last_peered': '2024-09-15T21:29:13.984381+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:29:13.984381+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9102278, 'num_objects': 2224, 'num_object_clones': 0, 'num_object_copies': 4448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14759, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1937}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:16.287 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:29:16.287 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'18334", 'reported_seq': 18467, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.468144+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:29:14.468144+0000', 'last_peered': '2024-09-15T21:29:14.468144+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:29:14.468144+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8925667, 'num_objects': 2180, 'num_object_clones': 0, 'num_object_copies': 4360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2180, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16266, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1907}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:16.287 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:16.287 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18197", 'reported_seq': 18331, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:13.924910+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:29:13.924910+0000', 'last_peered': '2024-09-15T21:29:13.924910+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:29:13.924910+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9359360, 'num_objects': 2285, 'num_object_clones': 0, 'num_object_copies': 4570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2285, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14870, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1735}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:16.288 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:29:16.288 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18442", 'reported_seq': 18567, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:14.402172+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:29:14.402172+0000', 'last_peered': '2024-09-15T21:29:14.402172+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:29:14.402172+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9015779, 'num_objects': 2202, 'num_object_clones': 0, 'num_object_copies': 4404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2202, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16301, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -1888}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:16.288 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:29:16.288 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18432", 'reported_seq': 18572, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:13.842323+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:29:13.842323+0000', 'last_peered': '2024-09-15T21:29:13.842323+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:29:13.842323+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9232384, 'num_objects': 2254, 'num_object_clones': 0, 'num_object_copies': 4508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2254, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15376, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1840}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:16.288 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:29:16.288 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'18371", 'reported_seq': 18510, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:13.970598+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:29:13.970598+0000', 'last_peered': '2024-09-15T21:29:13.970598+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:29:13.970598+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9089024, 'num_objects': 2219, 'num_object_clones': 0, 'num_object_copies': 4438, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2219, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15280, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1920}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:16.288 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:29:16.288 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'18526", 'reported_seq': 18670, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:13.965039+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:29:13.965039+0000', 'last_peered': '2024-09-15T21:29:13.965039+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:29:13.965039+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9585606, 'num_objects': 2342, 'num_object_clones': 0, 'num_object_copies': 4684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2342, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15230, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1805}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:16.289 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:16.964 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:16.964 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38539/71606 objects degraded (53.821%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":608,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":35803,"data_bytes":146600713,"bytes_used":1778356224,"bytes_avail":288131936256,"bytes_total":289910292480,"degraded_objects":38539,"degraded_total":71606,"degraded_ratio":0.5382090886238583,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":358,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":582},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:16.987 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:16.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:17.586 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:17.589 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:17.610 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-15T21:29:16.852743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":145998601,"num_objects":35656,"num_object_clones":0,"num_object_copies":71312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35656,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":254803,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10936,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1716812,"kb_used_data":484756,"kb_used_omap":12,"kb_used_meta":1231987,"kb_avail":281398708,"statfs":{"total":289910292480,"available":288152276992,"internally_reserved":0,"allocated":496390144,"data_stored":495501384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1261554691},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,4,9,13,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20004864,"num_objects":-4884,"num_object_clones":0,"num_object_copies":-9768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-228,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4884,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":228,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003658"},"pg_stats":[{"pgid":"2.f","version":"47'18863","reported_seq":18945,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.942540+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:13.942540+0000","last_peered":"2024-09-15T21:29:13.942540+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:13.942540+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9375744,"num_objects":2289,"num_object_clones":0,"num_object_copies":4578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2289,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18863,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1843}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18464","reported_seq":18571,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.436773+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:14.436773+0000","last_peered":"2024-09-15T21:29:14.436773+0000","last_clean":"2024-09-15T21:29:14.436773+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:14.436773+0000","last_undegraded":"2024-09-15T21:29:14.436773+0000","last_fullsized":"2024-09-15T21:29:14.436773+0000","mapping_epoch":35,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9053126,"num_objects":2212,"num_object_clones":0,"num_object_copies":4424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2212,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16745,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'18873","reported_seq":24378,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.458042+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:14.458042+0000","last_peered":"2024-09-15T21:29:14.458042+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:14.458042+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9142296,"num_objects":2233,"num_object_clones":0,"num_object_copies":4466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1582,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2233,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16973,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":651}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18105","reported_seq":18181,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.446511+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:14.446511+0000","last_peered":"2024-09-15T21:29:14.446511+0000","last_clean":"2024-09-15T21:29:14.446511+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:14.446511+0000","last_undegraded":"2024-09-15T21:29:14.446511+0000","last_fullsized":"2024-09-15T21:29:14.446511+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9048064,"num_objects":2209,"num_object_clones":0,"num_object_copies":4418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18105,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18290","reported_seq":18429,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:13.996123+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:13.996123+0000","last_peered":"2024-09-15T21:29:13.996123+0000","last_clean":"2024-09-15T21:29:13.996123+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:13.996123+0000","last_undegraded":"2024-09-15T21:29:13.996123+0000","last_fullsized":"2024-09-15T21:29:13.996123+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9073123,"num_objects":2216,"num_object_clones":0,"num_object_copies":4432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14745,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18394","reported_seq":18536,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.809600+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:13.809600+0000","last_peered":"2024-09-15T21:29:13.809600+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:13.809600+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9179619,"num_objects":2242,"num_object_clones":0,"num_object_copies":4484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2242,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15070,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1834},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18448","reported_seq":18590,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.984381+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:13.984381+0000","last_peered":"2024-09-15T21:29:13.984381+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:13.984381+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9102278,"num_objects":2224,"num_object_clones":0,"num_object_copies":4448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14759,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1937},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18660","reported_seq":18795,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:13.920603+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:13.920603+0000","last_peered":"2024-09-15T21:29:13.920603+0000","last_clean":"2024-09-15T21:29:13.920603+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:13.920603+0000","last_undegraded":"2024-09-15T21:29:13.920603+0000","last_fullsized":"2024-09-15T21:29:13.920603+0000","mapping_epoch":34,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9043968,"num_objects":2208,"num_object_clones":0,"num_object_copies":4416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15432,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18334","reported_seq":18467,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.468144+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:14.468144+0000","last_peered":"2024-09-15T21:29:14.468144+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:14.468144+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8925667,"num_objects":2180,"num_object_clones":0,"num_object_copies":4360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2180,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16266,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1907}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18197","reported_seq":18331,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.924910+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:13.924910+0000","last_peered":"2024-09-15T21:29:13.924910+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:13.924910+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9359360,"num_objects":2285,"num_object_clones":0,"num_object_copies":4570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14870,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1735}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18442","reported_seq":18567,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:14.402172+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:14.402172+0000","last_peered":"2024-09-15T21:29:14.402172+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:14.402172+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9015779,"num_objects":2202,"num_object_clones":0,"num_object_copies":4404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2202,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16301,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-1888}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18434","reported_seq":26899,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:14.449503+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:14.449503+0000","last_peered":"2024-09-15T21:29:14.449503+0000","last_clean":"2024-09-15T21:29:14.449503+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:14.449503+0000","last_undegraded":"2024-09-15T21:29:14.449503+0000","last_fullsized":"2024-09-15T21:29:14.449503+0000","mapping_epoch":44,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8708096,"num_objects":2126,"num_object_clones":0,"num_object_copies":4252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16127,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18432","reported_seq":18572,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.842323+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:13.842323+0000","last_peered":"2024-09-15T21:29:13.842323+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:13.842323+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9232384,"num_objects":2254,"num_object_clones":0,"num_object_copies":4508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15376,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18371","reported_seq":18510,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.970598+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:13.970598+0000","last_peered":"2024-09-15T21:29:13.970598+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:13.970598+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9089024,"num_objects":2219,"num_object_clones":0,"num_object_copies":4438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15280,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18175","reported_seq":26491,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:15.438304+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:15.438304+0000","last_peered":"2024-09-15T21:29:15.438304+0000","last_clean":"2024-09-15T21:29:15.438304+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:15.438304+0000","last_undegraded":"2024-09-15T21:29:15.438304+0000","last_fullsized":"2024-09-15T21:29:15.438304+0000","mapping_epoch":46,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9064448,"num_objects":2213,"num_object_clones":0,"num_object_copies":4426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14659,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18526","reported_seq":18670,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:13.965039+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:13.965039+0000","last_peered":"2024-09-15T21:29:13.965039+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:13.965039+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9585606,"num_objects":2342,"num_object_clones":0,"num_object_copies":4684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2342,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15230,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1805},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":145998582,"num_objects":35654,"num_object_clones":0,"num_object_copies":71308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35654,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":254801,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10935,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":493076480,"data_stored":492931042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160908,"ondisk_log_size":160908,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544968,"kb_used_data":202296,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93826872,"statfs":{"total":96636764160,"available":96078716928,"internally_reserved":0,"allocated":207151104,"data_stored":206865846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622448,"kb_used_data":175704,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93749392,"statfs":{"total":96636764160,"available":95999377408,"internally_reserved":0,"allocated":179920896,"data_stored":179635633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457436843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,1,4,2,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549396,"kb_used_data":106756,"kb_used_omap":4,"kb_used_meta":442619,"kb_avail":93822444,"statfs":{"total":96636764160,"available":96074182656,"internally_reserved":0,"allocated":109318144,"data_stored":108999905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":453242543},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":178540544,"data_stored":178504414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":108064768,"data_stored":107995662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":206471168,"data_stored":206430966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:20.613 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:21.130 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:21.131 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:21.158 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-15T21:29:20.854312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":136532745,"num_objects":33345,"num_object_clones":0,"num_object_copies":66690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38416,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33345,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":257114,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11059,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1714708,"kb_used_data":466008,"kb_used_omap":12,"kb_used_meta":1248627,"kb_avail":281400812,"statfs":{"total":289910292480,"available":288154431488,"internally_reserved":0,"allocated":477192192,"data_stored":476303432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1278594051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,8,2,7,1,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28872704,"num_objects":-7049,"num_object_clones":0,"num_object_copies":-14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-351,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7049,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004072"},"pg_stats":[{"pgid":"2.f","version":"47'19001","reported_seq":19083,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.910145+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:18.910145+0000","last_peered":"2024-09-15T21:29:18.910145+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:18.910145+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"44'9000","ondisk_log_start":"44'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8810496,"num_objects":2151,"num_object_clones":0,"num_object_copies":4302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19001,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1981}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18592","reported_seq":18703,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:19.489199+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:19.489199+0000","last_peered":"2024-09-15T21:29:19.489199+0000","last_clean":"2024-09-15T21:29:19.489199+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:19.489199+0000","last_undegraded":"2024-09-15T21:29:19.489199+0000","last_fullsized":"2024-09-15T21:29:19.489199+0000","mapping_epoch":35,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8528838,"num_objects":2084,"num_object_clones":0,"num_object_copies":4168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2084,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16873,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19015","reported_seq":24766,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:19.501470+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:19.501470+0000","last_peered":"2024-09-15T21:29:19.501470+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:19.501470+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8560664,"num_objects":2091,"num_object_clones":0,"num_object_copies":4182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1459,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2091,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17115,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2809,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":632}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18259","reported_seq":18336,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:19.506998+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:19.506998+0000","last_peered":"2024-09-15T21:29:19.506998+0000","last_clean":"2024-09-15T21:29:19.506998+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:19.506998+0000","last_undegraded":"2024-09-15T21:29:19.506998+0000","last_fullsized":"2024-09-15T21:29:19.506998+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8417280,"num_objects":2055,"num_object_clones":0,"num_object_copies":4110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18259,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18432","reported_seq":18571,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:18.908655+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:18.908655+0000","last_peered":"2024-09-15T21:29:18.908655+0000","last_clean":"2024-09-15T21:29:18.908655+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:18.908655+0000","last_undegraded":"2024-09-15T21:29:18.908655+0000","last_fullsized":"2024-09-15T21:29:18.908655+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8491491,"num_objects":2074,"num_object_clones":0,"num_object_copies":4148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14887,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18536","reported_seq":18675,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.944132+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:18.944132+0000","last_peered":"2024-09-15T21:29:18.944132+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:18.944132+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8597987,"num_objects":2100,"num_object_clones":0,"num_object_copies":4200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2100,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15212,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1976},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18603","reported_seq":18745,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.984085+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:18.984085+0000","last_peered":"2024-09-15T21:29:18.984085+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:18.984085+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8467398,"num_objects":2069,"num_object_clones":0,"num_object_copies":4138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14914,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2092},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18802","reported_seq":18937,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:18.939208+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:18.939208+0000","last_peered":"2024-09-15T21:29:18.939208+0000","last_clean":"2024-09-15T21:29:18.939208+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:18.939208+0000","last_undegraded":"2024-09-15T21:29:18.939208+0000","last_fullsized":"2024-09-15T21:29:18.939208+0000","mapping_epoch":34,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8462336,"num_objects":2066,"num_object_clones":0,"num_object_copies":4132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15574,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18498","reported_seq":18631,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:19.477196+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:19.477196+0000","last_peered":"2024-09-15T21:29:19.477196+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:19.477196+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8253923,"num_objects":2016,"num_object_clones":0,"num_object_copies":4032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2016,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16430,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2071}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18347","reported_seq":18483,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.972467+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:18.972467+0000","last_peered":"2024-09-15T21:29:18.972467+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:18.972467+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8744960,"num_objects":2135,"num_object_clones":0,"num_object_copies":4270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1885}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18581","reported_seq":18705,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:19.415333+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:19.415333+0000","last_peered":"2024-09-15T21:29:19.415333+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:19.415333+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8446435,"num_objects":2063,"num_object_clones":0,"num_object_copies":4126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2063,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16440,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2027}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18582","reported_seq":27046,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:19.418557+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:19.418557+0000","last_peered":"2024-09-15T21:29:19.418557+0000","last_clean":"2024-09-15T21:29:19.418557+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:19.418557+0000","last_undegraded":"2024-09-15T21:29:19.418557+0000","last_fullsized":"2024-09-15T21:29:19.418557+0000","mapping_epoch":44,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8101888,"num_objects":1978,"num_object_clones":0,"num_object_copies":3956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16275,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18571","reported_seq":18710,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:19.021322+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:19.021322+0000","last_peered":"2024-09-15T21:29:19.021322+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:19.021322+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8663040,"num_objects":2115,"num_object_clones":0,"num_object_copies":4230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15515,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18488","reported_seq":18627,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.934126+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:18.934126+0000","last_peered":"2024-09-15T21:29:18.934126+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:18.934126+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8609792,"num_objects":2102,"num_object_clones":0,"num_object_copies":4204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15397,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2037}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18320","reported_seq":26636,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:20.506107+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:20.506107+0000","last_peered":"2024-09-15T21:29:20.506107+0000","last_clean":"2024-09-15T21:29:20.506107+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:20.506107+0000","last_undegraded":"2024-09-15T21:29:20.506107+0000","last_fullsized":"2024-09-15T21:29:20.506107+0000","mapping_epoch":46,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8470528,"num_objects":2068,"num_object_clones":0,"num_object_copies":4136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14804,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18692","reported_seq":18837,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.932378+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:18.932378+0000","last_peered":"2024-09-15T21:29:18.932378+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:18.932378+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8905670,"num_objects":2176,"num_object_clones":0,"num_object_copies":4352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15396,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1971},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":136532726,"num_objects":33343,"num_object_clones":0,"num_object_copies":66686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38416,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33343,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":257112,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11058,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":474017792,"data_stored":473872354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544388,"kb_used_data":197556,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93827452,"statfs":{"total":96636764160,"available":96079310848,"internally_reserved":0,"allocated":202297344,"data_stored":202012086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619544,"kb_used_data":168636,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93752296,"statfs":{"total":96636764160,"available":96002351104,"internally_reserved":0,"allocated":172683264,"data_stored":172398001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,1,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550776,"kb_used_data":99816,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93821064,"statfs":{"total":96636764160,"available":96072769536,"internally_reserved":0,"allocated":102211584,"data_stored":101893345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,0,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171487232,"data_stored":171451102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100831232,"data_stored":100762126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":201699328,"data_stored":201659126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:21.159 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:21.722 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:21.725 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:21.744 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-15T21:29:20.854312+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":136532745,"num_objects":33345,"num_object_clones":0,"num_object_copies":66690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38416,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33345,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":257114,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11059,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1714708,"kb_used_data":466008,"kb_used_omap":12,"kb_used_meta":1248627,"kb_avail":281400812,"statfs":{"total":289910292480,"available":288154431488,"internally_reserved":0,"allocated":477192192,"data_stored":476303432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1278594051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,8,2,7,1,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28872704,"num_objects":-7049,"num_object_clones":0,"num_object_copies":-14098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-351,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7049,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004072"},"pg_stats":[{"pgid":"2.f","version":"47'19001","reported_seq":19083,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.910145+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:18.910145+0000","last_peered":"2024-09-15T21:29:18.910145+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:18.910145+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"44'9000","ondisk_log_start":"44'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8810496,"num_objects":2151,"num_object_clones":0,"num_object_copies":4302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19001,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1981}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18592","reported_seq":18703,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:19.489199+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:19.489199+0000","last_peered":"2024-09-15T21:29:19.489199+0000","last_clean":"2024-09-15T21:29:19.489199+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:19.489199+0000","last_undegraded":"2024-09-15T21:29:19.489199+0000","last_fullsized":"2024-09-15T21:29:19.489199+0000","mapping_epoch":35,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8528838,"num_objects":2084,"num_object_clones":0,"num_object_copies":4168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2084,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16873,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19015","reported_seq":24766,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:19.501470+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:19.501470+0000","last_peered":"2024-09-15T21:29:19.501470+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:19.501470+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8560664,"num_objects":2091,"num_object_clones":0,"num_object_copies":4182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1459,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2091,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17115,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2809,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":632}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18259","reported_seq":18336,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:19.506998+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:19.506998+0000","last_peered":"2024-09-15T21:29:19.506998+0000","last_clean":"2024-09-15T21:29:19.506998+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:19.506998+0000","last_undegraded":"2024-09-15T21:29:19.506998+0000","last_fullsized":"2024-09-15T21:29:19.506998+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8417280,"num_objects":2055,"num_object_clones":0,"num_object_copies":4110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18259,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18432","reported_seq":18571,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:18.908655+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:18.908655+0000","last_peered":"2024-09-15T21:29:18.908655+0000","last_clean":"2024-09-15T21:29:18.908655+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:18.908655+0000","last_undegraded":"2024-09-15T21:29:18.908655+0000","last_fullsized":"2024-09-15T21:29:18.908655+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8491491,"num_objects":2074,"num_object_clones":0,"num_object_copies":4148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14887,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18536","reported_seq":18675,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.944132+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:18.944132+0000","last_peered":"2024-09-15T21:29:18.944132+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:18.944132+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8597987,"num_objects":2100,"num_object_clones":0,"num_object_copies":4200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2100,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15212,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1976},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18603","reported_seq":18745,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.984085+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:18.984085+0000","last_peered":"2024-09-15T21:29:18.984085+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:18.984085+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8467398,"num_objects":2069,"num_object_clones":0,"num_object_copies":4138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14914,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2092},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18802","reported_seq":18937,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:18.939208+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:18.939208+0000","last_peered":"2024-09-15T21:29:18.939208+0000","last_clean":"2024-09-15T21:29:18.939208+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:18.939208+0000","last_undegraded":"2024-09-15T21:29:18.939208+0000","last_fullsized":"2024-09-15T21:29:18.939208+0000","mapping_epoch":34,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8462336,"num_objects":2066,"num_object_clones":0,"num_object_copies":4132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15574,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18498","reported_seq":18631,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:19.477196+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:19.477196+0000","last_peered":"2024-09-15T21:29:19.477196+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:19.477196+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8253923,"num_objects":2016,"num_object_clones":0,"num_object_copies":4032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2016,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16430,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2071}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18347","reported_seq":18483,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.972467+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:18.972467+0000","last_peered":"2024-09-15T21:29:18.972467+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:18.972467+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8744960,"num_objects":2135,"num_object_clones":0,"num_object_copies":4270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1885}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18581","reported_seq":18705,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:19.415333+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:19.415333+0000","last_peered":"2024-09-15T21:29:19.415333+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:19.415333+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8446435,"num_objects":2063,"num_object_clones":0,"num_object_copies":4126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2063,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16440,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2027}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18582","reported_seq":27046,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:19.418557+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:19.418557+0000","last_peered":"2024-09-15T21:29:19.418557+0000","last_clean":"2024-09-15T21:29:19.418557+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:19.418557+0000","last_undegraded":"2024-09-15T21:29:19.418557+0000","last_fullsized":"2024-09-15T21:29:19.418557+0000","mapping_epoch":44,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8101888,"num_objects":1978,"num_object_clones":0,"num_object_copies":3956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16275,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18571","reported_seq":18710,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:19.021322+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:19.021322+0000","last_peered":"2024-09-15T21:29:19.021322+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:19.021322+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8663040,"num_objects":2115,"num_object_clones":0,"num_object_copies":4230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15515,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18488","reported_seq":18627,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.934126+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:18.934126+0000","last_peered":"2024-09-15T21:29:18.934126+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:18.934126+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8609792,"num_objects":2102,"num_object_clones":0,"num_object_copies":4204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15397,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2037}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18320","reported_seq":26636,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:20.506107+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:20.506107+0000","last_peered":"2024-09-15T21:29:20.506107+0000","last_clean":"2024-09-15T21:29:20.506107+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:20.506107+0000","last_undegraded":"2024-09-15T21:29:20.506107+0000","last_fullsized":"2024-09-15T21:29:20.506107+0000","mapping_epoch":46,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8470528,"num_objects":2068,"num_object_clones":0,"num_object_copies":4136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14804,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18692","reported_seq":18837,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.932378+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:18.932378+0000","last_peered":"2024-09-15T21:29:18.932378+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:18.932378+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8905670,"num_objects":2176,"num_object_clones":0,"num_object_copies":4352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15396,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1971},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":136532726,"num_objects":33343,"num_object_clones":0,"num_object_copies":66686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38416,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33343,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":257112,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11058,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":474017792,"data_stored":473872354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544388,"kb_used_data":197556,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93827452,"statfs":{"total":96636764160,"available":96079310848,"internally_reserved":0,"allocated":202297344,"data_stored":202012086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619544,"kb_used_data":168636,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93752296,"statfs":{"total":96636764160,"available":96002351104,"internally_reserved":0,"allocated":172683264,"data_stored":172398001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,1,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550776,"kb_used_data":99816,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93821064,"statfs":{"total":96636764160,"available":96072769536,"internally_reserved":0,"allocated":102211584,"data_stored":101893345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,0,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171487232,"data_stored":171451102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100831232,"data_stored":100762126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":201699328,"data_stored":201659126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:21.746 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:29:21.746 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19001", 'reported_seq': 19083, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:18.910145+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:29:18.910145+0000', 'last_peered': '2024-09-15T21:29:18.910145+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:29:18.910145+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "44'9000", 'ondisk_log_start': "44'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8810496, 'num_objects': 2151, 'num_object_clones': 0, 'num_object_copies': 4302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2151, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19001, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1981}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:21.746 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:29:21.746 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'19015", 'reported_seq': 24766, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:19.501470+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:29:19.501470+0000', 'last_peered': '2024-09-15T21:29:19.501470+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:29:19.501470+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8560664, 'num_objects': 2091, 'num_object_clones': 0, 'num_object_copies': 4182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1459, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2091, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17115, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2809, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 632}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:21.746 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:29:21.746 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18536", 'reported_seq': 18675, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:18.944132+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:29:18.944132+0000', 'last_peered': '2024-09-15T21:29:18.944132+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:29:18.944132+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8597987, 'num_objects': 2100, 'num_object_clones': 0, 'num_object_copies': 4200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2100, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15212, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1976}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:21.746 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:21.747 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18603", 'reported_seq': 18745, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:18.984085+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:29:18.984085+0000', 'last_peered': '2024-09-15T21:29:18.984085+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:29:18.984085+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8467398, 'num_objects': 2069, 'num_object_clones': 0, 'num_object_copies': 4138, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2069, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14914, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2092}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:21.747 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:29:21.747 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'18498", 'reported_seq': 18631, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:19.477196+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:29:19.477196+0000', 'last_peered': '2024-09-15T21:29:19.477196+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:29:19.477196+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8253923, 'num_objects': 2016, 'num_object_clones': 0, 'num_object_copies': 4032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2016, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16430, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2071}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:21.747 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:21.747 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18347", 'reported_seq': 18483, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:18.972467+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:29:18.972467+0000', 'last_peered': '2024-09-15T21:29:18.972467+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:29:18.972467+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8744960, 'num_objects': 2135, 'num_object_clones': 0, 'num_object_copies': 4270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2135, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15020, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1885}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:21.747 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:29:21.747 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18581", 'reported_seq': 18705, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:19.415333+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:29:19.415333+0000', 'last_peered': '2024-09-15T21:29:19.415333+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:29:19.415333+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8446435, 'num_objects': 2063, 'num_object_clones': 0, 'num_object_copies': 4126, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2063, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16440, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2027}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:21.747 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:29:21.747 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18571", 'reported_seq': 18710, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:19.021322+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:29:19.021322+0000', 'last_peered': '2024-09-15T21:29:19.021322+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:29:19.021322+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'8500", 'ondisk_log_start': "43'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8663040, 'num_objects': 2115, 'num_object_clones': 0, 'num_object_copies': 4230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2115, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15515, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1979}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:21.747 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:29:21.747 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'18488", 'reported_seq': 18627, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:18.934126+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:29:18.934126+0000', 'last_peered': '2024-09-15T21:29:18.934126+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:29:18.934126+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8609792, 'num_objects': 2102, 'num_object_clones': 0, 'num_object_copies': 4204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2102, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15397, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2037}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:21.748 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:29:21.748 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'18692", 'reported_seq': 18837, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:18.932378+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:29:18.932378+0000', 'last_peered': '2024-09-15T21:29:18.932378+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:29:18.932378+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8905670, 'num_objects': 2176, 'num_object_clones': 0, 'num_object_copies': 4352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2176, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15396, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1971}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:21.748 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:22.419 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:22.420 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38416/66690 objects degraded (57.604%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":614,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":33345,"data_bytes":136532745,"bytes_used":1755860992,"bytes_avail":288154431488,"bytes_total":289910292480,"degraded_objects":38416,"degraded_total":66690,"degraded_ratio":0.57603838656470241,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":351,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":587},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:22.448 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:22.449 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:22.997 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:22.997 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:23.029 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-15T21:29:22.855025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":136532745,"num_objects":33345,"num_object_clones":0,"num_object_copies":66690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38416,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33345,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":257114,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11059,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1714708,"kb_used_data":466008,"kb_used_omap":12,"kb_used_meta":1248627,"kb_avail":281400812,"statfs":{"total":289910292480,"available":288154431488,"internally_reserved":0,"allocated":477192192,"data_stored":476303432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1278594051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,8,2,7,1,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19140608,"num_objects":-4673,"num_object_clones":0,"num_object_copies":-9346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4673,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004216"},"pg_stats":[{"pgid":"2.f","version":"47'19001","reported_seq":19083,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.910145+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:18.910145+0000","last_peered":"2024-09-15T21:29:18.910145+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:18.910145+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"44'9000","ondisk_log_start":"44'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8810496,"num_objects":2151,"num_object_clones":0,"num_object_copies":4302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19001,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1981}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18592","reported_seq":18703,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:19.489199+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:19.489199+0000","last_peered":"2024-09-15T21:29:19.489199+0000","last_clean":"2024-09-15T21:29:19.489199+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:19.489199+0000","last_undegraded":"2024-09-15T21:29:19.489199+0000","last_fullsized":"2024-09-15T21:29:19.489199+0000","mapping_epoch":35,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8528838,"num_objects":2084,"num_object_clones":0,"num_object_copies":4168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2084,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16873,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19015","reported_seq":24766,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:19.501470+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:19.501470+0000","last_peered":"2024-09-15T21:29:19.501470+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:19.501470+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8560664,"num_objects":2091,"num_object_clones":0,"num_object_copies":4182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1459,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2091,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17115,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2809,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":632}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18259","reported_seq":18336,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:19.506998+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:19.506998+0000","last_peered":"2024-09-15T21:29:19.506998+0000","last_clean":"2024-09-15T21:29:19.506998+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:19.506998+0000","last_undegraded":"2024-09-15T21:29:19.506998+0000","last_fullsized":"2024-09-15T21:29:19.506998+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8417280,"num_objects":2055,"num_object_clones":0,"num_object_copies":4110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18259,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18432","reported_seq":18571,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:18.908655+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:18.908655+0000","last_peered":"2024-09-15T21:29:18.908655+0000","last_clean":"2024-09-15T21:29:18.908655+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:18.908655+0000","last_undegraded":"2024-09-15T21:29:18.908655+0000","last_fullsized":"2024-09-15T21:29:18.908655+0000","mapping_epoch":34,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8491491,"num_objects":2074,"num_object_clones":0,"num_object_copies":4148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14887,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18536","reported_seq":18675,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.944132+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:18.944132+0000","last_peered":"2024-09-15T21:29:18.944132+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:18.944132+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8597987,"num_objects":2100,"num_object_clones":0,"num_object_copies":4200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2100,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15212,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1976},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18603","reported_seq":18745,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.984085+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:18.984085+0000","last_peered":"2024-09-15T21:29:18.984085+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:18.984085+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8467398,"num_objects":2069,"num_object_clones":0,"num_object_copies":4138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14914,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2092},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18802","reported_seq":18937,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:18.939208+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:18.939208+0000","last_peered":"2024-09-15T21:29:18.939208+0000","last_clean":"2024-09-15T21:29:18.939208+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:18.939208+0000","last_undegraded":"2024-09-15T21:29:18.939208+0000","last_fullsized":"2024-09-15T21:29:18.939208+0000","mapping_epoch":34,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8462336,"num_objects":2066,"num_object_clones":0,"num_object_copies":4132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15574,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18498","reported_seq":18631,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:19.477196+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:19.477196+0000","last_peered":"2024-09-15T21:29:19.477196+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:19.477196+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8253923,"num_objects":2016,"num_object_clones":0,"num_object_copies":4032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2016,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16430,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2071}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18347","reported_seq":18483,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.972467+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:18.972467+0000","last_peered":"2024-09-15T21:29:18.972467+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:18.972467+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8744960,"num_objects":2135,"num_object_clones":0,"num_object_copies":4270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15020,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1885}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18581","reported_seq":18705,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:19.415333+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:19.415333+0000","last_peered":"2024-09-15T21:29:19.415333+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:19.415333+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8446435,"num_objects":2063,"num_object_clones":0,"num_object_copies":4126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2063,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16440,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2027}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18582","reported_seq":27046,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:19.418557+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:19.418557+0000","last_peered":"2024-09-15T21:29:19.418557+0000","last_clean":"2024-09-15T21:29:19.418557+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:19.418557+0000","last_undegraded":"2024-09-15T21:29:19.418557+0000","last_fullsized":"2024-09-15T21:29:19.418557+0000","mapping_epoch":44,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8101888,"num_objects":1978,"num_object_clones":0,"num_object_copies":3956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16275,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18571","reported_seq":18710,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:19.021322+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:19.021322+0000","last_peered":"2024-09-15T21:29:19.021322+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:19.021322+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8663040,"num_objects":2115,"num_object_clones":0,"num_object_copies":4230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15515,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1979}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18488","reported_seq":18627,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.934126+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:18.934126+0000","last_peered":"2024-09-15T21:29:18.934126+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:18.934126+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8609792,"num_objects":2102,"num_object_clones":0,"num_object_copies":4204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15397,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2037}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18320","reported_seq":26636,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:20.506107+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:20.506107+0000","last_peered":"2024-09-15T21:29:20.506107+0000","last_clean":"2024-09-15T21:29:20.506107+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:20.506107+0000","last_undegraded":"2024-09-15T21:29:20.506107+0000","last_fullsized":"2024-09-15T21:29:20.506107+0000","mapping_epoch":46,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8470528,"num_objects":2068,"num_object_clones":0,"num_object_copies":4136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14804,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18692","reported_seq":18837,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:18.932378+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:18.932378+0000","last_peered":"2024-09-15T21:29:18.932378+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:18.932378+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8905670,"num_objects":2176,"num_object_clones":0,"num_object_copies":4352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2176,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15396,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1971},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":136532726,"num_objects":33343,"num_object_clones":0,"num_object_copies":66686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38416,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33343,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":257112,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11058,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":474017792,"data_stored":473872354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544388,"kb_used_data":197556,"kb_used_omap":4,"kb_used_meta":346811,"kb_avail":93827452,"statfs":{"total":96636764160,"available":96079310848,"internally_reserved":0,"allocated":202297344,"data_stored":202012086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":355135145},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":619544,"kb_used_data":168636,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93752296,"statfs":{"total":96636764160,"available":96002351104,"internally_reserved":0,"allocated":172683264,"data_stored":172398001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,3,1,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550776,"kb_used_data":99816,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93821064,"statfs":{"total":96636764160,"available":96072769536,"internally_reserved":0,"allocated":102211584,"data_stored":101893345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,0,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":171487232,"data_stored":171451102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100831232,"data_stored":100762126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":201699328,"data_stored":201659126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:26.031 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:26.570 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:26.573 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:26.592 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":348,"stamp":"2024-09-15T21:29:24.855570+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":127623945,"num_objects":31170,"num_object_clones":0,"num_object_copies":62340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31170,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":259289,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11189,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160797,"ondisk_log_size":160797,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1699160,"kb_used_data":454624,"kb_used_omap":12,"kb_used_meta":1244467,"kb_avail":281416360,"statfs":{"total":289910292480,"available":288170352640,"internally_reserved":0,"allocated":465534976,"data_stored":464646216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1274334211},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,2,4,3,8,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28049408,"num_objects":-6848,"num_object_clones":0,"num_object_copies":-13696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-373,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6848,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6848,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":373,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004084"},"pg_stats":[{"pgid":"2.f","version":"47'19175","reported_seq":19257,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.021202+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:24.021202+0000","last_peered":"2024-09-15T21:29:24.021202+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:24.021202+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8097792,"num_objects":1977,"num_object_clones":0,"num_object_copies":3954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19175,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2155}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18740","reported_seq":18851,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:24.431486+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:24.431486+0000","last_peered":"2024-09-15T21:29:24.431486+0000","last_clean":"2024-09-15T21:29:24.431486+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:24.431486+0000","last_undegraded":"2024-09-15T21:29:24.431486+0000","last_fullsized":"2024-09-15T21:29:24.431486+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7922630,"num_objects":1936,"num_object_clones":0,"num_object_copies":3872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17021,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19143","reported_seq":25154,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.515062+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:24.515062+0000","last_peered":"2024-09-15T21:29:24.515062+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:24.515062+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8036376,"num_objects":1963,"num_object_clones":0,"num_object_copies":3926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17243,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2939,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":634}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18389","reported_seq":18466,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:24.512059+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:24.512059+0000","last_peered":"2024-09-15T21:29:24.512059+0000","last_clean":"2024-09-15T21:29:24.512059+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:24.512059+0000","last_undegraded":"2024-09-15T21:29:24.512059+0000","last_fullsized":"2024-09-15T21:29:24.512059+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7884800,"num_objects":1925,"num_object_clones":0,"num_object_copies":3850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18389,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18567","reported_seq":18706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:23.946517+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:23.946517+0000","last_peered":"2024-09-15T21:29:23.946517+0000","last_clean":"2024-09-15T21:29:23.946517+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:23.946517+0000","last_undegraded":"2024-09-15T21:29:23.946517+0000","last_fullsized":"2024-09-15T21:29:23.946517+0000","mapping_epoch":34,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7938531,"num_objects":1939,"num_object_clones":0,"num_object_copies":3878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15022,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18682","reported_seq":18823,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.025089+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:24.025089+0000","last_peered":"2024-09-15T21:29:24.025089+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:24.025089+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7999971,"num_objects":1954,"num_object_clones":0,"num_object_copies":3908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15358,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2122},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18742","reported_seq":18882,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:23.971305+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:23.971305+0000","last_peered":"2024-09-15T21:29:23.971305+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:23.971305+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7898054,"num_objects":1930,"num_object_clones":0,"num_object_copies":3860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15053,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2231},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18942","reported_seq":19077,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:23.996266+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:23.996266+0000","last_peered":"2024-09-15T21:29:23.996266+0000","last_clean":"2024-09-15T21:29:23.996266+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:23.996266+0000","last_undegraded":"2024-09-15T21:29:23.996266+0000","last_fullsized":"2024-09-15T21:29:23.996266+0000","mapping_epoch":34,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7888896,"num_objects":1926,"num_object_clones":0,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15714,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18641","reported_seq":18773,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.527739+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:24.527739+0000","last_peered":"2024-09-15T21:29:24.527739+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:24.527739+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7668195,"num_objects":1873,"num_object_clones":0,"num_object_copies":3746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1873,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16573,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2214}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18489","reported_seq":18626,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:23.900396+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:23.900396+0000","last_peered":"2024-09-15T21:29:23.900396+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:23.900396+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8163328,"num_objects":1993,"num_object_clones":0,"num_object_copies":3986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15162,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2027}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18730","reported_seq":18855,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.482561+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:24.482561+0000","last_peered":"2024-09-15T21:29:24.482561+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:24.482561+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7836131,"num_objects":1914,"num_object_clones":0,"num_object_copies":3828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1914,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16589,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2176}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18717","reported_seq":27182,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:24.454662+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:24.454662+0000","last_peered":"2024-09-15T21:29:24.454662+0000","last_clean":"2024-09-15T21:29:24.454662+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:24.454662+0000","last_undegraded":"2024-09-15T21:29:24.454662+0000","last_fullsized":"2024-09-15T21:29:24.454662+0000","mapping_epoch":44,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7548928,"num_objects":1843,"num_object_clones":0,"num_object_copies":3686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16410,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18729","reported_seq":18869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.020997+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:24.020997+0000","last_peered":"2024-09-15T21:29:24.020997+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:24.020997+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8015872,"num_objects":1957,"num_object_clones":0,"num_object_copies":3914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15673,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18642","reported_seq":18781,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.006202+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:24.006202+0000","last_peered":"2024-09-15T21:29:24.006202+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:24.006202+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7979008,"num_objects":1948,"num_object_clones":0,"num_object_copies":3896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15551,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2191}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18320","reported_seq":26636,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:20.506107+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:20.506107+0000","last_peered":"2024-09-15T21:29:20.506107+0000","last_clean":"2024-09-15T21:29:20.506107+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:20.506107+0000","last_undegraded":"2024-09-15T21:29:20.506107+0000","last_fullsized":"2024-09-15T21:29:20.506107+0000","mapping_epoch":46,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8470528,"num_objects":2068,"num_object_clones":0,"num_object_copies":4136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14804,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18846","reported_seq":18986,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.060989+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:24.060989+0000","last_peered":"2024-09-15T21:29:24.060989+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:24.060989+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8274886,"num_objects":2022,"num_object_clones":0,"num_object_copies":4044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15550,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2125},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":127623926,"num_objects":31168,"num_object_clones":0,"num_object_copies":62336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31168,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":259287,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11188,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":462184448,"data_stored":462039010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160794,"ondisk_log_size":160794,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527520,"kb_used_data":193164,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93844320,"statfs":{"total":96636764160,"available":96096583680,"internally_reserved":0,"allocated":197799936,"data_stored":197514678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620864,"kb_used_data":161644,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93750976,"statfs":{"total":96636764160,"available":96000999424,"internally_reserved":0,"allocated":165523456,"data_stored":165238193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,2,0,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550776,"kb_used_data":99816,"kb_used_omap":4,"kb_used_meta":450939,"kb_avail":93821064,"statfs":{"total":96636764160,"available":96072769536,"internally_reserved":0,"allocated":102211584,"data_stored":101893345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":461762223},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,0,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164188160,"data_stored":164152030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":100831232,"data_stored":100762126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":197165056,"data_stored":197124854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:26.593 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:27.136 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:27.139 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:27.157 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-15T21:29:26.856384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":126931721,"num_objects":31001,"num_object_clones":0,"num_object_copies":62002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31001,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":259458,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11189,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1696264,"kb_used_data":447568,"kb_used_omap":12,"kb_used_meta":1248627,"kb_avail":281419256,"statfs":{"total":289910292480,"available":288173318144,"internally_reserved":0,"allocated":458309632,"data_stored":457420872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1278594051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,6,0,7,3,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19668992,"num_objects":-4802,"num_object_clones":0,"num_object_copies":-9604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-253,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4802,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004376"},"pg_stats":[{"pgid":"2.f","version":"47'19175","reported_seq":19257,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.021202+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:24.021202+0000","last_peered":"2024-09-15T21:29:24.021202+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:24.021202+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8097792,"num_objects":1977,"num_object_clones":0,"num_object_copies":3954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19175,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2155}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18740","reported_seq":18851,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:24.431486+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:24.431486+0000","last_peered":"2024-09-15T21:29:24.431486+0000","last_clean":"2024-09-15T21:29:24.431486+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:24.431486+0000","last_undegraded":"2024-09-15T21:29:24.431486+0000","last_fullsized":"2024-09-15T21:29:24.431486+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7922630,"num_objects":1936,"num_object_clones":0,"num_object_copies":3872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17021,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19143","reported_seq":25154,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.515062+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:24.515062+0000","last_peered":"2024-09-15T21:29:24.515062+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:24.515062+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8036376,"num_objects":1963,"num_object_clones":0,"num_object_copies":3926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17243,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2939,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":634}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18389","reported_seq":18466,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:24.512059+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:24.512059+0000","last_peered":"2024-09-15T21:29:24.512059+0000","last_clean":"2024-09-15T21:29:24.512059+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:24.512059+0000","last_undegraded":"2024-09-15T21:29:24.512059+0000","last_fullsized":"2024-09-15T21:29:24.512059+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7884800,"num_objects":1925,"num_object_clones":0,"num_object_copies":3850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18389,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18567","reported_seq":18706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:23.946517+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:23.946517+0000","last_peered":"2024-09-15T21:29:23.946517+0000","last_clean":"2024-09-15T21:29:23.946517+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:23.946517+0000","last_undegraded":"2024-09-15T21:29:23.946517+0000","last_fullsized":"2024-09-15T21:29:23.946517+0000","mapping_epoch":34,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7938531,"num_objects":1939,"num_object_clones":0,"num_object_copies":3878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15022,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18682","reported_seq":18823,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.025089+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:24.025089+0000","last_peered":"2024-09-15T21:29:24.025089+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:24.025089+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7999971,"num_objects":1954,"num_object_clones":0,"num_object_copies":3908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15358,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2122},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18742","reported_seq":18882,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:23.971305+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:23.971305+0000","last_peered":"2024-09-15T21:29:23.971305+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:23.971305+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7898054,"num_objects":1930,"num_object_clones":0,"num_object_copies":3860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15053,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2231},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18942","reported_seq":19077,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:23.996266+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:23.996266+0000","last_peered":"2024-09-15T21:29:23.996266+0000","last_clean":"2024-09-15T21:29:23.996266+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:23.996266+0000","last_undegraded":"2024-09-15T21:29:23.996266+0000","last_fullsized":"2024-09-15T21:29:23.996266+0000","mapping_epoch":34,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7888896,"num_objects":1926,"num_object_clones":0,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15714,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18641","reported_seq":18773,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.527739+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:24.527739+0000","last_peered":"2024-09-15T21:29:24.527739+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:24.527739+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7668195,"num_objects":1873,"num_object_clones":0,"num_object_copies":3746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1873,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16573,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2214}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18489","reported_seq":18626,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:23.900396+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:23.900396+0000","last_peered":"2024-09-15T21:29:23.900396+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:23.900396+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8163328,"num_objects":1993,"num_object_clones":0,"num_object_copies":3986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15162,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2027}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18730","reported_seq":18855,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.482561+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:24.482561+0000","last_peered":"2024-09-15T21:29:24.482561+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:24.482561+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7836131,"num_objects":1914,"num_object_clones":0,"num_object_copies":3828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1914,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16589,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2176}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18717","reported_seq":27182,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:24.454662+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:24.454662+0000","last_peered":"2024-09-15T21:29:24.454662+0000","last_clean":"2024-09-15T21:29:24.454662+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:24.454662+0000","last_undegraded":"2024-09-15T21:29:24.454662+0000","last_fullsized":"2024-09-15T21:29:24.454662+0000","mapping_epoch":44,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7548928,"num_objects":1843,"num_object_clones":0,"num_object_copies":3686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16410,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18729","reported_seq":18869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.020997+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:24.020997+0000","last_peered":"2024-09-15T21:29:24.020997+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:24.020997+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8015872,"num_objects":1957,"num_object_clones":0,"num_object_copies":3914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15673,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18642","reported_seq":18781,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.006202+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:24.006202+0000","last_peered":"2024-09-15T21:29:24.006202+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:24.006202+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7979008,"num_objects":1948,"num_object_clones":0,"num_object_copies":3896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15551,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2191}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18489","reported_seq":26805,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:25.576067+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:25.576067+0000","last_peered":"2024-09-15T21:29:25.576067+0000","last_clean":"2024-09-15T21:29:25.576067+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:25.576067+0000","last_undegraded":"2024-09-15T21:29:25.576067+0000","last_fullsized":"2024-09-15T21:29:25.576067+0000","mapping_epoch":46,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7778304,"num_objects":1899,"num_object_clones":0,"num_object_copies":3798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14973,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18846","reported_seq":18986,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.060989+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:24.060989+0000","last_peered":"2024-09-15T21:29:24.060989+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:24.060989+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8274886,"num_objects":2022,"num_object_clones":0,"num_object_copies":4044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15550,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2125},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":126931702,"num_objects":30999,"num_object_clones":0,"num_object_copies":61998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30999,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":259456,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11188,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454832128,"data_stored":454686690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527520,"kb_used_data":193164,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93844320,"statfs":{"total":96636764160,"available":96096583680,"internally_reserved":0,"allocated":197799936,"data_stored":197514678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620864,"kb_used_data":161644,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93750976,"statfs":{"total":96636764160,"available":96000999424,"internally_reserved":0,"allocated":165523456,"data_stored":165238193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,2,0,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547880,"kb_used_data":92760,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93823960,"statfs":{"total":96636764160,"available":96075735040,"internally_reserved":0,"allocated":94986240,"data_stored":94668001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,3,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164188160,"data_stored":164152030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":93478912,"data_stored":93409806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":197165056,"data_stored":197124854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:27.159 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:29:27.159 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19175", 'reported_seq': 19257, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:24.021202+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:29:24.021202+0000', 'last_peered': '2024-09-15T21:29:24.021202+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:29:24.021202+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8097792, 'num_objects': 1977, 'num_object_clones': 0, 'num_object_copies': 3954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1977, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19175, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2155}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:27.159 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:29:27.159 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'19143", 'reported_seq': 25154, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:24.515062+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:29:24.515062+0000', 'last_peered': '2024-09-15T21:29:24.515062+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:29:24.515062+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8036376, 'num_objects': 1963, 'num_object_clones': 0, 'num_object_copies': 3926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1329, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1963, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17243, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2939, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 634}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:27.159 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:29:27.159 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18682", 'reported_seq': 18823, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:24.025089+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:29:24.025089+0000', 'last_peered': '2024-09-15T21:29:24.025089+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:29:24.025089+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7999971, 'num_objects': 1954, 'num_object_clones': 0, 'num_object_copies': 3908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1954, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15358, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2122}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:27.159 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:27.159 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18742", 'reported_seq': 18882, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:23.971305+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:29:23.971305+0000', 'last_peered': '2024-09-15T21:29:23.971305+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:29:23.971305+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7898054, 'num_objects': 1930, 'num_object_clones': 0, 'num_object_copies': 3860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1930, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15053, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2231}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:27.160 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:29:27.160 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'18641", 'reported_seq': 18773, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:24.527739+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:29:24.527739+0000', 'last_peered': '2024-09-15T21:29:24.527739+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:29:24.527739+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7668195, 'num_objects': 1873, 'num_object_clones': 0, 'num_object_copies': 3746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1873, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16573, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2214}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:27.160 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:27.160 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18489", 'reported_seq': 18626, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:23.900396+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:29:23.900396+0000', 'last_peered': '2024-09-15T21:29:23.900396+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:29:23.900396+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'8400", 'ondisk_log_start': "43'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8163328, 'num_objects': 1993, 'num_object_clones': 0, 'num_object_copies': 3986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1993, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15162, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2027}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:27.160 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:29:27.160 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18730", 'reported_seq': 18855, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:24.482561+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:29:24.482561+0000', 'last_peered': '2024-09-15T21:29:24.482561+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:29:24.482561+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7836131, 'num_objects': 1914, 'num_object_clones': 0, 'num_object_copies': 3828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1914, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16589, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2176}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:27.160 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:29:27.160 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18729", 'reported_seq': 18869, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:24.020997+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:29:24.020997+0000', 'last_peered': '2024-09-15T21:29:24.020997+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:29:24.020997+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8015872, 'num_objects': 1957, 'num_object_clones': 0, 'num_object_copies': 3914, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1957, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15673, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2137}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:27.160 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:29:27.161 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'18642", 'reported_seq': 18781, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:24.006202+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:29:24.006202+0000', 'last_peered': '2024-09-15T21:29:24.006202+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:29:24.006202+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7979008, 'num_objects': 1948, 'num_object_clones': 0, 'num_object_copies': 3896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1948, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15551, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2191}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:27.161 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:29:27.161 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'18846", 'reported_seq': 18986, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:24.060989+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:29:24.060989+0000', 'last_peered': '2024-09-15T21:29:24.060989+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:29:24.060989+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8274886, 'num_objects': 2022, 'num_object_clones': 0, 'num_object_copies': 4044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2022, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15550, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2125}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:27.161 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:27.825 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:27.825 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38286/62002 objects degraded (61.750%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":619,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":31001,"data_bytes":126931721,"bytes_used":1736974336,"bytes_avail":288173318144,"bytes_total":289910292480,"degraded_objects":38286,"degraded_total":62002,"degraded_ratio":0.61749620979968389,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":253,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":400},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:27.852 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:27.852 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:28.427 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:28.430 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:28.452 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-15T21:29:26.856384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":126931721,"num_objects":31001,"num_object_clones":0,"num_object_copies":62002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31001,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":259458,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11189,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1696264,"kb_used_data":447568,"kb_used_omap":12,"kb_used_meta":1248627,"kb_avail":281419256,"statfs":{"total":289910292480,"available":288173318144,"internally_reserved":0,"allocated":458309632,"data_stored":457420872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1278594051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,6,0,7,3,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19668992,"num_objects":-4802,"num_object_clones":0,"num_object_copies":-9604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-253,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4802,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":253,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004376"},"pg_stats":[{"pgid":"2.f","version":"47'19175","reported_seq":19257,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.021202+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:24.021202+0000","last_peered":"2024-09-15T21:29:24.021202+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:24.021202+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8097792,"num_objects":1977,"num_object_clones":0,"num_object_copies":3954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19175,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2155}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18740","reported_seq":18851,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:24.431486+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:24.431486+0000","last_peered":"2024-09-15T21:29:24.431486+0000","last_clean":"2024-09-15T21:29:24.431486+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:24.431486+0000","last_undegraded":"2024-09-15T21:29:24.431486+0000","last_fullsized":"2024-09-15T21:29:24.431486+0000","mapping_epoch":35,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7922630,"num_objects":1936,"num_object_clones":0,"num_object_copies":3872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1936,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17021,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19143","reported_seq":25154,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.515062+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:24.515062+0000","last_peered":"2024-09-15T21:29:24.515062+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:24.515062+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8036376,"num_objects":1963,"num_object_clones":0,"num_object_copies":3926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1329,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1963,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17243,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2939,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":634}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18389","reported_seq":18466,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:24.512059+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:24.512059+0000","last_peered":"2024-09-15T21:29:24.512059+0000","last_clean":"2024-09-15T21:29:24.512059+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:24.512059+0000","last_undegraded":"2024-09-15T21:29:24.512059+0000","last_fullsized":"2024-09-15T21:29:24.512059+0000","mapping_epoch":35,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7884800,"num_objects":1925,"num_object_clones":0,"num_object_copies":3850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18389,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18567","reported_seq":18706,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:23.946517+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:23.946517+0000","last_peered":"2024-09-15T21:29:23.946517+0000","last_clean":"2024-09-15T21:29:23.946517+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:23.946517+0000","last_undegraded":"2024-09-15T21:29:23.946517+0000","last_fullsized":"2024-09-15T21:29:23.946517+0000","mapping_epoch":34,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7938531,"num_objects":1939,"num_object_clones":0,"num_object_copies":3878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15022,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18682","reported_seq":18823,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.025089+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:24.025089+0000","last_peered":"2024-09-15T21:29:24.025089+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:24.025089+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7999971,"num_objects":1954,"num_object_clones":0,"num_object_copies":3908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1954,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15358,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2122},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18742","reported_seq":18882,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:23.971305+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:23.971305+0000","last_peered":"2024-09-15T21:29:23.971305+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:23.971305+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7898054,"num_objects":1930,"num_object_clones":0,"num_object_copies":3860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15053,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2231},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'18942","reported_seq":19077,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:23.996266+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:23.996266+0000","last_peered":"2024-09-15T21:29:23.996266+0000","last_clean":"2024-09-15T21:29:23.996266+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:23.996266+0000","last_undegraded":"2024-09-15T21:29:23.996266+0000","last_fullsized":"2024-09-15T21:29:23.996266+0000","mapping_epoch":34,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7888896,"num_objects":1926,"num_object_clones":0,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15714,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18641","reported_seq":18773,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.527739+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:24.527739+0000","last_peered":"2024-09-15T21:29:24.527739+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:24.527739+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7668195,"num_objects":1873,"num_object_clones":0,"num_object_copies":3746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1873,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16573,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2214}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18489","reported_seq":18626,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:23.900396+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:23.900396+0000","last_peered":"2024-09-15T21:29:23.900396+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:23.900396+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8163328,"num_objects":1993,"num_object_clones":0,"num_object_copies":3986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15162,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2027}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18730","reported_seq":18855,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.482561+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:24.482561+0000","last_peered":"2024-09-15T21:29:24.482561+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:24.482561+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7836131,"num_objects":1914,"num_object_clones":0,"num_object_copies":3828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1914,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16589,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2176}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18717","reported_seq":27182,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:24.454662+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:24.454662+0000","last_peered":"2024-09-15T21:29:24.454662+0000","last_clean":"2024-09-15T21:29:24.454662+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:24.454662+0000","last_undegraded":"2024-09-15T21:29:24.454662+0000","last_fullsized":"2024-09-15T21:29:24.454662+0000","mapping_epoch":44,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7548928,"num_objects":1843,"num_object_clones":0,"num_object_copies":3686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16410,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18729","reported_seq":18869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.020997+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:24.020997+0000","last_peered":"2024-09-15T21:29:24.020997+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:24.020997+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8015872,"num_objects":1957,"num_object_clones":0,"num_object_copies":3914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15673,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18642","reported_seq":18781,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.006202+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:24.006202+0000","last_peered":"2024-09-15T21:29:24.006202+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:24.006202+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7979008,"num_objects":1948,"num_object_clones":0,"num_object_copies":3896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15551,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2191}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18489","reported_seq":26805,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:25.576067+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:25.576067+0000","last_peered":"2024-09-15T21:29:25.576067+0000","last_clean":"2024-09-15T21:29:25.576067+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:25.576067+0000","last_undegraded":"2024-09-15T21:29:25.576067+0000","last_fullsized":"2024-09-15T21:29:25.576067+0000","mapping_epoch":46,"log_start":"43'8400","ondisk_log_start":"43'8400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7778304,"num_objects":1899,"num_object_clones":0,"num_object_copies":3798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14973,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'18846","reported_seq":18986,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:24.060989+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:24.060989+0000","last_peered":"2024-09-15T21:29:24.060989+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:24.060989+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8274886,"num_objects":2022,"num_object_clones":0,"num_object_copies":4044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15550,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2125},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":126931702,"num_objects":30999,"num_object_clones":0,"num_object_copies":61998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38286,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30999,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":259456,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11188,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454832128,"data_stored":454686690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527520,"kb_used_data":193164,"kb_used_omap":4,"kb_used_meta":334331,"kb_avail":93844320,"statfs":{"total":96636764160,"available":96096583680,"internally_reserved":0,"allocated":197799936,"data_stored":197514678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":342355625},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":620864,"kb_used_data":161644,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93750976,"statfs":{"total":96636764160,"available":96000999424,"internally_reserved":0,"allocated":165523456,"data_stored":165238193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,0,2,0,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547880,"kb_used_data":92760,"kb_used_omap":4,"kb_used_meta":455099,"kb_avail":93823960,"statfs":{"total":96636764160,"available":96075735040,"internally_reserved":0,"allocated":94986240,"data_stored":94668001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":466022063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,3,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":164188160,"data_stored":164152030,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":93478912,"data_stored":93409806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":197165056,"data_stored":197124854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:31.454 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:32.026 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:32.026 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:32.056 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-09-15T21:29:30.857915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117302025,"num_objects":28650,"num_object_clones":0,"num_object_copies":57300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28650,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":261809,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11329,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161017,"ondisk_log_size":161017,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1679088,"kb_used_data":428980,"kb_used_omap":12,"kb_used_meta":1250035,"kb_avail":281436432,"statfs":{"total":289910292480,"available":288190906368,"internally_reserved":0,"allocated":439275520,"data_stored":438386760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1280035843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,7,4,5,6,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28696576,"num_objects":-7006,"num_object_clones":0,"num_object_copies":-14012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7006,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004377"},"pg_stats":[{"pgid":"2.f","version":"47'19339","reported_seq":19421,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.021283+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:29.021283+0000","last_peered":"2024-09-15T21:29:29.021283+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:29.021283+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7426048,"num_objects":1813,"num_object_clones":0,"num_object_copies":3626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19339,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2319}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18897","reported_seq":19008,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.534957+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:29.534957+0000","last_peered":"2024-09-15T21:29:29.534957+0000","last_clean":"2024-09-15T21:29:29.534957+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:29.534957+0000","last_undegraded":"2024-09-15T21:29:29.534957+0000","last_fullsized":"2024-09-15T21:29:29.534957+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7279558,"num_objects":1779,"num_object_clones":0,"num_object_copies":3558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1779,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17178,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19287","reported_seq":25579,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.562434+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:29.562434+0000","last_peered":"2024-09-15T21:29:29.562434+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:29.562434+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7446552,"num_objects":1819,"num_object_clones":0,"num_object_copies":3638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1819,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17387,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3079,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":630}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18548","reported_seq":18625,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.495684+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:29.495684+0000","last_peered":"2024-09-15T21:29:29.495684+0000","last_clean":"2024-09-15T21:29:29.495684+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:29.495684+0000","last_undegraded":"2024-09-15T21:29:29.495684+0000","last_fullsized":"2024-09-15T21:29:29.495684+0000","mapping_epoch":35,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7233536,"num_objects":1766,"num_object_clones":0,"num_object_copies":3532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18548,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18696","reported_seq":18835,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.010209+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:29.010209+0000","last_peered":"2024-09-15T21:29:29.010209+0000","last_clean":"2024-09-15T21:29:29.010209+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:29.010209+0000","last_undegraded":"2024-09-15T21:29:29.010209+0000","last_fullsized":"2024-09-15T21:29:29.010209+0000","mapping_epoch":34,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7410147,"num_objects":1810,"num_object_clones":0,"num_object_copies":3620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15151,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18819","reported_seq":18961,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:28.988904+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:28.988904+0000","last_peered":"2024-09-15T21:29:28.988904+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:28.988904+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7438819,"num_objects":1817,"num_object_clones":0,"num_object_copies":3634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1817,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15495,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2259},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18897","reported_seq":19036,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:28.979421+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:28.979421+0000","last_peered":"2024-09-15T21:29:28.979421+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:28.979421+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7263174,"num_objects":1775,"num_object_clones":0,"num_object_copies":3550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15208,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2386},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19083","reported_seq":19218,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.047717+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:29.047717+0000","last_peered":"2024-09-15T21:29:29.047717+0000","last_clean":"2024-09-15T21:29:29.047717+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:29.047717+0000","last_undegraded":"2024-09-15T21:29:29.047717+0000","last_fullsized":"2024-09-15T21:29:29.047717+0000","mapping_epoch":34,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7311360,"num_objects":1785,"num_object_clones":0,"num_object_copies":3570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15855,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18792","reported_seq":18925,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.485422+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:29.485422+0000","last_peered":"2024-09-15T21:29:29.485422+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:29.485422+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7049699,"num_objects":1722,"num_object_clones":0,"num_object_copies":3444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16724,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2365}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18631","reported_seq":18768,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.006392+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:29.006392+0000","last_peered":"2024-09-15T21:29:29.006392+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:29.006392+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7581696,"num_objects":1851,"num_object_clones":0,"num_object_copies":3702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15304,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2169}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18887","reported_seq":19012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.502226+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:29.502226+0000","last_peered":"2024-09-15T21:29:29.502226+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:29.502226+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"44'8800","ondisk_log_start":"44'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193059,"num_objects":1757,"num_object_clones":0,"num_object_copies":3514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16746,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2333}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18869","reported_seq":27334,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.502430+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:29.502430+0000","last_peered":"2024-09-15T21:29:29.502430+0000","last_clean":"2024-09-15T21:29:29.502430+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:29.502430+0000","last_undegraded":"2024-09-15T21:29:29.502430+0000","last_fullsized":"2024-09-15T21:29:29.502430+0000","mapping_epoch":44,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6926336,"num_objects":1691,"num_object_clones":0,"num_object_copies":3382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16562,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18875","reported_seq":19013,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:28.988466+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:28.988466+0000","last_peered":"2024-09-15T21:29:28.988466+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:28.988466+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7417856,"num_objects":1811,"num_object_clones":0,"num_object_copies":3622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15819,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18768","reported_seq":18906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.073004+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:29.073004+0000","last_peered":"2024-09-15T21:29:29.073004+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:29.073004+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"44'8700","ondisk_log_start":"44'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7462912,"num_objects":1822,"num_object_clones":0,"num_object_copies":3644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15677,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2317}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18622","reported_seq":26938,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:30.514091+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:30.514091+0000","last_peered":"2024-09-15T21:29:30.514091+0000","last_clean":"2024-09-15T21:29:30.514091+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:30.514091+0000","last_undegraded":"2024-09-15T21:29:30.514091+0000","last_fullsized":"2024-09-15T21:29:30.514091+0000","mapping_epoch":46,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7233536,"num_objects":1766,"num_object_clones":0,"num_object_copies":3532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15106,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19004","reported_seq":19149,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.063153+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:29.063153+0000","last_peered":"2024-09-15T21:29:29.063153+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:29.063153+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7627718,"num_objects":1864,"num_object_clones":0,"num_object_copies":3728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1864,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15708,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2283},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":117302006,"num_objects":28648,"num_object_clones":0,"num_object_copies":57296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28648,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":261807,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11328,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":435671040,"data_stored":435525602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161014,"ondisk_log_size":161014,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526932,"kb_used_data":188420,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93844908,"statfs":{"total":96636764160,"available":96097185792,"internally_reserved":0,"allocated":192942080,"data_stored":192656822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601508,"kb_used_data":154760,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93770332,"statfs":{"total":96636764160,"available":96020819968,"internally_reserved":0,"allocated":158474240,"data_stored":158188977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457436843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,3,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659236,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550648,"kb_used_data":85800,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93821192,"statfs":{"total":96636764160,"available":96072900608,"internally_reserved":0,"allocated":87859200,"data_stored":87540961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,3,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157048832,"data_stored":157012702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86278144,"data_stored":86209038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192344064,"data_stored":192303862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:32.058 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:32.628 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:32.632 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:32.657 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":351,"stamp":"2024-09-15T21:29:30.857915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117302025,"num_objects":28650,"num_object_clones":0,"num_object_copies":57300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28650,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":261809,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11329,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161017,"ondisk_log_size":161017,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1679088,"kb_used_data":428980,"kb_used_omap":12,"kb_used_meta":1250035,"kb_avail":281436432,"statfs":{"total":289910292480,"available":288190906368,"internally_reserved":0,"allocated":439275520,"data_stored":438386760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1280035843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,7,4,5,6,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28696576,"num_objects":-7006,"num_object_clones":0,"num_object_copies":-14012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7006,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004377"},"pg_stats":[{"pgid":"2.f","version":"47'19339","reported_seq":19421,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.021283+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:29.021283+0000","last_peered":"2024-09-15T21:29:29.021283+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:29.021283+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7426048,"num_objects":1813,"num_object_clones":0,"num_object_copies":3626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19339,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2319}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18897","reported_seq":19008,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.534957+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:29.534957+0000","last_peered":"2024-09-15T21:29:29.534957+0000","last_clean":"2024-09-15T21:29:29.534957+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:29.534957+0000","last_undegraded":"2024-09-15T21:29:29.534957+0000","last_fullsized":"2024-09-15T21:29:29.534957+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7279558,"num_objects":1779,"num_object_clones":0,"num_object_copies":3558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1779,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17178,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19287","reported_seq":25579,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.562434+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:29.562434+0000","last_peered":"2024-09-15T21:29:29.562434+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:29.562434+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7446552,"num_objects":1819,"num_object_clones":0,"num_object_copies":3638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1819,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17387,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3079,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":630}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18548","reported_seq":18625,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.495684+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:29.495684+0000","last_peered":"2024-09-15T21:29:29.495684+0000","last_clean":"2024-09-15T21:29:29.495684+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:29.495684+0000","last_undegraded":"2024-09-15T21:29:29.495684+0000","last_fullsized":"2024-09-15T21:29:29.495684+0000","mapping_epoch":35,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7233536,"num_objects":1766,"num_object_clones":0,"num_object_copies":3532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18548,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18696","reported_seq":18835,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.010209+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:29.010209+0000","last_peered":"2024-09-15T21:29:29.010209+0000","last_clean":"2024-09-15T21:29:29.010209+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:29.010209+0000","last_undegraded":"2024-09-15T21:29:29.010209+0000","last_fullsized":"2024-09-15T21:29:29.010209+0000","mapping_epoch":34,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7410147,"num_objects":1810,"num_object_clones":0,"num_object_copies":3620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15151,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18819","reported_seq":18961,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:28.988904+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:28.988904+0000","last_peered":"2024-09-15T21:29:28.988904+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:28.988904+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7438819,"num_objects":1817,"num_object_clones":0,"num_object_copies":3634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1817,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15495,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2259},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18897","reported_seq":19036,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:28.979421+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:28.979421+0000","last_peered":"2024-09-15T21:29:28.979421+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:28.979421+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7263174,"num_objects":1775,"num_object_clones":0,"num_object_copies":3550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15208,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2386},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19083","reported_seq":19218,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.047717+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:29.047717+0000","last_peered":"2024-09-15T21:29:29.047717+0000","last_clean":"2024-09-15T21:29:29.047717+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:29.047717+0000","last_undegraded":"2024-09-15T21:29:29.047717+0000","last_fullsized":"2024-09-15T21:29:29.047717+0000","mapping_epoch":34,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7311360,"num_objects":1785,"num_object_clones":0,"num_object_copies":3570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15855,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18792","reported_seq":18925,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.485422+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:29.485422+0000","last_peered":"2024-09-15T21:29:29.485422+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:29.485422+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7049699,"num_objects":1722,"num_object_clones":0,"num_object_copies":3444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16724,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2365}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18631","reported_seq":18768,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.006392+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:29.006392+0000","last_peered":"2024-09-15T21:29:29.006392+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:29.006392+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7581696,"num_objects":1851,"num_object_clones":0,"num_object_copies":3702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15304,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2169}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18887","reported_seq":19012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.502226+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:29.502226+0000","last_peered":"2024-09-15T21:29:29.502226+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:29.502226+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"44'8800","ondisk_log_start":"44'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193059,"num_objects":1757,"num_object_clones":0,"num_object_copies":3514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16746,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2333}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18869","reported_seq":27334,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.502430+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:29.502430+0000","last_peered":"2024-09-15T21:29:29.502430+0000","last_clean":"2024-09-15T21:29:29.502430+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:29.502430+0000","last_undegraded":"2024-09-15T21:29:29.502430+0000","last_fullsized":"2024-09-15T21:29:29.502430+0000","mapping_epoch":44,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6926336,"num_objects":1691,"num_object_clones":0,"num_object_copies":3382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16562,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18875","reported_seq":19013,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:28.988466+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:28.988466+0000","last_peered":"2024-09-15T21:29:28.988466+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:28.988466+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7417856,"num_objects":1811,"num_object_clones":0,"num_object_copies":3622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15819,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18768","reported_seq":18906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.073004+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:29.073004+0000","last_peered":"2024-09-15T21:29:29.073004+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:29.073004+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"44'8700","ondisk_log_start":"44'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7462912,"num_objects":1822,"num_object_clones":0,"num_object_copies":3644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15677,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2317}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18622","reported_seq":26938,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:30.514091+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:30.514091+0000","last_peered":"2024-09-15T21:29:30.514091+0000","last_clean":"2024-09-15T21:29:30.514091+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:30.514091+0000","last_undegraded":"2024-09-15T21:29:30.514091+0000","last_fullsized":"2024-09-15T21:29:30.514091+0000","mapping_epoch":46,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7233536,"num_objects":1766,"num_object_clones":0,"num_object_copies":3532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15106,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19004","reported_seq":19149,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.063153+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:29.063153+0000","last_peered":"2024-09-15T21:29:29.063153+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:29.063153+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7627718,"num_objects":1864,"num_object_clones":0,"num_object_copies":3728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1864,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15708,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2283},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":117302006,"num_objects":28648,"num_object_clones":0,"num_object_copies":57296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28648,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":261807,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11328,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":435671040,"data_stored":435525602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161014,"ondisk_log_size":161014,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526932,"kb_used_data":188420,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93844908,"statfs":{"total":96636764160,"available":96097185792,"internally_reserved":0,"allocated":192942080,"data_stored":192656822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601508,"kb_used_data":154760,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93770332,"statfs":{"total":96636764160,"available":96020819968,"internally_reserved":0,"allocated":158474240,"data_stored":158188977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457436843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,3,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659236,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550648,"kb_used_data":85800,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93821192,"statfs":{"total":96636764160,"available":96072900608,"internally_reserved":0,"allocated":87859200,"data_stored":87540961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,3,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157048832,"data_stored":157012702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86278144,"data_stored":86209038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192344064,"data_stored":192303862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:32.659 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:29:32.659 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19339", 'reported_seq': 19421, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:29.021283+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:29:29.021283+0000', 'last_peered': '2024-09-15T21:29:29.021283+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:29:29.021283+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7426048, 'num_objects': 1813, 'num_object_clones': 0, 'num_object_copies': 3626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1813, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19339, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2319}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:32.659 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:29:32.659 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'19287", 'reported_seq': 25579, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:29.562434+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:29:29.562434+0000', 'last_peered': '2024-09-15T21:29:29.562434+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:29:29.562434+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7446552, 'num_objects': 1819, 'num_object_clones': 0, 'num_object_copies': 3638, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1189, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1819, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17387, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3079, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 630}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:32.659 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:29:32.659 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18819", 'reported_seq': 18961, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:28.988904+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:29:28.988904+0000', 'last_peered': '2024-09-15T21:29:28.988904+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:29:28.988904+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7438819, 'num_objects': 1817, 'num_object_clones': 0, 'num_object_copies': 3634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1817, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15495, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2259}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:32.659 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:32.659 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'18897", 'reported_seq': 19036, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:28.979421+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:29:28.979421+0000', 'last_peered': '2024-09-15T21:29:28.979421+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:29:28.979421+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "43'8800", 'ondisk_log_start': "43'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7263174, 'num_objects': 1775, 'num_object_clones': 0, 'num_object_copies': 3550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1775, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15208, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2386}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:32.659 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:29:32.660 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'18792", 'reported_seq': 18925, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:29.485422+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:29:29.485422+0000', 'last_peered': '2024-09-15T21:29:29.485422+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:29:29.485422+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "43'8700", 'ondisk_log_start': "43'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7049699, 'num_objects': 1722, 'num_object_clones': 0, 'num_object_copies': 3444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1722, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16724, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2365}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:32.660 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:32.660 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18631", 'reported_seq': 18768, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:29.006392+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:29:29.006392+0000', 'last_peered': '2024-09-15T21:29:29.006392+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:29:29.006392+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "43'8600", 'ondisk_log_start': "43'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7581696, 'num_objects': 1851, 'num_object_clones': 0, 'num_object_copies': 3702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15304, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2169}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:32.660 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:29:32.660 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'18887", 'reported_seq': 19012, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:29.502226+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:29:29.502226+0000', 'last_peered': '2024-09-15T21:29:29.502226+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:29:29.502226+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "44'8800", 'ondisk_log_start': "44'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7193059, 'num_objects': 1757, 'num_object_clones': 0, 'num_object_copies': 3514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1757, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16746, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2333}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:32.660 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:29:32.660 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'18875", 'reported_seq': 19013, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:28.988466+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:29:28.988466+0000', 'last_peered': '2024-09-15T21:29:28.988466+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:29:28.988466+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7417856, 'num_objects': 1811, 'num_object_clones': 0, 'num_object_copies': 3622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1811, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15819, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2283}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:32.660 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:29:32.660 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'18768", 'reported_seq': 18906, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:29.073004+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:29:29.073004+0000', 'last_peered': '2024-09-15T21:29:29.073004+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:29:29.073004+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "44'8700", 'ondisk_log_start': "44'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7462912, 'num_objects': 1822, 'num_object_clones': 0, 'num_object_copies': 3644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1822, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15677, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2317}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:32.660 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:29:32.661 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'19004", 'reported_seq': 19149, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:29.063153+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:29:29.063153+0000', 'last_peered': '2024-09-15T21:29:29.063153+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:29:29.063153+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7627718, 'num_objects': 1864, 'num_object_clones': 0, 'num_object_copies': 3728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1864, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15708, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2283}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:32.661 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:33.348 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:33.463 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38146/57300 objects degraded (66.572%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":625,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":28650,"data_bytes":117302025,"bytes_used":1719386112,"bytes_avail":288190906368,"bytes_total":289910292480,"degraded_objects":38146,"degraded_total":57300,"degraded_ratio":0.66572425828970327,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":391},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:33.464 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:33.464 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:34.015 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:34.034 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-15T21:29:32.858580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":117302025,"num_objects":28650,"num_object_clones":0,"num_object_copies":57300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28650,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":261809,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11329,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161017,"ondisk_log_size":161017,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1679088,"kb_used_data":428980,"kb_used_omap":12,"kb_used_meta":1250035,"kb_avail":281436432,"statfs":{"total":289910292480,"available":288190906368,"internally_reserved":0,"allocated":439275520,"data_stored":438386760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1280035843},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,7,4,5,6,4,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19230720,"num_objects":-4695,"num_object_clones":0,"num_object_copies":-9390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4695,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004268"},"pg_stats":[{"pgid":"2.f","version":"47'19339","reported_seq":19421,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.021283+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:29.021283+0000","last_peered":"2024-09-15T21:29:29.021283+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:29.021283+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7426048,"num_objects":1813,"num_object_clones":0,"num_object_copies":3626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19339,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2319}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'18897","reported_seq":19008,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.534957+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:29.534957+0000","last_peered":"2024-09-15T21:29:29.534957+0000","last_clean":"2024-09-15T21:29:29.534957+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:29.534957+0000","last_undegraded":"2024-09-15T21:29:29.534957+0000","last_fullsized":"2024-09-15T21:29:29.534957+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7279558,"num_objects":1779,"num_object_clones":0,"num_object_copies":3558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1779,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17178,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19287","reported_seq":25579,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.562434+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:29.562434+0000","last_peered":"2024-09-15T21:29:29.562434+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:29.562434+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7446552,"num_objects":1819,"num_object_clones":0,"num_object_copies":3638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1819,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17387,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3079,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":630}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18548","reported_seq":18625,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.495684+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:29.495684+0000","last_peered":"2024-09-15T21:29:29.495684+0000","last_clean":"2024-09-15T21:29:29.495684+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:29.495684+0000","last_undegraded":"2024-09-15T21:29:29.495684+0000","last_fullsized":"2024-09-15T21:29:29.495684+0000","mapping_epoch":35,"log_start":"43'8500","ondisk_log_start":"43'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7233536,"num_objects":1766,"num_object_clones":0,"num_object_copies":3532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18548,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18696","reported_seq":18835,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.010209+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:29.010209+0000","last_peered":"2024-09-15T21:29:29.010209+0000","last_clean":"2024-09-15T21:29:29.010209+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:29.010209+0000","last_undegraded":"2024-09-15T21:29:29.010209+0000","last_fullsized":"2024-09-15T21:29:29.010209+0000","mapping_epoch":34,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7410147,"num_objects":1810,"num_object_clones":0,"num_object_copies":3620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1810,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15151,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18819","reported_seq":18961,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:28.988904+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:28.988904+0000","last_peered":"2024-09-15T21:29:28.988904+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:28.988904+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7438819,"num_objects":1817,"num_object_clones":0,"num_object_copies":3634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1817,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15495,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2259},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'18897","reported_seq":19036,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:28.979421+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:28.979421+0000","last_peered":"2024-09-15T21:29:28.979421+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:28.979421+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7263174,"num_objects":1775,"num_object_clones":0,"num_object_copies":3550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15208,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2386},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19083","reported_seq":19218,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.047717+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:29.047717+0000","last_peered":"2024-09-15T21:29:29.047717+0000","last_clean":"2024-09-15T21:29:29.047717+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:29.047717+0000","last_undegraded":"2024-09-15T21:29:29.047717+0000","last_fullsized":"2024-09-15T21:29:29.047717+0000","mapping_epoch":34,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7311360,"num_objects":1785,"num_object_clones":0,"num_object_copies":3570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15855,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18792","reported_seq":18925,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.485422+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:29.485422+0000","last_peered":"2024-09-15T21:29:29.485422+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:29.485422+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"43'8700","ondisk_log_start":"43'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7049699,"num_objects":1722,"num_object_clones":0,"num_object_copies":3444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1722,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16724,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2365}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18631","reported_seq":18768,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.006392+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:29.006392+0000","last_peered":"2024-09-15T21:29:29.006392+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:29.006392+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7581696,"num_objects":1851,"num_object_clones":0,"num_object_copies":3702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15304,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2169}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'18887","reported_seq":19012,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.502226+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:29.502226+0000","last_peered":"2024-09-15T21:29:29.502226+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:29.502226+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"44'8800","ondisk_log_start":"44'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7193059,"num_objects":1757,"num_object_clones":0,"num_object_copies":3514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16746,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2333}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317475+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:09.317475+0000","last_peered":"2024-09-15T21:27:09.317475+0000","last_clean":"2024-09-15T21:27:09.317475+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:09.317475+0000","last_undegraded":"2024-09-15T21:27:09.317475+0000","last_fullsized":"2024-09-15T21:27:09.317475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'18869","reported_seq":27334,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:29.502430+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:29.502430+0000","last_peered":"2024-09-15T21:29:29.502430+0000","last_clean":"2024-09-15T21:29:29.502430+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:29.502430+0000","last_undegraded":"2024-09-15T21:29:29.502430+0000","last_fullsized":"2024-09-15T21:29:29.502430+0000","mapping_epoch":44,"log_start":"43'8800","ondisk_log_start":"43'8800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6926336,"num_objects":1691,"num_object_clones":0,"num_object_copies":3382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16562,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'18875","reported_seq":19013,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:28.988466+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:28.988466+0000","last_peered":"2024-09-15T21:29:28.988466+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:28.988466+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7417856,"num_objects":1811,"num_object_clones":0,"num_object_copies":3622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15819,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2283}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317449+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:09.317449+0000","last_peered":"2024-09-15T21:27:09.317449+0000","last_clean":"2024-09-15T21:27:09.317449+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:09.317449+0000","last_undegraded":"2024-09-15T21:27:09.317449+0000","last_fullsized":"2024-09-15T21:27:09.317449+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18768","reported_seq":18906,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.073004+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:29.073004+0000","last_peered":"2024-09-15T21:29:29.073004+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:29.073004+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"44'8700","ondisk_log_start":"44'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7462912,"num_objects":1822,"num_object_clones":0,"num_object_copies":3644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15677,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2317}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18622","reported_seq":26938,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:30.514091+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:30.514091+0000","last_peered":"2024-09-15T21:29:30.514091+0000","last_clean":"2024-09-15T21:29:30.514091+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:30.514091+0000","last_undegraded":"2024-09-15T21:29:30.514091+0000","last_fullsized":"2024-09-15T21:29:30.514091+0000","mapping_epoch":46,"log_start":"43'8600","ondisk_log_start":"43'8600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7233536,"num_objects":1766,"num_object_clones":0,"num_object_copies":3532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15106,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317327+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:09.317327+0000","last_peered":"2024-09-15T21:27:09.317327+0000","last_clean":"2024-09-15T21:27:09.317327+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:09.317327+0000","last_undegraded":"2024-09-15T21:27:09.317327+0000","last_fullsized":"2024-09-15T21:27:09.317327+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19004","reported_seq":19149,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:29.063153+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:29.063153+0000","last_peered":"2024-09-15T21:29:29.063153+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:29.063153+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7627718,"num_objects":1864,"num_object_clones":0,"num_object_copies":3728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1864,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15708,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2283},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":117302006,"num_objects":28648,"num_object_clones":0,"num_object_copies":57296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38146,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28648,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":261807,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11328,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":435671040,"data_stored":435525602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161014,"ondisk_log_size":161014,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526932,"kb_used_data":188420,"kb_used_omap":4,"kb_used_meta":338491,"kb_avail":93844908,"statfs":{"total":96636764160,"available":96097185792,"internally_reserved":0,"allocated":192942080,"data_stored":192656822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":346615465},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601508,"kb_used_data":154760,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93770332,"statfs":{"total":96636764160,"available":96020819968,"internally_reserved":0,"allocated":158474240,"data_stored":158188977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457436843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,3,3,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659236,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":550648,"kb_used_data":85800,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93821192,"statfs":{"total":96636764160,"available":96072900608,"internally_reserved":0,"allocated":87859200,"data_stored":87540961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,3,3,0,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157048832,"data_stored":157012702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86278144,"data_stored":86209038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192344064,"data_stored":192303862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:37.037 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:37.601 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:37.602 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:37.630 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-09-15T21:29:36.860098+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":107586313,"num_objects":26278,"num_object_clones":0,"num_object_copies":52556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26278,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":264181,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11448,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160689,"ondisk_log_size":160689,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1672696,"kb_used_data":410112,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281442824,"statfs":{"total":289910292480,"available":288197451776,"internally_reserved":0,"allocated":419954688,"data_stored":419065928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1292815363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,6,10,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20037632,"num_objects":-4892,"num_object_clones":0,"num_object_copies":-9784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4892,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004527"},"pg_stats":[{"pgid":"2.f","version":"47'19493","reported_seq":19575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.055111+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:34.055111+0000","last_peered":"2024-09-15T21:29:34.055111+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:34.055111+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6795264,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19493,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2473}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19037","reported_seq":19148,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.489201+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:34.489201+0000","last_peered":"2024-09-15T21:29:34.489201+0000","last_clean":"2024-09-15T21:29:34.489201+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:34.489201+0000","last_undegraded":"2024-09-15T21:29:34.489201+0000","last_fullsized":"2024-09-15T21:29:34.489201+0000","mapping_epoch":35,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6706118,"num_objects":1639,"num_object_clones":0,"num_object_copies":3278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1639,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17318,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19451","reported_seq":25980,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.572999+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:34.572999+0000","last_peered":"2024-09-15T21:29:34.572999+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:34.572999+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6774808,"num_objects":1655,"num_object_clones":0,"num_object_copies":3310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1655,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17551,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3198,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":585}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18707","reported_seq":18784,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.526289+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:34.526289+0000","last_peered":"2024-09-15T21:29:34.526289+0000","last_clean":"2024-09-15T21:29:34.526289+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:34.526289+0000","last_undegraded":"2024-09-15T21:29:34.526289+0000","last_fullsized":"2024-09-15T21:29:34.526289+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6582272,"num_objects":1607,"num_object_clones":0,"num_object_copies":3214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18707,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18832","reported_seq":18971,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.065997+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:34.065997+0000","last_peered":"2024-09-15T21:29:34.065997+0000","last_clean":"2024-09-15T21:29:34.065997+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:34.065997+0000","last_undegraded":"2024-09-15T21:29:34.065997+0000","last_fullsized":"2024-09-15T21:29:34.065997+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6853091,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15287,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18946","reported_seq":19087,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.100004+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:34.100004+0000","last_peered":"2024-09-15T21:29:34.100004+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:34.100004+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6918627,"num_objects":1690,"num_object_clones":0,"num_object_copies":3380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1690,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15622,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2386},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19053","reported_seq":19191,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.098939+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:34.098939+0000","last_peered":"2024-09-15T21:29:34.098939+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:34.098939+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6624198,"num_objects":1619,"num_object_clones":0,"num_object_copies":3238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15364,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2542},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19250","reported_seq":19384,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.100560+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:34.100560+0000","last_peered":"2024-09-15T21:29:34.100560+0000","last_clean":"2024-09-15T21:29:34.100560+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:34.100560+0000","last_undegraded":"2024-09-15T21:29:34.100560+0000","last_fullsized":"2024-09-15T21:29:34.100560+0000","mapping_epoch":34,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6627328,"num_objects":1618,"num_object_clones":0,"num_object_copies":3236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16022,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18925","reported_seq":19058,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.577046+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:34.577046+0000","last_peered":"2024-09-15T21:29:34.577046+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:34.577046+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6504931,"num_objects":1589,"num_object_clones":0,"num_object_copies":3178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1589,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16857,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2498}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18774","reported_seq":18911,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.064502+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:34.064502+0000","last_peered":"2024-09-15T21:29:34.064502+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:34.064502+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6995968,"num_objects":1708,"num_object_clones":0,"num_object_copies":3416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15447,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19042","reported_seq":19167,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.487747+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:34.487747+0000","last_peered":"2024-09-15T21:29:34.487747+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:34.487747+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6558179,"num_objects":1602,"num_object_clones":0,"num_object_copies":3204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1602,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16901,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2488}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19016","reported_seq":27481,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.541977+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:34.541977+0000","last_peered":"2024-09-15T21:29:34.541977+0000","last_clean":"2024-09-15T21:29:34.541977+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:34.541977+0000","last_undegraded":"2024-09-15T21:29:34.541977+0000","last_fullsized":"2024-09-15T21:29:34.541977+0000","mapping_epoch":44,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6324224,"num_objects":1544,"num_object_clones":0,"num_object_copies":3088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16709,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19019","reported_seq":19159,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.054206+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:34.054206+0000","last_peered":"2024-09-15T21:29:34.054206+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:34.054206+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6828032,"num_objects":1667,"num_object_clones":0,"num_object_copies":3334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15963,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2427}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18915","reported_seq":19054,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.031276+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:34.031276+0000","last_peered":"2024-09-15T21:29:34.031276+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:34.031276+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6860800,"num_objects":1675,"num_object_clones":0,"num_object_copies":3350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15824,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2464}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18768","reported_seq":27084,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:35.603147+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:35.603147+0000","last_peered":"2024-09-15T21:29:35.603147+0000","last_clean":"2024-09-15T21:29:35.603147+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:35.603147+0000","last_undegraded":"2024-09-15T21:29:35.603147+0000","last_fullsized":"2024-09-15T21:29:35.603147+0000","mapping_epoch":46,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6635520,"num_objects":1620,"num_object_clones":0,"num_object_copies":3240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15252,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19158","reported_seq":19303,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.089950+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:34.089950+0000","last_peered":"2024-09-15T21:29:34.089950+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:34.089950+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6996934,"num_objects":1710,"num_object_clones":0,"num_object_copies":3420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1710,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15862,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2437},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":107586294,"num_objects":26276,"num_object_clones":0,"num_object_copies":52552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26276,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":264179,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11447,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":416112640,"data_stored":415967202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526356,"kb_used_data":183684,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93845484,"statfs":{"total":96636764160,"available":96097775616,"internally_reserved":0,"allocated":188092416,"data_stored":187807158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598520,"kb_used_data":147620,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93773320,"statfs":{"total":96636764160,"available":96023879680,"internally_reserved":0,"allocated":151162880,"data_stored":150877617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,2,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547820,"kb_used_data":78808,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93824020,"statfs":{"total":96636764160,"available":96075796480,"internally_reserved":0,"allocated":80699392,"data_stored":80381153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":149786624,"data_stored":149750494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":78950400,"data_stored":78881294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":187375616,"data_stored":187335414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:37.632 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:38.184 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:38.184 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:38.209 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2024-09-15T21:29:36.860098+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":107586313,"num_objects":26278,"num_object_clones":0,"num_object_copies":52556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26278,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":264181,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11448,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160689,"ondisk_log_size":160689,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1672696,"kb_used_data":410112,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281442824,"statfs":{"total":289910292480,"available":288197451776,"internally_reserved":0,"allocated":419954688,"data_stored":419065928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1292815363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,6,10,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20037632,"num_objects":-4892,"num_object_clones":0,"num_object_copies":-9784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4892,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004527"},"pg_stats":[{"pgid":"2.f","version":"47'19493","reported_seq":19575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.055111+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:34.055111+0000","last_peered":"2024-09-15T21:29:34.055111+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:34.055111+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6795264,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19493,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2473}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19037","reported_seq":19148,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.489201+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:34.489201+0000","last_peered":"2024-09-15T21:29:34.489201+0000","last_clean":"2024-09-15T21:29:34.489201+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:34.489201+0000","last_undegraded":"2024-09-15T21:29:34.489201+0000","last_fullsized":"2024-09-15T21:29:34.489201+0000","mapping_epoch":35,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6706118,"num_objects":1639,"num_object_clones":0,"num_object_copies":3278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1639,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17318,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19451","reported_seq":25980,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.572999+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:34.572999+0000","last_peered":"2024-09-15T21:29:34.572999+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:34.572999+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6774808,"num_objects":1655,"num_object_clones":0,"num_object_copies":3310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1655,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17551,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3198,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":585}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18707","reported_seq":18784,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.526289+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:34.526289+0000","last_peered":"2024-09-15T21:29:34.526289+0000","last_clean":"2024-09-15T21:29:34.526289+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:34.526289+0000","last_undegraded":"2024-09-15T21:29:34.526289+0000","last_fullsized":"2024-09-15T21:29:34.526289+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6582272,"num_objects":1607,"num_object_clones":0,"num_object_copies":3214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18707,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18832","reported_seq":18971,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.065997+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:34.065997+0000","last_peered":"2024-09-15T21:29:34.065997+0000","last_clean":"2024-09-15T21:29:34.065997+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:34.065997+0000","last_undegraded":"2024-09-15T21:29:34.065997+0000","last_fullsized":"2024-09-15T21:29:34.065997+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6853091,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15287,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18946","reported_seq":19087,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.100004+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:34.100004+0000","last_peered":"2024-09-15T21:29:34.100004+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:34.100004+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6918627,"num_objects":1690,"num_object_clones":0,"num_object_copies":3380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1690,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15622,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2386},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19053","reported_seq":19191,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.098939+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:34.098939+0000","last_peered":"2024-09-15T21:29:34.098939+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:34.098939+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6624198,"num_objects":1619,"num_object_clones":0,"num_object_copies":3238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15364,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2542},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19250","reported_seq":19384,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.100560+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:34.100560+0000","last_peered":"2024-09-15T21:29:34.100560+0000","last_clean":"2024-09-15T21:29:34.100560+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:34.100560+0000","last_undegraded":"2024-09-15T21:29:34.100560+0000","last_fullsized":"2024-09-15T21:29:34.100560+0000","mapping_epoch":34,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6627328,"num_objects":1618,"num_object_clones":0,"num_object_copies":3236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16022,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18925","reported_seq":19058,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.577046+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:34.577046+0000","last_peered":"2024-09-15T21:29:34.577046+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:34.577046+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6504931,"num_objects":1589,"num_object_clones":0,"num_object_copies":3178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1589,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16857,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2498}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18774","reported_seq":18911,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.064502+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:34.064502+0000","last_peered":"2024-09-15T21:29:34.064502+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:34.064502+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6995968,"num_objects":1708,"num_object_clones":0,"num_object_copies":3416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15447,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19042","reported_seq":19167,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.487747+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:34.487747+0000","last_peered":"2024-09-15T21:29:34.487747+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:34.487747+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6558179,"num_objects":1602,"num_object_clones":0,"num_object_copies":3204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1602,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16901,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2488}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19016","reported_seq":27481,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.541977+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:34.541977+0000","last_peered":"2024-09-15T21:29:34.541977+0000","last_clean":"2024-09-15T21:29:34.541977+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:34.541977+0000","last_undegraded":"2024-09-15T21:29:34.541977+0000","last_fullsized":"2024-09-15T21:29:34.541977+0000","mapping_epoch":44,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6324224,"num_objects":1544,"num_object_clones":0,"num_object_copies":3088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16709,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19019","reported_seq":19159,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.054206+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:34.054206+0000","last_peered":"2024-09-15T21:29:34.054206+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:34.054206+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6828032,"num_objects":1667,"num_object_clones":0,"num_object_copies":3334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15963,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2427}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18915","reported_seq":19054,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.031276+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:34.031276+0000","last_peered":"2024-09-15T21:29:34.031276+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:34.031276+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6860800,"num_objects":1675,"num_object_clones":0,"num_object_copies":3350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15824,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2464}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18768","reported_seq":27084,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:35.603147+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:35.603147+0000","last_peered":"2024-09-15T21:29:35.603147+0000","last_clean":"2024-09-15T21:29:35.603147+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:35.603147+0000","last_undegraded":"2024-09-15T21:29:35.603147+0000","last_fullsized":"2024-09-15T21:29:35.603147+0000","mapping_epoch":46,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6635520,"num_objects":1620,"num_object_clones":0,"num_object_copies":3240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15252,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19158","reported_seq":19303,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.089950+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:34.089950+0000","last_peered":"2024-09-15T21:29:34.089950+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:34.089950+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6996934,"num_objects":1710,"num_object_clones":0,"num_object_copies":3420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1710,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15862,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2437},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":107586294,"num_objects":26276,"num_object_clones":0,"num_object_copies":52552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26276,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":264179,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11447,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":416112640,"data_stored":415967202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526356,"kb_used_data":183684,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93845484,"statfs":{"total":96636764160,"available":96097775616,"internally_reserved":0,"allocated":188092416,"data_stored":187807158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598520,"kb_used_data":147620,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93773320,"statfs":{"total":96636764160,"available":96023879680,"internally_reserved":0,"allocated":151162880,"data_stored":150877617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,2,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547820,"kb_used_data":78808,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93824020,"statfs":{"total":96636764160,"available":96075796480,"internally_reserved":0,"allocated":80699392,"data_stored":80381153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":149786624,"data_stored":149750494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":78950400,"data_stored":78881294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":187375616,"data_stored":187335414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:38.211 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:29:38.211 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19493", 'reported_seq': 19575, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:34.055111+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:29:34.055111+0000', 'last_peered': '2024-09-15T21:29:34.055111+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:29:34.055111+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6795264, 'num_objects': 1659, 'num_object_clones': 0, 'num_object_copies': 3318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1659, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19493, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2473}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:38.211 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:29:38.211 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'19451", 'reported_seq': 25980, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:34.572999+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:29:34.572999+0000', 'last_peered': '2024-09-15T21:29:34.572999+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:29:34.572999+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6774808, 'num_objects': 1655, 'num_object_clones': 0, 'num_object_copies': 3310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1655, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17551, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3198, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 585}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:38.211 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:29:38.211 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'18946", 'reported_seq': 19087, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:34.100004+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:29:34.100004+0000', 'last_peered': '2024-09-15T21:29:34.100004+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:29:34.100004+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6918627, 'num_objects': 1690, 'num_object_clones': 0, 'num_object_copies': 3380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1690, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15622, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2386}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:38.211 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:38.211 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19053", 'reported_seq': 19191, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:34.098939+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:29:34.098939+0000', 'last_peered': '2024-09-15T21:29:34.098939+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:29:34.098939+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6624198, 'num_objects': 1619, 'num_object_clones': 0, 'num_object_copies': 3238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1619, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15364, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2542}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:38.212 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:29:38.212 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'18925", 'reported_seq': 19058, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:34.577046+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:29:34.577046+0000', 'last_peered': '2024-09-15T21:29:34.577046+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:29:34.577046+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6504931, 'num_objects': 1589, 'num_object_clones': 0, 'num_object_copies': 3178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1589, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16857, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2498}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:38.212 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:38.212 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18774", 'reported_seq': 18911, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:34.064502+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:29:34.064502+0000', 'last_peered': '2024-09-15T21:29:34.064502+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:29:34.064502+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6995968, 'num_objects': 1708, 'num_object_clones': 0, 'num_object_copies': 3416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1708, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15447, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2312}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:38.212 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:29:38.212 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19042", 'reported_seq': 19167, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:34.487747+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:29:34.487747+0000', 'last_peered': '2024-09-15T21:29:34.487747+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:29:34.487747+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6558179, 'num_objects': 1602, 'num_object_clones': 0, 'num_object_copies': 3204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1602, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16901, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2488}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:38.212 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:29:38.212 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19019", 'reported_seq': 19159, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:34.054206+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:29:34.054206+0000', 'last_peered': '2024-09-15T21:29:34.054206+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:29:34.054206+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6828032, 'num_objects': 1667, 'num_object_clones': 0, 'num_object_copies': 3334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1667, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15963, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2427}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:38.212 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:29:38.212 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'18915", 'reported_seq': 19054, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:34.031276+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:29:34.031276+0000', 'last_peered': '2024-09-15T21:29:34.031276+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:29:34.031276+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6860800, 'num_objects': 1675, 'num_object_clones': 0, 'num_object_copies': 3350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1675, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15824, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2464}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:38.213 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:29:38.213 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'19158", 'reported_seq': 19303, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:34.089950+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:29:34.089950+0000', 'last_peered': '2024-09-15T21:29:34.089950+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:29:34.089950+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6996934, 'num_objects': 1710, 'num_object_clones': 0, 'num_object_copies': 3420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1710, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15862, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2437}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:38.213 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:38.935 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:38.935 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38027/52556 objects degraded (72.355%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":630,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":26278,"data_bytes":107586313,"bytes_used":1712840704,"bytes_avail":288197451776,"bytes_total":289910292480,"degraded_objects":38027,"degraded_total":52556,"degraded_ratio":0.72355202070172764,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":259,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":407},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:38.971 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:38.971 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:39.501 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:39.504 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:39.526 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-15T21:29:38.860842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":107586313,"num_objects":26278,"num_object_clones":0,"num_object_copies":52556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26278,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":264181,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11448,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160689,"ondisk_log_size":160689,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1672696,"kb_used_data":410112,"kb_used_omap":12,"kb_used_meta":1262515,"kb_avail":281442824,"statfs":{"total":289910292480,"available":288197451776,"internally_reserved":0,"allocated":419954688,"data_stored":419065928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1292815363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,6,10,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19345408,"num_objects":-4723,"num_object_clones":0,"num_object_copies":-9446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4723,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004458"},"pg_stats":[{"pgid":"2.f","version":"47'19493","reported_seq":19575,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.055111+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:34.055111+0000","last_peered":"2024-09-15T21:29:34.055111+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:34.055111+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6795264,"num_objects":1659,"num_object_clones":0,"num_object_copies":3318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19493,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2473}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19037","reported_seq":19148,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.489201+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:34.489201+0000","last_peered":"2024-09-15T21:29:34.489201+0000","last_clean":"2024-09-15T21:29:34.489201+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:34.489201+0000","last_undegraded":"2024-09-15T21:29:34.489201+0000","last_fullsized":"2024-09-15T21:29:34.489201+0000","mapping_epoch":35,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6706118,"num_objects":1639,"num_object_clones":0,"num_object_copies":3278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1639,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17318,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19451","reported_seq":25980,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.572999+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:34.572999+0000","last_peered":"2024-09-15T21:29:34.572999+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:34.572999+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6774808,"num_objects":1655,"num_object_clones":0,"num_object_copies":3310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1655,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17551,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3198,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":585}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18707","reported_seq":18784,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.526289+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:34.526289+0000","last_peered":"2024-09-15T21:29:34.526289+0000","last_clean":"2024-09-15T21:29:34.526289+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:34.526289+0000","last_undegraded":"2024-09-15T21:29:34.526289+0000","last_fullsized":"2024-09-15T21:29:34.526289+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6582272,"num_objects":1607,"num_object_clones":0,"num_object_copies":3214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18707,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18832","reported_seq":18971,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.065997+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:34.065997+0000","last_peered":"2024-09-15T21:29:34.065997+0000","last_clean":"2024-09-15T21:29:34.065997+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:34.065997+0000","last_undegraded":"2024-09-15T21:29:34.065997+0000","last_fullsized":"2024-09-15T21:29:34.065997+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6853091,"num_objects":1674,"num_object_clones":0,"num_object_copies":3348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15287,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'18946","reported_seq":19087,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.100004+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:34.100004+0000","last_peered":"2024-09-15T21:29:34.100004+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:34.100004+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6918627,"num_objects":1690,"num_object_clones":0,"num_object_copies":3380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1690,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15622,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2386},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19053","reported_seq":19191,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.098939+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:34.098939+0000","last_peered":"2024-09-15T21:29:34.098939+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:34.098939+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6624198,"num_objects":1619,"num_object_clones":0,"num_object_copies":3238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15364,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2542},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19250","reported_seq":19384,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.100560+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:34.100560+0000","last_peered":"2024-09-15T21:29:34.100560+0000","last_clean":"2024-09-15T21:29:34.100560+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:34.100560+0000","last_undegraded":"2024-09-15T21:29:34.100560+0000","last_fullsized":"2024-09-15T21:29:34.100560+0000","mapping_epoch":34,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6627328,"num_objects":1618,"num_object_clones":0,"num_object_copies":3236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16022,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'18925","reported_seq":19058,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.577046+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:34.577046+0000","last_peered":"2024-09-15T21:29:34.577046+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:34.577046+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6504931,"num_objects":1589,"num_object_clones":0,"num_object_copies":3178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1589,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16857,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2498}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18774","reported_seq":18911,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.064502+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:34.064502+0000","last_peered":"2024-09-15T21:29:34.064502+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:34.064502+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6995968,"num_objects":1708,"num_object_clones":0,"num_object_copies":3416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15447,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2312}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19042","reported_seq":19167,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.487747+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:34.487747+0000","last_peered":"2024-09-15T21:29:34.487747+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:34.487747+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6558179,"num_objects":1602,"num_object_clones":0,"num_object_copies":3204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1602,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16901,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2488}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19016","reported_seq":27481,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:34.541977+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:34.541977+0000","last_peered":"2024-09-15T21:29:34.541977+0000","last_clean":"2024-09-15T21:29:34.541977+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:34.541977+0000","last_undegraded":"2024-09-15T21:29:34.541977+0000","last_fullsized":"2024-09-15T21:29:34.541977+0000","mapping_epoch":44,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6324224,"num_objects":1544,"num_object_clones":0,"num_object_copies":3088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16709,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19019","reported_seq":19159,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.054206+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:34.054206+0000","last_peered":"2024-09-15T21:29:34.054206+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:34.054206+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6828032,"num_objects":1667,"num_object_clones":0,"num_object_copies":3334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15963,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2427}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'18915","reported_seq":19054,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.031276+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:34.031276+0000","last_peered":"2024-09-15T21:29:34.031276+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:34.031276+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6860800,"num_objects":1675,"num_object_clones":0,"num_object_copies":3350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15824,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2464}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18768","reported_seq":27084,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:35.603147+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:35.603147+0000","last_peered":"2024-09-15T21:29:35.603147+0000","last_clean":"2024-09-15T21:29:35.603147+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:35.603147+0000","last_undegraded":"2024-09-15T21:29:35.603147+0000","last_fullsized":"2024-09-15T21:29:35.603147+0000","mapping_epoch":46,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6635520,"num_objects":1620,"num_object_clones":0,"num_object_copies":3240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15252,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19158","reported_seq":19303,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:34.089950+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:34.089950+0000","last_peered":"2024-09-15T21:29:34.089950+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:34.089950+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6996934,"num_objects":1710,"num_object_clones":0,"num_object_copies":3420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1710,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15862,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2437},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":107586294,"num_objects":26276,"num_object_clones":0,"num_object_copies":52552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38027,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26276,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":264179,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11447,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":416112640,"data_stored":415967202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526356,"kb_used_data":183684,"kb_used_omap":4,"kb_used_meta":342651,"kb_avail":93845484,"statfs":{"total":96636764160,"available":96097775616,"internally_reserved":0,"allocated":188092416,"data_stored":187807158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":350875305},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":598520,"kb_used_data":147620,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93773320,"statfs":{"total":96636764160,"available":96023879680,"internally_reserved":0,"allocated":151162880,"data_stored":150877617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,4,2,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":547820,"kb_used_data":78808,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93824020,"statfs":{"total":96636764160,"available":96075796480,"internally_reserved":0,"allocated":80699392,"data_stored":80381153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,7],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":149786624,"data_stored":149750494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":78950400,"data_stored":78881294,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":187375616,"data_stored":187335414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:42.529 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:43.117 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:43.120 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:43.149 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-15T21:29:42.862378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":97899273,"num_objects":23913,"num_object_clones":0,"num_object_copies":47826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23913,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":266546,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11574,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161054,"ondisk_log_size":161054,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1675780,"kb_used_data":390996,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281439740,"statfs":{"total":289910292480,"available":288194293760,"internally_reserved":0,"allocated":400379904,"data_stored":399491144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1315556355},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,4,8,6,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19402752,"num_objects":-4737,"num_object_clones":0,"num_object_copies":-9474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-245,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4737,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004463"},"pg_stats":[{"pgid":"2.f","version":"47'19618","reported_seq":19700,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:38.973891+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:38.973891+0000","last_peered":"2024-09-15T21:29:38.973891+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:38.973891+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6283264,"num_objects":1534,"num_object_clones":0,"num_object_copies":3068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19618,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19188","reported_seq":19299,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:39.516007+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:39.516007+0000","last_peered":"2024-09-15T21:29:39.516007+0000","last_clean":"2024-09-15T21:29:39.516007+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:39.516007+0000","last_undegraded":"2024-09-15T21:29:39.516007+0000","last_fullsized":"2024-09-15T21:29:39.516007+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6087622,"num_objects":1488,"num_object_clones":0,"num_object_copies":2976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1488,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17469,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19590","reported_seq":26372,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.554004+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:39.554004+0000","last_peered":"2024-09-15T21:29:39.554004+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:39.554004+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6205464,"num_objects":1516,"num_object_clones":0,"num_object_copies":3032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1516,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17690,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3324,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":572}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18867","reported_seq":18943,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:39.584851+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:39.584851+0000","last_peered":"2024-09-15T21:29:39.584851+0000","last_clean":"2024-09-15T21:29:39.584851+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:39.584851+0000","last_undegraded":"2024-09-15T21:29:39.584851+0000","last_fullsized":"2024-09-15T21:29:39.584851+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5926912,"num_objects":1447,"num_object_clones":0,"num_object_copies":2894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18867,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18973","reported_seq":19112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:39.084543+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:39.084543+0000","last_peered":"2024-09-15T21:29:39.084543+0000","last_clean":"2024-09-15T21:29:39.084543+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:39.084543+0000","last_undegraded":"2024-09-15T21:29:39.084543+0000","last_fullsized":"2024-09-15T21:29:39.084543+0000","mapping_epoch":34,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6275555,"num_objects":1533,"num_object_clones":0,"num_object_copies":3066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15428,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19092","reported_seq":19234,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.069046+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:39.069046+0000","last_peered":"2024-09-15T21:29:39.069046+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:39.069046+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6320611,"num_objects":1544,"num_object_clones":0,"num_object_copies":3088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15768,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2532},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19199","reported_seq":19341,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.058222+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:39.058222+0000","last_peered":"2024-09-15T21:29:39.058222+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:39.058222+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6026182,"num_objects":1473,"num_object_clones":0,"num_object_copies":2946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15510,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2688},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19385","reported_seq":19520,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:39.103982+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:39.103982+0000","last_peered":"2024-09-15T21:29:39.103982+0000","last_clean":"2024-09-15T21:29:39.103982+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:39.103982+0000","last_undegraded":"2024-09-15T21:29:39.103982+0000","last_fullsized":"2024-09-15T21:29:39.103982+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6074368,"num_objects":1483,"num_object_clones":0,"num_object_copies":2966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16157,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19076","reported_seq":19209,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.570144+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:39.570144+0000","last_peered":"2024-09-15T21:29:39.570144+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:39.570144+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5886435,"num_objects":1438,"num_object_clones":0,"num_object_copies":2876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1438,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17008,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2649}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18918","reported_seq":19055,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.025816+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:39.025816+0000","last_peered":"2024-09-15T21:29:39.025816+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:39.025816+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6406144,"num_objects":1564,"num_object_clones":0,"num_object_copies":3128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15591,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19179","reported_seq":19301,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.455530+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:39.455530+0000","last_peered":"2024-09-15T21:29:39.455530+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:39.455530+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5997027,"num_objects":1465,"num_object_clones":0,"num_object_copies":2930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1465,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17038,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2625}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19165","reported_seq":27630,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:39.411903+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:39.411903+0000","last_peered":"2024-09-15T21:29:39.411903+0000","last_clean":"2024-09-15T21:29:39.411903+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:39.411903+0000","last_undegraded":"2024-09-15T21:29:39.411903+0000","last_fullsized":"2024-09-15T21:29:39.411903+0000","mapping_epoch":44,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5713920,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16858,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19175","reported_seq":19312,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.048838+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:39.048838+0000","last_peered":"2024-09-15T21:29:39.048838+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:39.048838+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6189056,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16119,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2583}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19061","reported_seq":19200,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.047639+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:39.047639+0000","last_peered":"2024-09-15T21:29:39.047639+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:39.047639+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6262784,"num_objects":1529,"num_object_clones":0,"num_object_copies":3058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15970,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2610}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18932","reported_seq":27248,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:40.726766+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:40.726766+0000","last_peered":"2024-09-15T21:29:40.726766+0000","last_clean":"2024-09-15T21:29:40.726766+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:40.726766+0000","last_undegraded":"2024-09-15T21:29:40.726766+0000","last_fullsized":"2024-09-15T21:29:40.726766+0000","mapping_epoch":46,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":1456,"num_object_clones":0,"num_object_copies":2912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15416,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19333","reported_seq":19478,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.055232+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:39.055232+0000","last_peered":"2024-09-15T21:29:39.055232+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:39.055232+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6280134,"num_objects":1535,"num_object_clones":0,"num_object_copies":3070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16037,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2612},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":97899254,"num_objects":23911,"num_object_clones":0,"num_object_copies":47822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23911,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":266544,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11573,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":396615680,"data_stored":396470242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161051,"ondisk_log_size":161051,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543836,"kb_used_data":178956,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93828004,"statfs":{"total":96636764160,"available":96079876096,"internally_reserved":0,"allocated":183250944,"data_stored":182965686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599584,"kb_used_data":140368,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93772256,"statfs":{"total":96636764160,"available":96022790144,"internally_reserved":0,"allocated":143736832,"data_stored":143451569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,0,0,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532360,"kb_used_data":71672,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93839480,"statfs":{"total":96636764160,"available":96091627520,"internally_reserved":0,"allocated":73392128,"data_stored":73073889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471723695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":142426112,"data_stored":142389982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":71585792,"data_stored":71516686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":182603776,"data_stored":182563574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:43.151 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:43.718 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:43.721 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:43.747 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-15T21:29:42.862378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":97899273,"num_objects":23913,"num_object_clones":0,"num_object_copies":47826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23913,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":266546,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11574,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161054,"ondisk_log_size":161054,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1675780,"kb_used_data":390996,"kb_used_omap":12,"kb_used_meta":1284723,"kb_avail":281439740,"statfs":{"total":289910292480,"available":288194293760,"internally_reserved":0,"allocated":400379904,"data_stored":399491144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1315556355},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,4,8,6,11,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19402752,"num_objects":-4737,"num_object_clones":0,"num_object_copies":-9474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-245,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4737,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4737,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004463"},"pg_stats":[{"pgid":"2.f","version":"47'19618","reported_seq":19700,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:38.973891+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:38.973891+0000","last_peered":"2024-09-15T21:29:38.973891+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:38.973891+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6283264,"num_objects":1534,"num_object_clones":0,"num_object_copies":3068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19618,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19188","reported_seq":19299,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:39.516007+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:39.516007+0000","last_peered":"2024-09-15T21:29:39.516007+0000","last_clean":"2024-09-15T21:29:39.516007+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:39.516007+0000","last_undegraded":"2024-09-15T21:29:39.516007+0000","last_fullsized":"2024-09-15T21:29:39.516007+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6087622,"num_objects":1488,"num_object_clones":0,"num_object_copies":2976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1488,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17469,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19590","reported_seq":26372,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.554004+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:39.554004+0000","last_peered":"2024-09-15T21:29:39.554004+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:39.554004+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6205464,"num_objects":1516,"num_object_clones":0,"num_object_copies":3032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1516,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17690,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3324,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":572}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'18867","reported_seq":18943,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:39.584851+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:39.584851+0000","last_peered":"2024-09-15T21:29:39.584851+0000","last_clean":"2024-09-15T21:29:39.584851+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:39.584851+0000","last_undegraded":"2024-09-15T21:29:39.584851+0000","last_fullsized":"2024-09-15T21:29:39.584851+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5926912,"num_objects":1447,"num_object_clones":0,"num_object_copies":2894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18867,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'18973","reported_seq":19112,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:39.084543+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:39.084543+0000","last_peered":"2024-09-15T21:29:39.084543+0000","last_clean":"2024-09-15T21:29:39.084543+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:39.084543+0000","last_undegraded":"2024-09-15T21:29:39.084543+0000","last_fullsized":"2024-09-15T21:29:39.084543+0000","mapping_epoch":34,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6275555,"num_objects":1533,"num_object_clones":0,"num_object_copies":3066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15428,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19092","reported_seq":19234,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.069046+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:39.069046+0000","last_peered":"2024-09-15T21:29:39.069046+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:39.069046+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6320611,"num_objects":1544,"num_object_clones":0,"num_object_copies":3088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1544,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15768,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2532},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19199","reported_seq":19341,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.058222+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:39.058222+0000","last_peered":"2024-09-15T21:29:39.058222+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:39.058222+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6026182,"num_objects":1473,"num_object_clones":0,"num_object_copies":2946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15510,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2688},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19385","reported_seq":19520,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:39.103982+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:39.103982+0000","last_peered":"2024-09-15T21:29:39.103982+0000","last_clean":"2024-09-15T21:29:39.103982+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:39.103982+0000","last_undegraded":"2024-09-15T21:29:39.103982+0000","last_fullsized":"2024-09-15T21:29:39.103982+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6074368,"num_objects":1483,"num_object_clones":0,"num_object_copies":2966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16157,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19076","reported_seq":19209,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.570144+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:39.570144+0000","last_peered":"2024-09-15T21:29:39.570144+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:39.570144+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5886435,"num_objects":1438,"num_object_clones":0,"num_object_copies":2876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1438,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17008,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2649}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'18918","reported_seq":19055,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.025816+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:39.025816+0000","last_peered":"2024-09-15T21:29:39.025816+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:39.025816+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6406144,"num_objects":1564,"num_object_clones":0,"num_object_copies":3128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15591,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19179","reported_seq":19301,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.455530+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:39.455530+0000","last_peered":"2024-09-15T21:29:39.455530+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:39.455530+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5997027,"num_objects":1465,"num_object_clones":0,"num_object_copies":2930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1465,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17038,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2625}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19165","reported_seq":27630,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:39.411903+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:39.411903+0000","last_peered":"2024-09-15T21:29:39.411903+0000","last_clean":"2024-09-15T21:29:39.411903+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:39.411903+0000","last_undegraded":"2024-09-15T21:29:39.411903+0000","last_fullsized":"2024-09-15T21:29:39.411903+0000","mapping_epoch":44,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5713920,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16858,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19175","reported_seq":19312,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.048838+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:39.048838+0000","last_peered":"2024-09-15T21:29:39.048838+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:39.048838+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6189056,"num_objects":1511,"num_object_clones":0,"num_object_copies":3022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16119,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2583}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19061","reported_seq":19200,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.047639+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:39.047639+0000","last_peered":"2024-09-15T21:29:39.047639+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:39.047639+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6262784,"num_objects":1529,"num_object_clones":0,"num_object_copies":3058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15970,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2610}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18932","reported_seq":27248,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:40.726766+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:40.726766+0000","last_peered":"2024-09-15T21:29:40.726766+0000","last_clean":"2024-09-15T21:29:40.726766+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:40.726766+0000","last_undegraded":"2024-09-15T21:29:40.726766+0000","last_fullsized":"2024-09-15T21:29:40.726766+0000","mapping_epoch":46,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":1456,"num_object_clones":0,"num_object_copies":2912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15416,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19333","reported_seq":19478,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:39.055232+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:39.055232+0000","last_peered":"2024-09-15T21:29:39.055232+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:39.055232+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6280134,"num_objects":1535,"num_object_clones":0,"num_object_copies":3070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16037,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2612},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":97899254,"num_objects":23911,"num_object_clones":0,"num_object_copies":47822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23911,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":266544,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11573,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":396615680,"data_stored":396470242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161051,"ondisk_log_size":161051,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543836,"kb_used_data":178956,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93828004,"statfs":{"total":96636764160,"available":96079876096,"internally_reserved":0,"allocated":183250944,"data_stored":182965686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,5,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599584,"kb_used_data":140368,"kb_used_omap":4,"kb_used_meta":459195,"kb_avail":93772256,"statfs":{"total":96636764160,"available":96022790144,"internally_reserved":0,"allocated":143736832,"data_stored":143451569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":470216363},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,0,0,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532360,"kb_used_data":71672,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93839480,"statfs":{"total":96636764160,"available":96091627520,"internally_reserved":0,"allocated":73392128,"data_stored":73073889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471723695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":142426112,"data_stored":142389982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":71585792,"data_stored":71516686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":182603776,"data_stored":182563574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:43.749 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:29:43.749 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19618", 'reported_seq': 19700, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:38.973891+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:29:38.973891+0000', 'last_peered': '2024-09-15T21:29:38.973891+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:29:38.973891+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6283264, 'num_objects': 1534, 'num_object_clones': 0, 'num_object_copies': 3068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1534, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19618, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2598}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:43.749 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:29:43.749 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'19590", 'reported_seq': 26372, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:39.554004+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:29:39.554004+0000', 'last_peered': '2024-09-15T21:29:39.554004+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:29:39.554004+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6205464, 'num_objects': 1516, 'num_object_clones': 0, 'num_object_copies': 3032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 944, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1516, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17690, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3324, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 572}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:43.749 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:29:43.749 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19092", 'reported_seq': 19234, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:39.069046+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:29:39.069046+0000', 'last_peered': '2024-09-15T21:29:39.069046+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:29:39.069046+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6320611, 'num_objects': 1544, 'num_object_clones': 0, 'num_object_copies': 3088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1544, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15768, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2532}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:43.749 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:43.750 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19199", 'reported_seq': 19341, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:39.058222+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:29:39.058222+0000', 'last_peered': '2024-09-15T21:29:39.058222+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:29:39.058222+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6026182, 'num_objects': 1473, 'num_object_clones': 0, 'num_object_copies': 2946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15510, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2688}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:43.750 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:29:43.750 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'19076", 'reported_seq': 19209, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:39.570144+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:29:39.570144+0000', 'last_peered': '2024-09-15T21:29:39.570144+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:29:39.570144+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5886435, 'num_objects': 1438, 'num_object_clones': 0, 'num_object_copies': 2876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1438, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17008, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2649}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:43.750 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:43.750 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'18918", 'reported_seq': 19055, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:39.025816+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:29:39.025816+0000', 'last_peered': '2024-09-15T21:29:39.025816+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:29:39.025816+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6406144, 'num_objects': 1564, 'num_object_clones': 0, 'num_object_copies': 3128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1564, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15591, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2456}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:43.750 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:29:43.750 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19179", 'reported_seq': 19301, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:39.455530+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:29:39.455530+0000', 'last_peered': '2024-09-15T21:29:39.455530+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:29:39.455530+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5997027, 'num_objects': 1465, 'num_object_clones': 0, 'num_object_copies': 2930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1465, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17038, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2625}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:43.750 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:29:43.750 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19175", 'reported_seq': 19312, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:39.048838+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:29:39.048838+0000', 'last_peered': '2024-09-15T21:29:39.048838+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:29:39.048838+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6189056, 'num_objects': 1511, 'num_object_clones': 0, 'num_object_copies': 3022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1511, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16119, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2583}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:43.750 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:29:43.750 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'19061", 'reported_seq': 19200, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:39.047639+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:29:39.047639+0000', 'last_peered': '2024-09-15T21:29:39.047639+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:29:39.047639+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6262784, 'num_objects': 1529, 'num_object_clones': 0, 'num_object_copies': 3058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1529, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15970, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2610}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:43.751 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:29:43.751 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'19333", 'reported_seq': 19478, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:39.055232+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:29:39.055232+0000', 'last_peered': '2024-09-15T21:29:39.055232+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:29:39.055232+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6280134, 'num_objects': 1535, 'num_object_clones': 0, 'num_object_copies': 3070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1535, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16037, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2612}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:43.751 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:44.442 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:44.443 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37901/47826 objects degraded (79.248%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":636,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":23913,"data_bytes":97899273,"bytes_used":1715998720,"bytes_avail":288194293760,"bytes_total":289910292480,"degraded_objects":37901,"degraded_total":47826,"degraded_ratio":0.79247689541253707,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":394},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:44.462 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:44.462 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:45.016 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:45.016 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:45.047 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-15T21:29:44.863099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88757001,"num_objects":21681,"num_object_clones":0,"num_object_copies":43362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37753,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21681,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":268778,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11722,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1656800,"kb_used_data":378896,"kb_used_omap":12,"kb_used_meta":1277811,"kb_avail":281458720,"statfs":{"total":289910292480,"available":288213729280,"internally_reserved":0,"allocated":387989504,"data_stored":387100744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1308478467},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,8,10,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28545024,"num_objects":-6969,"num_object_clones":0,"num_object_copies":-13938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-393,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6969,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":393,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004518"},"pg_stats":[{"pgid":"2.f","version":"47'19777","reported_seq":19859,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.084287+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:44.084287+0000","last_peered":"2024-09-15T21:29:44.084287+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:44.084287+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5632000,"num_objects":1375,"num_object_clones":0,"num_object_copies":2750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19777,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2757}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19339","reported_seq":19451,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.590497+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:44.590497+0000","last_peered":"2024-09-15T21:29:44.590497+0000","last_clean":"2024-09-15T21:29:44.590497+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:44.590497+0000","last_undegraded":"2024-09-15T21:29:44.590497+0000","last_fullsized":"2024-09-15T21:29:44.590497+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5469126,"num_objects":1337,"num_object_clones":0,"num_object_copies":2674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1337,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17620,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19742","reported_seq":26819,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.569508+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:44.569508+0000","last_peered":"2024-09-15T21:29:44.569508+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:44.569508+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5582872,"num_objects":1364,"num_object_clones":0,"num_object_copies":2728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17842,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3472,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":568}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19004","reported_seq":19081,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.556976+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:44.556976+0000","last_peered":"2024-09-15T21:29:44.556976+0000","last_clean":"2024-09-15T21:29:44.556976+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:44.556976+0000","last_undegraded":"2024-09-15T21:29:44.556976+0000","last_fullsized":"2024-09-15T21:29:44.556976+0000","mapping_epoch":35,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5365760,"num_objects":1310,"num_object_clones":0,"num_object_copies":2620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19004,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19147","reported_seq":19285,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.102945+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:44.102945+0000","last_peered":"2024-09-15T21:29:44.102945+0000","last_clean":"2024-09-15T21:29:44.102945+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:44.102945+0000","last_undegraded":"2024-09-15T21:29:44.102945+0000","last_fullsized":"2024-09-15T21:29:44.102945+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5562851,"num_objects":1359,"num_object_clones":0,"num_object_copies":2718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15602,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19245","reported_seq":19384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.115768+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:44.115768+0000","last_peered":"2024-09-15T21:29:44.115768+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:44.115768+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5693923,"num_objects":1391,"num_object_clones":0,"num_object_copies":2782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15921,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2685},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19347","reported_seq":19487,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.011797+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:44.011797+0000","last_peered":"2024-09-15T21:29:44.011797+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:44.011797+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5419974,"num_objects":1325,"num_object_clones":0,"num_object_copies":2650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15658,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2836},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19528","reported_seq":19663,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.046227+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:44.046227+0000","last_peered":"2024-09-15T21:29:44.046227+0000","last_clean":"2024-09-15T21:29:44.046227+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:44.046227+0000","last_undegraded":"2024-09-15T21:29:44.046227+0000","last_fullsized":"2024-09-15T21:29:44.046227+0000","mapping_epoch":34,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5488640,"num_objects":1340,"num_object_clones":0,"num_object_copies":2680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16300,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19220","reported_seq":19353,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.512220+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:44.512220+0000","last_peered":"2024-09-15T21:29:44.512220+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:44.512220+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5296611,"num_objects":1294,"num_object_clones":0,"num_object_copies":2588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1294,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17152,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2793}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19083","reported_seq":19220,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:43.973812+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:43.973812+0000","last_peered":"2024-09-15T21:29:43.973812+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:43.973812+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5730304,"num_objects":1399,"num_object_clones":0,"num_object_copies":2798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15756,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2621}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19328","reported_seq":19453,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.566167+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:44.566167+0000","last_peered":"2024-09-15T21:29:44.566167+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:44.566167+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5386723,"num_objects":1316,"num_object_clones":0,"num_object_copies":2632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17187,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2774}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19290","reported_seq":27755,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.566800+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:44.566800+0000","last_peered":"2024-09-15T21:29:44.566800+0000","last_clean":"2024-09-15T21:29:44.566800+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:44.566800+0000","last_undegraded":"2024-09-15T21:29:44.566800+0000","last_fullsized":"2024-09-15T21:29:44.566800+0000","mapping_epoch":44,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5201920,"num_objects":1270,"num_object_clones":0,"num_object_copies":2540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16983,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19310","reported_seq":19450,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.082976+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:44.082976+0000","last_peered":"2024-09-15T21:29:44.082976+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:44.082976+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":1376,"num_object_clones":0,"num_object_copies":2752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16254,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2718}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19211","reported_seq":19350,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.033998+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:44.033998+0000","last_peered":"2024-09-15T21:29:44.033998+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:44.033998+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5648384,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16120,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'18932","reported_seq":27248,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:40.726766+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:40.726766+0000","last_peered":"2024-09-15T21:29:40.726766+0000","last_clean":"2024-09-15T21:29:40.726766+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:40.726766+0000","last_undegraded":"2024-09-15T21:29:40.726766+0000","last_fullsized":"2024-09-15T21:29:40.726766+0000","mapping_epoch":46,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5963776,"num_objects":1456,"num_object_clones":0,"num_object_copies":2912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15416,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19480","reported_seq":19625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.061334+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:44.061334+0000","last_peered":"2024-09-15T21:29:44.061334+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:44.061334+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5678022,"num_objects":1388,"num_object_clones":0,"num_object_copies":2776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1388,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16184,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2759},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":88756982,"num_objects":21679,"num_object_clones":0,"num_object_copies":43358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37753,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21679,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":268776,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11721,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":384319488,"data_stored":384174050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160683,"ondisk_log_size":160683,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526656,"kb_used_data":174248,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93845184,"statfs":{"total":96636764160,"available":96097468416,"internally_reserved":0,"allocated":178429952,"data_stored":178144694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597784,"kb_used_data":132976,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93774056,"statfs":{"total":96636764160,"available":96024633344,"internally_reserved":0,"allocated":136167424,"data_stored":135882161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475917995},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532360,"kb_used_data":71672,"kb_used_omap":4,"kb_used_meta":460667,"kb_avail":93839480,"statfs":{"total":96636764160,"available":96091627520,"internally_reserved":0,"allocated":73392128,"data_stored":73073889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471723695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,0,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":134971392,"data_stored":134935262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":71585792,"data_stored":71516686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177762304,"data_stored":177722102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:48.050 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:48.614 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:48.615 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:48.649 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-09-15T21:29:46.864018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88138505,"num_objects":21530,"num_object_clones":0,"num_object_copies":43060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37753,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21530,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":268929,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11722,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1656584,"kb_used_data":370356,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281458936,"statfs":{"total":289910292480,"available":288213950464,"internally_reserved":0,"allocated":379244544,"data_stored":378355784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1316998147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,7,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20045824,"num_objects":-4894,"num_object_clones":0,"num_object_copies":-9788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4894,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004664"},"pg_stats":[{"pgid":"2.f","version":"47'19777","reported_seq":19859,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.084287+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:44.084287+0000","last_peered":"2024-09-15T21:29:44.084287+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:44.084287+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5632000,"num_objects":1375,"num_object_clones":0,"num_object_copies":2750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19777,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2757}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19339","reported_seq":19451,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.590497+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:44.590497+0000","last_peered":"2024-09-15T21:29:44.590497+0000","last_clean":"2024-09-15T21:29:44.590497+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:44.590497+0000","last_undegraded":"2024-09-15T21:29:44.590497+0000","last_fullsized":"2024-09-15T21:29:44.590497+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5469126,"num_objects":1337,"num_object_clones":0,"num_object_copies":2674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1337,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17620,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19742","reported_seq":26819,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.569508+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:44.569508+0000","last_peered":"2024-09-15T21:29:44.569508+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:44.569508+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5582872,"num_objects":1364,"num_object_clones":0,"num_object_copies":2728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17842,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3472,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":568}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19004","reported_seq":19081,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.556976+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:44.556976+0000","last_peered":"2024-09-15T21:29:44.556976+0000","last_clean":"2024-09-15T21:29:44.556976+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:44.556976+0000","last_undegraded":"2024-09-15T21:29:44.556976+0000","last_fullsized":"2024-09-15T21:29:44.556976+0000","mapping_epoch":35,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5365760,"num_objects":1310,"num_object_clones":0,"num_object_copies":2620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19004,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19147","reported_seq":19285,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.102945+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:44.102945+0000","last_peered":"2024-09-15T21:29:44.102945+0000","last_clean":"2024-09-15T21:29:44.102945+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:44.102945+0000","last_undegraded":"2024-09-15T21:29:44.102945+0000","last_fullsized":"2024-09-15T21:29:44.102945+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5562851,"num_objects":1359,"num_object_clones":0,"num_object_copies":2718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15602,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19245","reported_seq":19384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.115768+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:44.115768+0000","last_peered":"2024-09-15T21:29:44.115768+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:44.115768+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5693923,"num_objects":1391,"num_object_clones":0,"num_object_copies":2782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15921,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2685},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19347","reported_seq":19487,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.011797+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:44.011797+0000","last_peered":"2024-09-15T21:29:44.011797+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:44.011797+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5419974,"num_objects":1325,"num_object_clones":0,"num_object_copies":2650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15658,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2836},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19528","reported_seq":19663,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.046227+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:44.046227+0000","last_peered":"2024-09-15T21:29:44.046227+0000","last_clean":"2024-09-15T21:29:44.046227+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:44.046227+0000","last_undegraded":"2024-09-15T21:29:44.046227+0000","last_fullsized":"2024-09-15T21:29:44.046227+0000","mapping_epoch":34,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5488640,"num_objects":1340,"num_object_clones":0,"num_object_copies":2680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16300,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19220","reported_seq":19353,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.512220+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:44.512220+0000","last_peered":"2024-09-15T21:29:44.512220+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:44.512220+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5296611,"num_objects":1294,"num_object_clones":0,"num_object_copies":2588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1294,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17152,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2793}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19083","reported_seq":19220,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:43.973812+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:43.973812+0000","last_peered":"2024-09-15T21:29:43.973812+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:43.973812+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5730304,"num_objects":1399,"num_object_clones":0,"num_object_copies":2798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15756,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2621}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19328","reported_seq":19453,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.566167+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:44.566167+0000","last_peered":"2024-09-15T21:29:44.566167+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:44.566167+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5386723,"num_objects":1316,"num_object_clones":0,"num_object_copies":2632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17187,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2774}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19290","reported_seq":27755,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.566800+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:44.566800+0000","last_peered":"2024-09-15T21:29:44.566800+0000","last_clean":"2024-09-15T21:29:44.566800+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:44.566800+0000","last_undegraded":"2024-09-15T21:29:44.566800+0000","last_fullsized":"2024-09-15T21:29:44.566800+0000","mapping_epoch":44,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5201920,"num_objects":1270,"num_object_clones":0,"num_object_copies":2540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16983,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19310","reported_seq":19450,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.082976+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:44.082976+0000","last_peered":"2024-09-15T21:29:44.082976+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:44.082976+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":1376,"num_object_clones":0,"num_object_copies":2752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16254,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2718}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19211","reported_seq":19350,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.033998+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:44.033998+0000","last_peered":"2024-09-15T21:29:44.033998+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:44.033998+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5648384,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16120,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19083","reported_seq":27399,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.818747+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:45.818747+0000","last_peered":"2024-09-15T21:29:45.818747+0000","last_clean":"2024-09-15T21:29:45.818747+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:45.818747+0000","last_undegraded":"2024-09-15T21:29:45.818747+0000","last_fullsized":"2024-09-15T21:29:45.818747+0000","mapping_epoch":46,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5345280,"num_objects":1305,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15567,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19480","reported_seq":19625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.061334+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:44.061334+0000","last_peered":"2024-09-15T21:29:44.061334+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:44.061334+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5678022,"num_objects":1388,"num_object_clones":0,"num_object_copies":2776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1388,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16184,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2759},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":88138486,"num_objects":21528,"num_object_clones":0,"num_object_copies":43056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37753,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21528,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":268927,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11721,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376971264,"data_stored":376825826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526656,"kb_used_data":174248,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93845184,"statfs":{"total":96636764160,"available":96097468416,"internally_reserved":0,"allocated":178429952,"data_stored":178144694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597784,"kb_used_data":132976,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93774056,"statfs":{"total":96636764160,"available":96024633344,"internally_reserved":0,"allocated":136167424,"data_stored":135882161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475917995},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532144,"kb_used_data":63132,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93839696,"statfs":{"total":96636764160,"available":96091848704,"internally_reserved":0,"allocated":64647168,"data_stored":64328929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":134971392,"data_stored":134935262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64237568,"data_stored":64168462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177762304,"data_stored":177722102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:48.650 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:49.207 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:49.209 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:49.231 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-15T21:29:48.864686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88138505,"num_objects":21530,"num_object_clones":0,"num_object_copies":43060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37753,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21530,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":268929,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11722,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1656584,"kb_used_data":370356,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281458936,"statfs":{"total":289910292480,"available":288213950464,"internally_reserved":0,"allocated":379244544,"data_stored":378355784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1316998147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,7,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19447808,"num_objects":-4748,"num_object_clones":0,"num_object_copies":-9496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4748,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004588"},"pg_stats":[{"pgid":"2.f","version":"47'19777","reported_seq":19859,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.084287+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:44.084287+0000","last_peered":"2024-09-15T21:29:44.084287+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:44.084287+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5632000,"num_objects":1375,"num_object_clones":0,"num_object_copies":2750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19777,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2757}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19339","reported_seq":19451,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.590497+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:44.590497+0000","last_peered":"2024-09-15T21:29:44.590497+0000","last_clean":"2024-09-15T21:29:44.590497+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:44.590497+0000","last_undegraded":"2024-09-15T21:29:44.590497+0000","last_fullsized":"2024-09-15T21:29:44.590497+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5469126,"num_objects":1337,"num_object_clones":0,"num_object_copies":2674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1337,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17620,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19742","reported_seq":26819,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.569508+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:44.569508+0000","last_peered":"2024-09-15T21:29:44.569508+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:44.569508+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5582872,"num_objects":1364,"num_object_clones":0,"num_object_copies":2728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17842,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3472,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":568}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19004","reported_seq":19081,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.556976+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:44.556976+0000","last_peered":"2024-09-15T21:29:44.556976+0000","last_clean":"2024-09-15T21:29:44.556976+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:44.556976+0000","last_undegraded":"2024-09-15T21:29:44.556976+0000","last_fullsized":"2024-09-15T21:29:44.556976+0000","mapping_epoch":35,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5365760,"num_objects":1310,"num_object_clones":0,"num_object_copies":2620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19004,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19147","reported_seq":19285,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.102945+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:44.102945+0000","last_peered":"2024-09-15T21:29:44.102945+0000","last_clean":"2024-09-15T21:29:44.102945+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:44.102945+0000","last_undegraded":"2024-09-15T21:29:44.102945+0000","last_fullsized":"2024-09-15T21:29:44.102945+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5562851,"num_objects":1359,"num_object_clones":0,"num_object_copies":2718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15602,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19245","reported_seq":19384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.115768+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:44.115768+0000","last_peered":"2024-09-15T21:29:44.115768+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:44.115768+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5693923,"num_objects":1391,"num_object_clones":0,"num_object_copies":2782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15921,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2685},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19347","reported_seq":19487,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.011797+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:44.011797+0000","last_peered":"2024-09-15T21:29:44.011797+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:44.011797+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5419974,"num_objects":1325,"num_object_clones":0,"num_object_copies":2650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15658,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2836},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19528","reported_seq":19663,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.046227+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:44.046227+0000","last_peered":"2024-09-15T21:29:44.046227+0000","last_clean":"2024-09-15T21:29:44.046227+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:44.046227+0000","last_undegraded":"2024-09-15T21:29:44.046227+0000","last_fullsized":"2024-09-15T21:29:44.046227+0000","mapping_epoch":34,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5488640,"num_objects":1340,"num_object_clones":0,"num_object_copies":2680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16300,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19220","reported_seq":19353,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.512220+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:44.512220+0000","last_peered":"2024-09-15T21:29:44.512220+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:44.512220+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5296611,"num_objects":1294,"num_object_clones":0,"num_object_copies":2588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1294,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17152,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2793}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19083","reported_seq":19220,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:43.973812+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:43.973812+0000","last_peered":"2024-09-15T21:29:43.973812+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:43.973812+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5730304,"num_objects":1399,"num_object_clones":0,"num_object_copies":2798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15756,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2621}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19328","reported_seq":19453,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.566167+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:44.566167+0000","last_peered":"2024-09-15T21:29:44.566167+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:44.566167+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5386723,"num_objects":1316,"num_object_clones":0,"num_object_copies":2632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17187,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2774}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19290","reported_seq":27755,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.566800+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:44.566800+0000","last_peered":"2024-09-15T21:29:44.566800+0000","last_clean":"2024-09-15T21:29:44.566800+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:44.566800+0000","last_undegraded":"2024-09-15T21:29:44.566800+0000","last_fullsized":"2024-09-15T21:29:44.566800+0000","mapping_epoch":44,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5201920,"num_objects":1270,"num_object_clones":0,"num_object_copies":2540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16983,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19310","reported_seq":19450,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.082976+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:44.082976+0000","last_peered":"2024-09-15T21:29:44.082976+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:44.082976+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":1376,"num_object_clones":0,"num_object_copies":2752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16254,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2718}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19211","reported_seq":19350,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.033998+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:44.033998+0000","last_peered":"2024-09-15T21:29:44.033998+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:44.033998+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5648384,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16120,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19083","reported_seq":27399,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.818747+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:45.818747+0000","last_peered":"2024-09-15T21:29:45.818747+0000","last_clean":"2024-09-15T21:29:45.818747+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:45.818747+0000","last_undegraded":"2024-09-15T21:29:45.818747+0000","last_fullsized":"2024-09-15T21:29:45.818747+0000","mapping_epoch":46,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5345280,"num_objects":1305,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15567,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19480","reported_seq":19625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.061334+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:44.061334+0000","last_peered":"2024-09-15T21:29:44.061334+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:44.061334+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5678022,"num_objects":1388,"num_object_clones":0,"num_object_copies":2776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1388,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16184,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2759},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":88138486,"num_objects":21528,"num_object_clones":0,"num_object_copies":43056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37753,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21528,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":268927,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11721,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376971264,"data_stored":376825826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526656,"kb_used_data":174248,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93845184,"statfs":{"total":96636764160,"available":96097468416,"internally_reserved":0,"allocated":178429952,"data_stored":178144694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597784,"kb_used_data":132976,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93774056,"statfs":{"total":96636764160,"available":96024633344,"internally_reserved":0,"allocated":136167424,"data_stored":135882161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475917995},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532144,"kb_used_data":63132,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93839696,"statfs":{"total":96636764160,"available":96091848704,"internally_reserved":0,"allocated":64647168,"data_stored":64328929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":134971392,"data_stored":134935262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64237568,"data_stored":64168462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177762304,"data_stored":177722102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:49.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:29:49.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19777", 'reported_seq': 19859, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:44.084287+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:29:44.084287+0000', 'last_peered': '2024-09-15T21:29:44.084287+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:29:44.084287+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5632000, 'num_objects': 1375, 'num_object_clones': 0, 'num_object_copies': 2750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1375, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19777, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2757}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:49.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:29:49.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'19742", 'reported_seq': 26819, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:44.569508+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:29:44.569508+0000', 'last_peered': '2024-09-15T21:29:44.569508+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:29:44.569508+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5582872, 'num_objects': 1364, 'num_object_clones': 0, 'num_object_copies': 2728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 796, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1364, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17842, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3472, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 568}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:49.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:29:49.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19245", 'reported_seq': 19384, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:44.115768+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:29:44.115768+0000', 'last_peered': '2024-09-15T21:29:44.115768+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:29:44.115768+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5693923, 'num_objects': 1391, 'num_object_clones': 0, 'num_object_copies': 2782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1391, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15921, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2685}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:49.233 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:49.233 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19347", 'reported_seq': 19487, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:44.011797+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:29:44.011797+0000', 'last_peered': '2024-09-15T21:29:44.011797+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:29:44.011797+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5419974, 'num_objects': 1325, 'num_object_clones': 0, 'num_object_copies': 2650, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1325, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15658, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2836}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:49.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:29:49.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'19220", 'reported_seq': 19353, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:44.512220+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:29:44.512220+0000', 'last_peered': '2024-09-15T21:29:44.512220+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:29:44.512220+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5296611, 'num_objects': 1294, 'num_object_clones': 0, 'num_object_copies': 2588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1294, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17152, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2793}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:49.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:49.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19083", 'reported_seq': 19220, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:43.973812+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:29:43.973812+0000', 'last_peered': '2024-09-15T21:29:43.973812+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:29:43.973812+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5730304, 'num_objects': 1399, 'num_object_clones': 0, 'num_object_copies': 2798, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1399, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15756, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2621}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:49.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:29:49.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19328", 'reported_seq': 19453, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:44.566167+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:29:44.566167+0000', 'last_peered': '2024-09-15T21:29:44.566167+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:29:44.566167+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5386723, 'num_objects': 1316, 'num_object_clones': 0, 'num_object_copies': 2632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1316, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17187, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2774}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:49.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:29:49.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19310", 'reported_seq': 19450, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:44.082976+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:29:44.082976+0000', 'last_peered': '2024-09-15T21:29:44.082976+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:29:44.082976+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5636096, 'num_objects': 1376, 'num_object_clones': 0, 'num_object_copies': 2752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1376, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16254, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2718}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:49.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:29:49.234 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'19211", 'reported_seq': 19350, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:44.033998+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:29:44.033998+0000', 'last_peered': '2024-09-15T21:29:44.033998+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:29:44.033998+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5648384, 'num_objects': 1379, 'num_object_clones': 0, 'num_object_copies': 2758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1379, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16120, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2760}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:49.234 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:29:49.235 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'19480", 'reported_seq': 19625, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:44.061334+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:29:44.061334+0000', 'last_peered': '2024-09-15T21:29:44.061334+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:29:44.061334+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5678022, 'num_objects': 1388, 'num_object_clones': 0, 'num_object_copies': 2776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1388, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16184, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2759}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:49.235 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:49.930 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:49.931 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37753/43060 objects degraded (87.675%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":641,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":21530,"data_bytes":88138505,"bytes_used":1696342016,"bytes_avail":288213950464,"bytes_total":289910292480,"degraded_objects":37753,"degraded_total":43060,"degraded_ratio":0.87675336739433352,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":395},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:49.951 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:49.951 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:50.529 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:50.531 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:50.554 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-15T21:29:48.864686+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":88138505,"num_objects":21530,"num_object_clones":0,"num_object_copies":43060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37753,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21530,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":268929,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11722,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160737,"ondisk_log_size":160737,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1656584,"kb_used_data":370356,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281458936,"statfs":{"total":289910292480,"available":288213950464,"internally_reserved":0,"allocated":379244544,"data_stored":378355784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1316998147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,3,7,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19447808,"num_objects":-4748,"num_object_clones":0,"num_object_copies":-9496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-274,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4748,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004588"},"pg_stats":[{"pgid":"2.f","version":"47'19777","reported_seq":19859,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.084287+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:44.084287+0000","last_peered":"2024-09-15T21:29:44.084287+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:44.084287+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5632000,"num_objects":1375,"num_object_clones":0,"num_object_copies":2750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19777,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2757}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19339","reported_seq":19451,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.590497+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:44.590497+0000","last_peered":"2024-09-15T21:29:44.590497+0000","last_clean":"2024-09-15T21:29:44.590497+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:44.590497+0000","last_undegraded":"2024-09-15T21:29:44.590497+0000","last_fullsized":"2024-09-15T21:29:44.590497+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5469126,"num_objects":1337,"num_object_clones":0,"num_object_copies":2674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1337,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17620,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19742","reported_seq":26819,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.569508+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:44.569508+0000","last_peered":"2024-09-15T21:29:44.569508+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:44.569508+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5582872,"num_objects":1364,"num_object_clones":0,"num_object_copies":2728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17842,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3472,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":568}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19004","reported_seq":19081,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.556976+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:44.556976+0000","last_peered":"2024-09-15T21:29:44.556976+0000","last_clean":"2024-09-15T21:29:44.556976+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:44.556976+0000","last_undegraded":"2024-09-15T21:29:44.556976+0000","last_fullsized":"2024-09-15T21:29:44.556976+0000","mapping_epoch":35,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5365760,"num_objects":1310,"num_object_clones":0,"num_object_copies":2620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19004,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19147","reported_seq":19285,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.102945+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:44.102945+0000","last_peered":"2024-09-15T21:29:44.102945+0000","last_clean":"2024-09-15T21:29:44.102945+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:44.102945+0000","last_undegraded":"2024-09-15T21:29:44.102945+0000","last_fullsized":"2024-09-15T21:29:44.102945+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5562851,"num_objects":1359,"num_object_clones":0,"num_object_copies":2718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15602,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19245","reported_seq":19384,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.115768+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:44.115768+0000","last_peered":"2024-09-15T21:29:44.115768+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:44.115768+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5693923,"num_objects":1391,"num_object_clones":0,"num_object_copies":2782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1391,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15921,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2685},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19347","reported_seq":19487,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.011797+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:44.011797+0000","last_peered":"2024-09-15T21:29:44.011797+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:44.011797+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5419974,"num_objects":1325,"num_object_clones":0,"num_object_copies":2650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1325,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15658,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2836},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19528","reported_seq":19663,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.046227+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:44.046227+0000","last_peered":"2024-09-15T21:29:44.046227+0000","last_clean":"2024-09-15T21:29:44.046227+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:44.046227+0000","last_undegraded":"2024-09-15T21:29:44.046227+0000","last_fullsized":"2024-09-15T21:29:44.046227+0000","mapping_epoch":34,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5488640,"num_objects":1340,"num_object_clones":0,"num_object_copies":2680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16300,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19220","reported_seq":19353,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.512220+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:44.512220+0000","last_peered":"2024-09-15T21:29:44.512220+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:44.512220+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5296611,"num_objects":1294,"num_object_clones":0,"num_object_copies":2588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1294,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17152,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2793}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19083","reported_seq":19220,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:43.973812+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:43.973812+0000","last_peered":"2024-09-15T21:29:43.973812+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:43.973812+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5730304,"num_objects":1399,"num_object_clones":0,"num_object_copies":2798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15756,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2621}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19328","reported_seq":19453,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.566167+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:44.566167+0000","last_peered":"2024-09-15T21:29:44.566167+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:44.566167+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5386723,"num_objects":1316,"num_object_clones":0,"num_object_copies":2632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1316,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17187,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2774}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19290","reported_seq":27755,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:44.566800+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:44.566800+0000","last_peered":"2024-09-15T21:29:44.566800+0000","last_clean":"2024-09-15T21:29:44.566800+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:44.566800+0000","last_undegraded":"2024-09-15T21:29:44.566800+0000","last_fullsized":"2024-09-15T21:29:44.566800+0000","mapping_epoch":44,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5201920,"num_objects":1270,"num_object_clones":0,"num_object_copies":2540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16983,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19310","reported_seq":19450,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.082976+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:44.082976+0000","last_peered":"2024-09-15T21:29:44.082976+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:44.082976+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5636096,"num_objects":1376,"num_object_clones":0,"num_object_copies":2752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1376,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16254,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2718}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19211","reported_seq":19350,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.033998+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:44.033998+0000","last_peered":"2024-09-15T21:29:44.033998+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:44.033998+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5648384,"num_objects":1379,"num_object_clones":0,"num_object_copies":2758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16120,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2760}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19083","reported_seq":27399,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:45.818747+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:45.818747+0000","last_peered":"2024-09-15T21:29:45.818747+0000","last_clean":"2024-09-15T21:29:45.818747+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:45.818747+0000","last_undegraded":"2024-09-15T21:29:45.818747+0000","last_fullsized":"2024-09-15T21:29:45.818747+0000","mapping_epoch":46,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5345280,"num_objects":1305,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15567,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19480","reported_seq":19625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:44.061334+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:44.061334+0000","last_peered":"2024-09-15T21:29:44.061334+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:44.061334+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5678022,"num_objects":1388,"num_object_clones":0,"num_object_copies":2776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1388,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16184,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2759},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":88138486,"num_objects":21528,"num_object_clones":0,"num_object_copies":43056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37753,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21528,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":268927,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11721,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376971264,"data_stored":376825826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160734,"ondisk_log_size":160734,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526656,"kb_used_data":174248,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93845184,"statfs":{"total":96636764160,"available":96097468416,"internally_reserved":0,"allocated":178429952,"data_stored":178144694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597784,"kb_used_data":132976,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93774056,"statfs":{"total":96636764160,"available":96024633344,"internally_reserved":0,"allocated":136167424,"data_stored":135882161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475917995},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532144,"kb_used_data":63132,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93839696,"statfs":{"total":96636764160,"available":96091848704,"internally_reserved":0,"allocated":64647168,"data_stored":64328929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":134971392,"data_stored":134935262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":64237568,"data_stored":64168462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177762304,"data_stored":177722102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:53.557 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:54.151 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:54.152 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:54.176 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-15T21:29:52.866204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":78209801,"num_objects":19106,"num_object_clones":0,"num_object_copies":38212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19106,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":271353,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11843,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160761,"ondisk_log_size":160761,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1649976,"kb_used_data":351264,"kb_used_omap":12,"kb_used_meta":1298611,"kb_avail":281465544,"statfs":{"total":289910292480,"available":288220717056,"internally_reserved":0,"allocated":359694336,"data_stored":358805576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1329777667},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,9,6,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19689472,"num_objects":-4807,"num_object_clones":0,"num_object_copies":-9614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4807,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004563"},"pg_stats":[{"pgid":"2.f","version":"47'19932","reported_seq":20014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.113690+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:49.113690+0000","last_peered":"2024-09-15T21:29:49.113690+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:49.113690+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4997120,"num_objects":1220,"num_object_clones":0,"num_object_copies":2440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19932,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2912}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19489","reported_seq":19601,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:49.380344+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:49.380344+0000","last_peered":"2024-09-15T21:29:49.380344+0000","last_clean":"2024-09-15T21:29:49.380344+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:49.380344+0000","last_undegraded":"2024-09-15T21:29:49.380344+0000","last_fullsized":"2024-09-15T21:29:49.380344+0000","mapping_epoch":35,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4854726,"num_objects":1187,"num_object_clones":0,"num_object_copies":2374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1187,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17770,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19919","reported_seq":27237,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.608387+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:49.608387+0000","last_peered":"2024-09-15T21:29:49.608387+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:49.608387+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4857880,"num_objects":1187,"num_object_clones":0,"num_object_copies":2374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1187,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18019,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3593,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":512}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19160","reported_seq":19237,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:49.510966+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:49.510966+0000","last_peered":"2024-09-15T21:29:49.510966+0000","last_clean":"2024-09-15T21:29:49.510966+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:49.510966+0000","last_undegraded":"2024-09-15T21:29:49.510966+0000","last_fullsized":"2024-09-15T21:29:49.510966+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4726784,"num_objects":1154,"num_object_clones":0,"num_object_copies":2308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19160,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19314","reported_seq":19452,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:49.177937+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:49.177937+0000","last_peered":"2024-09-15T21:29:49.177937+0000","last_clean":"2024-09-15T21:29:49.177937+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:49.177937+0000","last_undegraded":"2024-09-15T21:29:49.177937+0000","last_fullsized":"2024-09-15T21:29:49.177937+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4878819,"num_objects":1192,"num_object_clones":0,"num_object_copies":2384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15769,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19383","reported_seq":19526,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.131793+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:49.131793+0000","last_peered":"2024-09-15T21:29:49.131793+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:49.131793+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5128675,"num_objects":1253,"num_object_clones":0,"num_object_copies":2506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1253,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16059,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2823},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19501","reported_seq":19641,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.082533+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:49.082533+0000","last_peered":"2024-09-15T21:29:49.082533+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:49.082533+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4789190,"num_objects":1171,"num_object_clones":0,"num_object_copies":2342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15812,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2990},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19663","reported_seq":19798,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:49.118378+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:49.118378+0000","last_peered":"2024-09-15T21:29:49.118378+0000","last_clean":"2024-09-15T21:29:49.118378+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:49.118378+0000","last_undegraded":"2024-09-15T21:29:49.118378+0000","last_fullsized":"2024-09-15T21:29:49.118378+0000","mapping_epoch":34,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4935680,"num_objects":1205,"num_object_clones":0,"num_object_copies":2410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16435,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19370","reported_seq":19503,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.588441+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:49.588441+0000","last_peered":"2024-09-15T21:29:49.588441+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:49.588441+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4682211,"num_objects":1144,"num_object_clones":0,"num_object_copies":2288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1144,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17302,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2943}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19244","reported_seq":19381,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.126591+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:49.126591+0000","last_peered":"2024-09-15T21:29:49.126591+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:49.126591+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5070848,"num_objects":1238,"num_object_clones":0,"num_object_copies":2476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15917,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2782}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19477","reported_seq":19601,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.611226+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:49.611226+0000","last_peered":"2024-09-15T21:29:49.611226+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:49.611226+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4776419,"num_objects":1167,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1167,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17336,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2923}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19441","reported_seq":27906,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:49.599128+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:49.599128+0000","last_peered":"2024-09-15T21:29:49.599128+0000","last_clean":"2024-09-15T21:29:49.599128+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:49.599128+0000","last_undegraded":"2024-09-15T21:29:49.599128+0000","last_fullsized":"2024-09-15T21:29:49.599128+0000","mapping_epoch":44,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4583424,"num_objects":1119,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17134,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19452","reported_seq":19591,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.150121+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:49.150121+0000","last_peered":"2024-09-15T21:29:49.150121+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:49.150121+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5054464,"num_objects":1234,"num_object_clones":0,"num_object_copies":2468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16396,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19363","reported_seq":19502,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.060998+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:49.060998+0000","last_peered":"2024-09-15T21:29:49.060998+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:49.060998+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5025792,"num_objects":1227,"num_object_clones":0,"num_object_copies":2454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16272,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2912}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19228","reported_seq":27544,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:50.835349+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:50.835349+0000","last_peered":"2024-09-15T21:29:50.835349+0000","last_clean":"2024-09-15T21:29:50.835349+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:50.835349+0000","last_undegraded":"2024-09-15T21:29:50.835349+0000","last_fullsized":"2024-09-15T21:29:50.835349+0000","mapping_epoch":46,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4751360,"num_objects":1160,"num_object_clones":0,"num_object_copies":2320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15712,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19622","reported_seq":19767,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.055981+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:49.055981+0000","last_peered":"2024-09-15T21:29:49.055981+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:49.055981+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5096390,"num_objects":1246,"num_object_clones":0,"num_object_copies":2492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1246,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16326,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2901},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":78209782,"num_objects":19104,"num_object_clones":0,"num_object_copies":38208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19104,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":271351,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11842,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":357269504,"data_stored":357124066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525920,"kb_used_data":169356,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93845920,"statfs":{"total":96636764160,"available":96098222080,"internally_reserved":0,"allocated":173420544,"data_stored":173135286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594748,"kb_used_data":125800,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93777092,"statfs":{"total":96636764160,"available":96027742208,"internally_reserved":0,"allocated":128819200,"data_stored":128533937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529308,"kb_used_data":56108,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93842532,"statfs":{"total":96636764160,"available":96094752768,"internally_reserved":0,"allocated":57454592,"data_stored":57136353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":127574016,"data_stored":127537886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":56958976,"data_stored":56889870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":172736512,"data_stored":172696310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:54.178 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:54.726 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:54.728 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:54.753 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-15T21:29:52.866204+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":78209801,"num_objects":19106,"num_object_clones":0,"num_object_copies":38212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19106,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":271353,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11843,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160761,"ondisk_log_size":160761,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1649976,"kb_used_data":351264,"kb_used_omap":12,"kb_used_meta":1298611,"kb_avail":281465544,"statfs":{"total":289910292480,"available":288220717056,"internally_reserved":0,"allocated":359694336,"data_stored":358805576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1329777667},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,9,6,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19689472,"num_objects":-4807,"num_object_clones":0,"num_object_copies":-9614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-269,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4807,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":269,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004563"},"pg_stats":[{"pgid":"2.f","version":"47'19932","reported_seq":20014,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.113690+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:49.113690+0000","last_peered":"2024-09-15T21:29:49.113690+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:49.113690+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4997120,"num_objects":1220,"num_object_clones":0,"num_object_copies":2440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19932,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2912}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19489","reported_seq":19601,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:49.380344+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:49.380344+0000","last_peered":"2024-09-15T21:29:49.380344+0000","last_clean":"2024-09-15T21:29:49.380344+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:49.380344+0000","last_undegraded":"2024-09-15T21:29:49.380344+0000","last_fullsized":"2024-09-15T21:29:49.380344+0000","mapping_epoch":35,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4854726,"num_objects":1187,"num_object_clones":0,"num_object_copies":2374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1187,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17770,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'19919","reported_seq":27237,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.608387+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:49.608387+0000","last_peered":"2024-09-15T21:29:49.608387+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:49.608387+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4857880,"num_objects":1187,"num_object_clones":0,"num_object_copies":2374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":675,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1187,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18019,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3593,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":512}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19160","reported_seq":19237,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:49.510966+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:49.510966+0000","last_peered":"2024-09-15T21:29:49.510966+0000","last_clean":"2024-09-15T21:29:49.510966+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:49.510966+0000","last_undegraded":"2024-09-15T21:29:49.510966+0000","last_fullsized":"2024-09-15T21:29:49.510966+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4726784,"num_objects":1154,"num_object_clones":0,"num_object_copies":2308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19160,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19314","reported_seq":19452,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:49.177937+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:49.177937+0000","last_peered":"2024-09-15T21:29:49.177937+0000","last_clean":"2024-09-15T21:29:49.177937+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:49.177937+0000","last_undegraded":"2024-09-15T21:29:49.177937+0000","last_fullsized":"2024-09-15T21:29:49.177937+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4878819,"num_objects":1192,"num_object_clones":0,"num_object_copies":2384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15769,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19383","reported_seq":19526,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.131793+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:49.131793+0000","last_peered":"2024-09-15T21:29:49.131793+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:49.131793+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5128675,"num_objects":1253,"num_object_clones":0,"num_object_copies":2506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1253,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16059,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2823},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19501","reported_seq":19641,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.082533+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:49.082533+0000","last_peered":"2024-09-15T21:29:49.082533+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:49.082533+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4789190,"num_objects":1171,"num_object_clones":0,"num_object_copies":2342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1171,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15812,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2990},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19663","reported_seq":19798,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:49.118378+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:49.118378+0000","last_peered":"2024-09-15T21:29:49.118378+0000","last_clean":"2024-09-15T21:29:49.118378+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:49.118378+0000","last_undegraded":"2024-09-15T21:29:49.118378+0000","last_fullsized":"2024-09-15T21:29:49.118378+0000","mapping_epoch":34,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4935680,"num_objects":1205,"num_object_clones":0,"num_object_copies":2410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16435,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19370","reported_seq":19503,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.588441+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:49.588441+0000","last_peered":"2024-09-15T21:29:49.588441+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:49.588441+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4682211,"num_objects":1144,"num_object_clones":0,"num_object_copies":2288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1144,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17302,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2943}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19244","reported_seq":19381,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.126591+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:49.126591+0000","last_peered":"2024-09-15T21:29:49.126591+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:49.126591+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5070848,"num_objects":1238,"num_object_clones":0,"num_object_copies":2476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15917,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2782}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19477","reported_seq":19601,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.611226+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:49.611226+0000","last_peered":"2024-09-15T21:29:49.611226+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:49.611226+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4776419,"num_objects":1167,"num_object_clones":0,"num_object_copies":2334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1167,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17336,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-2923}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19441","reported_seq":27906,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:49.599128+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:49.599128+0000","last_peered":"2024-09-15T21:29:49.599128+0000","last_clean":"2024-09-15T21:29:49.599128+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:49.599128+0000","last_undegraded":"2024-09-15T21:29:49.599128+0000","last_fullsized":"2024-09-15T21:29:49.599128+0000","mapping_epoch":44,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4583424,"num_objects":1119,"num_object_clones":0,"num_object_copies":2238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17134,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19452","reported_seq":19591,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.150121+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:49.150121+0000","last_peered":"2024-09-15T21:29:49.150121+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:49.150121+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5054464,"num_objects":1234,"num_object_clones":0,"num_object_copies":2468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16396,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2860}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19363","reported_seq":19502,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.060998+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:49.060998+0000","last_peered":"2024-09-15T21:29:49.060998+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:49.060998+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5025792,"num_objects":1227,"num_object_clones":0,"num_object_copies":2454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16272,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2912}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19228","reported_seq":27544,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:50.835349+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:50.835349+0000","last_peered":"2024-09-15T21:29:50.835349+0000","last_clean":"2024-09-15T21:29:50.835349+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:50.835349+0000","last_undegraded":"2024-09-15T21:29:50.835349+0000","last_fullsized":"2024-09-15T21:29:50.835349+0000","mapping_epoch":46,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4751360,"num_objects":1160,"num_object_clones":0,"num_object_copies":2320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15712,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19622","reported_seq":19767,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:49.055981+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:49.055981+0000","last_peered":"2024-09-15T21:29:49.055981+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:49.055981+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5096390,"num_objects":1246,"num_object_clones":0,"num_object_copies":2492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1246,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16326,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2901},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":78209782,"num_objects":19104,"num_object_clones":0,"num_object_copies":38208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19104,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":271351,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11842,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":357269504,"data_stored":357124066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160758,"ondisk_log_size":160758,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525920,"kb_used_data":169356,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93845920,"statfs":{"total":96636764160,"available":96098222080,"internally_reserved":0,"allocated":173420544,"data_stored":173135286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594748,"kb_used_data":125800,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93777092,"statfs":{"total":96636764160,"available":96027742208,"internally_reserved":0,"allocated":128819200,"data_stored":128533937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529308,"kb_used_data":56108,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93842532,"statfs":{"total":96636764160,"available":96094752768,"internally_reserved":0,"allocated":57454592,"data_stored":57136353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":127574016,"data_stored":127537886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":56958976,"data_stored":56889870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":172736512,"data_stored":172696310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:54.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:29:54.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'19932", 'reported_seq': 20014, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:49.113690+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:29:49.113690+0000', 'last_peered': '2024-09-15T21:29:49.113690+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:29:49.113690+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4997120, 'num_objects': 1220, 'num_object_clones': 0, 'num_object_copies': 2440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19932, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2912}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:54.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:29:54.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'19919", 'reported_seq': 27237, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:49.608387+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:29:49.608387+0000', 'last_peered': '2024-09-15T21:29:49.608387+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:29:49.608387+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4857880, 'num_objects': 1187, 'num_object_clones': 0, 'num_object_copies': 2374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 675, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1187, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18019, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3593, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 512}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:54.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:29:54.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19383", 'reported_seq': 19526, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:49.131793+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:29:49.131793+0000', 'last_peered': '2024-09-15T21:29:49.131793+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:29:49.131793+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5128675, 'num_objects': 1253, 'num_object_clones': 0, 'num_object_copies': 2506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1253, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16059, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2823}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:54.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:29:54.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19501", 'reported_seq': 19641, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:49.082533+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:29:49.082533+0000', 'last_peered': '2024-09-15T21:29:49.082533+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:29:49.082533+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4789190, 'num_objects': 1171, 'num_object_clones': 0, 'num_object_copies': 2342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1171, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15812, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2990}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:54.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:29:54.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'19370", 'reported_seq': 19503, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:49.588441+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:29:49.588441+0000', 'last_peered': '2024-09-15T21:29:49.588441+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:29:49.588441+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4682211, 'num_objects': 1144, 'num_object_clones': 0, 'num_object_copies': 2288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1144, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17302, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2943}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:54.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:29:54.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19244", 'reported_seq': 19381, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:49.126591+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:29:49.126591+0000', 'last_peered': '2024-09-15T21:29:49.126591+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:29:49.126591+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5070848, 'num_objects': 1238, 'num_object_clones': 0, 'num_object_copies': 2476, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1238, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15917, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2782}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:54.755 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:29:54.755 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19477", 'reported_seq': 19601, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:49.611226+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:29:49.611226+0000', 'last_peered': '2024-09-15T21:29:49.611226+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:29:49.611226+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4776419, 'num_objects': 1167, 'num_object_clones': 0, 'num_object_copies': 2334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1167, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17336, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -2923}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:29:54.756 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:29:54.756 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19452", 'reported_seq': 19591, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:49.150121+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:29:49.150121+0000', 'last_peered': '2024-09-15T21:29:49.150121+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:29:49.150121+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5054464, 'num_objects': 1234, 'num_object_clones': 0, 'num_object_copies': 2468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1234, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16396, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2860}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:54.756 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:29:54.756 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'19363", 'reported_seq': 19502, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:49.060998+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:29:49.060998+0000', 'last_peered': '2024-09-15T21:29:49.060998+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:29:49.060998+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5025792, 'num_objects': 1227, 'num_object_clones': 0, 'num_object_copies': 2454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1227, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16272, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2912}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:54.756 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:29:54.756 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'19622", 'reported_seq': 19767, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:49.055981+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:29:49.055981+0000', 'last_peered': '2024-09-15T21:29:49.055981+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:29:49.055981+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5096390, 'num_objects': 1246, 'num_object_clones': 0, 'num_object_copies': 2492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1246, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16326, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2901}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:29:54.756 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:29:55.454 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:55.454 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37632/38212 objects degraded (98.482%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":647,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":19106,"data_bytes":78209801,"bytes_used":1689575424,"bytes_avail":288220717056,"bytes_total":289910292480,"degraded_objects":37632,"degraded_total":38212,"degraded_ratio":0.98482152203496287,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":269,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":400},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:29:55.478 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:29:55.479 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:56.080 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:56.083 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:56.107 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-09-15T21:29:54.867022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":68977417,"num_objects":16852,"num_object_clones":0,"num_object_copies":33704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16852,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":273607,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11990,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160715,"ondisk_log_size":160715,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1633804,"kb_used_data":339256,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281481716,"statfs":{"total":289910292480,"available":288237277184,"internally_reserved":0,"allocated":347398144,"data_stored":346509384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1325517827},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,4,8,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28921856,"num_objects":-7061,"num_object_clones":0,"num_object_copies":-14122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-416,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7061,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":416,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004643"},"pg_stats":[{"pgid":"2.f","version":"47'20094","reported_seq":20176,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.005848+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:54.005848+0000","last_peered":"2024-09-15T21:29:54.005848+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:54.005848+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4333568,"num_objects":1058,"num_object_clones":0,"num_object_copies":2116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20094,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3074}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19636","reported_seq":19747,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:54.544216+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:54.544216+0000","last_peered":"2024-09-15T21:29:54.544216+0000","last_clean":"2024-09-15T21:29:54.544216+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:54.544216+0000","last_undegraded":"2024-09-15T21:29:54.544216+0000","last_fullsized":"2024-09-15T21:29:54.544216+0000","mapping_epoch":35,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4252614,"num_objects":1040,"num_object_clones":0,"num_object_copies":2080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1040,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17917,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20070","reported_seq":27684,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.626740+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:54.626740+0000","last_peered":"2024-09-15T21:29:54.626740+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:54.626740+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4239384,"num_objects":1036,"num_object_clones":0,"num_object_copies":2072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1036,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18170,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3740,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":508}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19309","reported_seq":19384,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:54.618339+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:54.618339+0000","last_peered":"2024-09-15T21:29:54.618339+0000","last_clean":"2024-09-15T21:29:54.618339+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:54.618339+0000","last_undegraded":"2024-09-15T21:29:54.618339+0000","last_fullsized":"2024-09-15T21:29:54.618339+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4116480,"num_objects":1005,"num_object_clones":0,"num_object_copies":2010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19309,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19486","reported_seq":19625,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:53.893571+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:53.893571+0000","last_peered":"2024-09-15T21:29:53.893571+0000","last_clean":"2024-09-15T21:29:53.893571+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:53.893571+0000","last_undegraded":"2024-09-15T21:29:53.893571+0000","last_fullsized":"2024-09-15T21:29:53.893571+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4174307,"num_objects":1020,"num_object_clones":0,"num_object_copies":2040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15941,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19550","reported_seq":19692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.203323+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:54.203323+0000","last_peered":"2024-09-15T21:29:54.203323+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:54.203323+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4444643,"num_objects":1086,"num_object_clones":0,"num_object_copies":2172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1086,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16226,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2990},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19658","reported_seq":19801,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.142011+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:54.142011+0000","last_peered":"2024-09-15T21:29:54.142011+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:54.142011+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4146118,"num_objects":1014,"num_object_clones":0,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15969,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3147},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19811","reported_seq":19946,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:54.169002+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:54.169002+0000","last_peered":"2024-09-15T21:29:54.169002+0000","last_clean":"2024-09-15T21:29:54.169002+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:54.169002+0000","last_undegraded":"2024-09-15T21:29:54.169002+0000","last_fullsized":"2024-09-15T21:29:54.169002+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4329472,"num_objects":1057,"num_object_clones":0,"num_object_copies":2114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16583,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19519","reported_seq":19652,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.641664+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:54.641664+0000","last_peered":"2024-09-15T21:29:54.641664+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:54.641664+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4071907,"num_objects":995,"num_object_clones":0,"num_object_copies":1990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":995,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17451,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3092}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19392","reported_seq":19529,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.092398+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:54.092398+0000","last_peered":"2024-09-15T21:29:54.092398+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:54.092398+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4464640,"num_objects":1090,"num_object_clones":0,"num_object_copies":2180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16065,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2930}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19622","reported_seq":19747,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.546680+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:54.546680+0000","last_peered":"2024-09-15T21:29:54.546680+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:54.546680+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4182499,"num_objects":1022,"num_object_clones":0,"num_object_copies":2044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17481,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3068}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19556","reported_seq":28021,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:54.592018+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:54.592018+0000","last_peered":"2024-09-15T21:29:54.592018+0000","last_clean":"2024-09-15T21:29:54.592018+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:54.592018+0000","last_undegraded":"2024-09-15T21:29:54.592018+0000","last_fullsized":"2024-09-15T21:29:54.592018+0000","mapping_epoch":44,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4112384,"num_objects":1004,"num_object_clones":0,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17249,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19608","reported_seq":19748,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.139479+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:54.139479+0000","last_peered":"2024-09-15T21:29:54.139479+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:54.139479+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4415488,"num_objects":1078,"num_object_clones":0,"num_object_copies":2156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16552,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3016}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19501","reported_seq":19638,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.203220+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:54.203220+0000","last_peered":"2024-09-15T21:29:54.203220+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:54.203220+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4460544,"num_objects":1089,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16410,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3050}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19228","reported_seq":27544,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:50.835349+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:50.835349+0000","last_peered":"2024-09-15T21:29:50.835349+0000","last_clean":"2024-09-15T21:29:50.835349+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:50.835349+0000","last_undegraded":"2024-09-15T21:29:50.835349+0000","last_fullsized":"2024-09-15T21:29:50.835349+0000","mapping_epoch":46,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4751360,"num_objects":1160,"num_object_clones":0,"num_object_copies":2320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15712,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.317401+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:09.317401+0000","last_peered":"2024-09-15T21:27:09.317401+0000","last_clean":"2024-09-15T21:27:09.317401+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:09.317401+0000","last_undegraded":"2024-09-15T21:27:09.317401+0000","last_fullsized":"2024-09-15T21:27:09.317401+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19772","reported_seq":19916,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.158217+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:54.158217+0000","last_peered":"2024-09-15T21:29:54.158217+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:54.158217+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4481990,"num_objects":1096,"num_object_clones":0,"num_object_copies":2192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16476,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3051},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":68977398,"num_objects":16850,"num_object_clones":0,"num_object_copies":33700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16850,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":273605,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11989,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":344989696,"data_stored":344844258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160712,"ondisk_log_size":160712,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525504,"kb_used_data":164784,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93846336,"statfs":{"total":96636764160,"available":96098648064,"internally_reserved":0,"allocated":168738816,"data_stored":168453558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,0,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578992,"kb_used_data":118364,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93792848,"statfs":{"total":96636764160,"available":96043876352,"internally_reserved":0,"allocated":121204736,"data_stored":120919473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471658155},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":529308,"kb_used_data":56108,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93842532,"statfs":{"total":96636764160,"available":96094752768,"internally_reserved":0,"allocated":57454592,"data_stored":57136353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":120025088,"data_stored":119988958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":56958976,"data_stored":56889870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":168005632,"data_stored":167965430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:59.109 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:29:59.646 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:29:59.647 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:29:59.672 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-15T21:29:58.868410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":68338441,"num_objects":16696,"num_object_clones":0,"num_object_copies":33392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16696,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":273763,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11990,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1618124,"kb_used_data":331916,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281497396,"statfs":{"total":289910292480,"available":288253333504,"internally_reserved":0,"allocated":339881984,"data_stored":338993224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1316998147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,3,15,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19800064,"num_objects":-4834,"num_object_clones":0,"num_object_copies":-9668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4834,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004392"},"pg_stats":[{"pgid":"2.f","version":"47'20094","reported_seq":20176,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.005848+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:54.005848+0000","last_peered":"2024-09-15T21:29:54.005848+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:54.005848+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4333568,"num_objects":1058,"num_object_clones":0,"num_object_copies":2116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20094,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3074}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19636","reported_seq":19747,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:54.544216+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:54.544216+0000","last_peered":"2024-09-15T21:29:54.544216+0000","last_clean":"2024-09-15T21:29:54.544216+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:54.544216+0000","last_undegraded":"2024-09-15T21:29:54.544216+0000","last_fullsized":"2024-09-15T21:29:54.544216+0000","mapping_epoch":35,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4252614,"num_objects":1040,"num_object_clones":0,"num_object_copies":2080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1040,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17917,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20070","reported_seq":27684,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.626740+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:54.626740+0000","last_peered":"2024-09-15T21:29:54.626740+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:54.626740+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4239384,"num_objects":1036,"num_object_clones":0,"num_object_copies":2072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1036,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18170,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3740,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":508}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19309","reported_seq":19384,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:54.618339+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:54.618339+0000","last_peered":"2024-09-15T21:29:54.618339+0000","last_clean":"2024-09-15T21:29:54.618339+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:54.618339+0000","last_undegraded":"2024-09-15T21:29:54.618339+0000","last_fullsized":"2024-09-15T21:29:54.618339+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4116480,"num_objects":1005,"num_object_clones":0,"num_object_copies":2010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19309,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19486","reported_seq":19625,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:53.893571+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:53.893571+0000","last_peered":"2024-09-15T21:29:53.893571+0000","last_clean":"2024-09-15T21:29:53.893571+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:53.893571+0000","last_undegraded":"2024-09-15T21:29:53.893571+0000","last_fullsized":"2024-09-15T21:29:53.893571+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4174307,"num_objects":1020,"num_object_clones":0,"num_object_copies":2040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15941,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19550","reported_seq":19692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.203323+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:54.203323+0000","last_peered":"2024-09-15T21:29:54.203323+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:54.203323+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4444643,"num_objects":1086,"num_object_clones":0,"num_object_copies":2172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1086,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16226,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2990},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19658","reported_seq":19801,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.142011+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:54.142011+0000","last_peered":"2024-09-15T21:29:54.142011+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:54.142011+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4146118,"num_objects":1014,"num_object_clones":0,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15969,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3147},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19811","reported_seq":19946,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:54.169002+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:54.169002+0000","last_peered":"2024-09-15T21:29:54.169002+0000","last_clean":"2024-09-15T21:29:54.169002+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:54.169002+0000","last_undegraded":"2024-09-15T21:29:54.169002+0000","last_fullsized":"2024-09-15T21:29:54.169002+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4329472,"num_objects":1057,"num_object_clones":0,"num_object_copies":2114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16583,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19519","reported_seq":19652,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.641664+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:54.641664+0000","last_peered":"2024-09-15T21:29:54.641664+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:54.641664+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4071907,"num_objects":995,"num_object_clones":0,"num_object_copies":1990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":995,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17451,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3092}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19392","reported_seq":19529,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.092398+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:54.092398+0000","last_peered":"2024-09-15T21:29:54.092398+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:54.092398+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4464640,"num_objects":1090,"num_object_clones":0,"num_object_copies":2180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16065,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2930}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19622","reported_seq":19747,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.546680+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:54.546680+0000","last_peered":"2024-09-15T21:29:54.546680+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:54.546680+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4182499,"num_objects":1022,"num_object_clones":0,"num_object_copies":2044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17481,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3068}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19556","reported_seq":28021,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:54.592018+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:54.592018+0000","last_peered":"2024-09-15T21:29:54.592018+0000","last_clean":"2024-09-15T21:29:54.592018+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:54.592018+0000","last_undegraded":"2024-09-15T21:29:54.592018+0000","last_fullsized":"2024-09-15T21:29:54.592018+0000","mapping_epoch":44,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4112384,"num_objects":1004,"num_object_clones":0,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17249,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19608","reported_seq":19748,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.139479+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:54.139479+0000","last_peered":"2024-09-15T21:29:54.139479+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:54.139479+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4415488,"num_objects":1078,"num_object_clones":0,"num_object_copies":2156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16552,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3016}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19501","reported_seq":19638,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.203220+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:54.203220+0000","last_peered":"2024-09-15T21:29:54.203220+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:54.203220+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4460544,"num_objects":1089,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16410,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3050}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19384","reported_seq":27698,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:55.917892+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:55.917892+0000","last_peered":"2024-09-15T21:29:55.917892+0000","last_clean":"2024-09-15T21:29:55.917892+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:55.917892+0000","last_undegraded":"2024-09-15T21:29:55.917892+0000","last_fullsized":"2024-09-15T21:29:55.917892+0000","mapping_epoch":46,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4112384,"num_objects":1004,"num_object_clones":0,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15868,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.526495+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:44.526495+0000","last_peered":"2024-09-15T21:27:44.526495+0000","last_clean":"2024-09-15T21:27:44.526495+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:44.526495+0000","last_undegraded":"2024-09-15T21:27:44.526495+0000","last_fullsized":"2024-09-15T21:27:44.526495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19772","reported_seq":19916,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.158217+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:54.158217+0000","last_peered":"2024-09-15T21:29:54.158217+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:54.158217+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4481990,"num_objects":1096,"num_object_clones":0,"num_object_copies":2192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16476,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3051},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":68338422,"num_objects":16694,"num_object_clones":0,"num_object_copies":33388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16694,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":273761,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11989,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":337514496,"data_stored":337369058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525504,"kb_used_data":164784,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93846336,"statfs":{"total":96636764160,"available":96098648064,"internally_reserved":0,"allocated":168738816,"data_stored":168453558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,0,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578992,"kb_used_data":118364,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93792848,"statfs":{"total":96636764160,"available":96043876352,"internally_reserved":0,"allocated":121204736,"data_stored":120919473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471658155},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513628,"kb_used_data":48768,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93858212,"statfs":{"total":96636764160,"available":96110809088,"internally_reserved":0,"allocated":49938432,"data_stored":49620193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":120025088,"data_stored":119988958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49483776,"data_stored":49414670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":168005632,"data_stored":167965430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:29:59.673 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:00.223 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:00.223 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:00.245 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-15T21:29:58.868410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":68338441,"num_objects":16696,"num_object_clones":0,"num_object_copies":33392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16696,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":273763,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11990,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160771,"ondisk_log_size":160771,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1618124,"kb_used_data":331916,"kb_used_omap":12,"kb_used_meta":1286131,"kb_avail":281497396,"statfs":{"total":289910292480,"available":288253333504,"internally_reserved":0,"allocated":339881984,"data_stored":338993224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1316998147},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,3,15,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19800064,"num_objects":-4834,"num_object_clones":0,"num_object_copies":-9668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4834,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004392"},"pg_stats":[{"pgid":"2.f","version":"47'20094","reported_seq":20176,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.005848+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:54.005848+0000","last_peered":"2024-09-15T21:29:54.005848+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:54.005848+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4333568,"num_objects":1058,"num_object_clones":0,"num_object_copies":2116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20094,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3074}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19636","reported_seq":19747,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:54.544216+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:54.544216+0000","last_peered":"2024-09-15T21:29:54.544216+0000","last_clean":"2024-09-15T21:29:54.544216+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:54.544216+0000","last_undegraded":"2024-09-15T21:29:54.544216+0000","last_fullsized":"2024-09-15T21:29:54.544216+0000","mapping_epoch":35,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4252614,"num_objects":1040,"num_object_clones":0,"num_object_copies":2080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1040,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":17917,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20070","reported_seq":27684,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.626740+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:54.626740+0000","last_peered":"2024-09-15T21:29:54.626740+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:54.626740+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4239384,"num_objects":1036,"num_object_clones":0,"num_object_copies":2072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":528,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1036,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18170,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3740,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":508}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19309","reported_seq":19384,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:54.618339+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:54.618339+0000","last_peered":"2024-09-15T21:29:54.618339+0000","last_clean":"2024-09-15T21:29:54.618339+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:54.618339+0000","last_undegraded":"2024-09-15T21:29:54.618339+0000","last_fullsized":"2024-09-15T21:29:54.618339+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4116480,"num_objects":1005,"num_object_clones":0,"num_object_copies":2010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19309,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19486","reported_seq":19625,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:53.893571+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:53.893571+0000","last_peered":"2024-09-15T21:29:53.893571+0000","last_clean":"2024-09-15T21:29:53.893571+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:53.893571+0000","last_undegraded":"2024-09-15T21:29:53.893571+0000","last_fullsized":"2024-09-15T21:29:53.893571+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4174307,"num_objects":1020,"num_object_clones":0,"num_object_copies":2040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1020,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15941,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19550","reported_seq":19692,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.203323+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:54.203323+0000","last_peered":"2024-09-15T21:29:54.203323+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:54.203323+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4444643,"num_objects":1086,"num_object_clones":0,"num_object_copies":2172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1086,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16226,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2990},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19658","reported_seq":19801,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.142011+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:54.142011+0000","last_peered":"2024-09-15T21:29:54.142011+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:54.142011+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4146118,"num_objects":1014,"num_object_clones":0,"num_object_copies":2028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15969,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3147},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19811","reported_seq":19946,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:54.169002+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:54.169002+0000","last_peered":"2024-09-15T21:29:54.169002+0000","last_clean":"2024-09-15T21:29:54.169002+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:54.169002+0000","last_undegraded":"2024-09-15T21:29:54.169002+0000","last_fullsized":"2024-09-15T21:29:54.169002+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4329472,"num_objects":1057,"num_object_clones":0,"num_object_copies":2114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16583,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19519","reported_seq":19652,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.641664+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:54.641664+0000","last_peered":"2024-09-15T21:29:54.641664+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:54.641664+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4071907,"num_objects":995,"num_object_clones":0,"num_object_copies":1990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":995,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17451,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3092}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19392","reported_seq":19529,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.092398+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:54.092398+0000","last_peered":"2024-09-15T21:29:54.092398+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:54.092398+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4464640,"num_objects":1090,"num_object_clones":0,"num_object_copies":2180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16065,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2930}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19622","reported_seq":19747,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.546680+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:54.546680+0000","last_peered":"2024-09-15T21:29:54.546680+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:54.546680+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4182499,"num_objects":1022,"num_object_clones":0,"num_object_copies":2044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1022,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17481,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3068}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19556","reported_seq":28021,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:54.592018+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:54.592018+0000","last_peered":"2024-09-15T21:29:54.592018+0000","last_clean":"2024-09-15T21:29:54.592018+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:54.592018+0000","last_undegraded":"2024-09-15T21:29:54.592018+0000","last_fullsized":"2024-09-15T21:29:54.592018+0000","mapping_epoch":44,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4112384,"num_objects":1004,"num_object_clones":0,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17249,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19608","reported_seq":19748,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.139479+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:54.139479+0000","last_peered":"2024-09-15T21:29:54.139479+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:54.139479+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4415488,"num_objects":1078,"num_object_clones":0,"num_object_copies":2156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1078,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16552,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3016}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19501","reported_seq":19638,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.203220+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:54.203220+0000","last_peered":"2024-09-15T21:29:54.203220+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:54.203220+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4460544,"num_objects":1089,"num_object_clones":0,"num_object_copies":2178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16410,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3050}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19384","reported_seq":27698,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:55.917892+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:55.917892+0000","last_peered":"2024-09-15T21:29:55.917892+0000","last_clean":"2024-09-15T21:29:55.917892+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:55.917892+0000","last_undegraded":"2024-09-15T21:29:55.917892+0000","last_fullsized":"2024-09-15T21:29:55.917892+0000","mapping_epoch":46,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4112384,"num_objects":1004,"num_object_clones":0,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15868,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.526495+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:44.526495+0000","last_peered":"2024-09-15T21:27:44.526495+0000","last_clean":"2024-09-15T21:27:44.526495+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:44.526495+0000","last_undegraded":"2024-09-15T21:27:44.526495+0000","last_fullsized":"2024-09-15T21:27:44.526495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19772","reported_seq":19916,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:54.158217+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:54.158217+0000","last_peered":"2024-09-15T21:29:54.158217+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:54.158217+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4481990,"num_objects":1096,"num_object_clones":0,"num_object_copies":2192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1096,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16476,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3051},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":68338422,"num_objects":16694,"num_object_clones":0,"num_object_copies":33388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16694,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":273761,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11989,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":337514496,"data_stored":337369058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160768,"ondisk_log_size":160768,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":525504,"kb_used_data":164784,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93846336,"statfs":{"total":96636764160,"available":96098648064,"internally_reserved":0,"allocated":168738816,"data_stored":168453558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,0,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578992,"kb_used_data":118364,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93792848,"statfs":{"total":96636764160,"available":96043876352,"internally_reserved":0,"allocated":121204736,"data_stored":120919473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471658155},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513628,"kb_used_data":48768,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93858212,"statfs":{"total":96636764160,"available":96110809088,"internally_reserved":0,"allocated":49938432,"data_stored":49620193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":120025088,"data_stored":119988958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49483776,"data_stored":49414670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":168005632,"data_stored":167965430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:00.246 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:30:00.246 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20094", 'reported_seq': 20176, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:54.005848+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:29:54.005848+0000', 'last_peered': '2024-09-15T21:29:54.005848+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:29:54.005848+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4333568, 'num_objects': 1058, 'num_object_clones': 0, 'num_object_copies': 2116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1058, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20094, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3074}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:00.246 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:30:00.246 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20070", 'reported_seq': 27684, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:54.626740+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:29:54.626740+0000', 'last_peered': '2024-09-15T21:29:54.626740+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:29:54.626740+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4239384, 'num_objects': 1036, 'num_object_clones': 0, 'num_object_copies': 2072, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 528, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1036, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18170, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3740, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 508}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:00.247 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:30:00.247 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19550", 'reported_seq': 19692, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:54.203323+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:29:54.203323+0000', 'last_peered': '2024-09-15T21:29:54.203323+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:29:54.203323+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4444643, 'num_objects': 1086, 'num_object_clones': 0, 'num_object_copies': 2172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1086, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16226, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2990}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:00.247 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:30:00.247 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19658", 'reported_seq': 19801, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:54.142011+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:29:54.142011+0000', 'last_peered': '2024-09-15T21:29:54.142011+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:29:54.142011+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4146118, 'num_objects': 1014, 'num_object_clones': 0, 'num_object_copies': 2028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1014, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15969, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3147}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:00.247 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:30:00.247 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'19519", 'reported_seq': 19652, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:54.641664+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:29:54.641664+0000', 'last_peered': '2024-09-15T21:29:54.641664+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:29:54.641664+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4071907, 'num_objects': 995, 'num_object_clones': 0, 'num_object_copies': 1990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 995, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17451, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3092}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:00.247 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:00.247 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19392", 'reported_seq': 19529, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:54.092398+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:29:54.092398+0000', 'last_peered': '2024-09-15T21:29:54.092398+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:29:54.092398+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4464640, 'num_objects': 1090, 'num_object_clones': 0, 'num_object_copies': 2180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1090, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16065, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2930}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:00.247 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:30:00.247 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19622", 'reported_seq': 19747, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:54.546680+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:29:54.546680+0000', 'last_peered': '2024-09-15T21:29:54.546680+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:29:54.546680+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4182499, 'num_objects': 1022, 'num_object_clones': 0, 'num_object_copies': 2044, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1022, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17481, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3068}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:00.248 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:30:00.248 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19608", 'reported_seq': 19748, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:54.139479+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:29:54.139479+0000', 'last_peered': '2024-09-15T21:29:54.139479+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:29:54.139479+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4415488, 'num_objects': 1078, 'num_object_clones': 0, 'num_object_copies': 2156, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1078, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16552, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3016}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:00.248 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:30:00.248 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'19501", 'reported_seq': 19638, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:54.203220+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:29:54.203220+0000', 'last_peered': '2024-09-15T21:29:54.203220+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:29:54.203220+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4460544, 'num_objects': 1089, 'num_object_clones': 0, 'num_object_copies': 2178, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1089, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16410, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3050}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:00.248 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:30:00.248 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'19772", 'reported_seq': 19916, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:29:54.158217+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:29:54.158217+0000', 'last_peered': '2024-09-15T21:29:54.158217+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:29:54.158217+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4481990, 'num_objects': 1096, 'num_object_clones': 0, 'num_object_copies': 2192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1096, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16476, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3051}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:00.248 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:00.969 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:00.969 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37485/33392 objects degraded (112.257%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":652,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":16696,"data_bytes":68338441,"bytes_used":1656958976,"bytes_avail":288253333504,"bytes_total":289910292480,"degraded_objects":37485,"degraded_total":33392,"degraded_ratio":1.1225742692860565,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":402},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:00.991 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:00.992 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:01.548 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:01.551 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:01.574 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-15T21:30:00.869033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":59196169,"num_objects":14464,"num_object_clones":0,"num_object_copies":28928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37333,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14464,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":275995,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12142,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160803,"ondisk_log_size":160803,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1614932,"kb_used_data":320392,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281500588,"statfs":{"total":289910292480,"available":288256602112,"internally_reserved":0,"allocated":328081408,"data_stored":327192648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1325517827},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,9,15,9,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28942336,"num_objects":-7066,"num_object_clones":0,"num_object_copies":-14132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7066,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004346"},"pg_stats":[{"pgid":"2.f","version":"47'20242","reported_seq":20324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:59.098777+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:29:59.098777+0000","last_peered":"2024-09-15T21:29:59.098777+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:29:59.098777+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3727360,"num_objects":910,"num_object_clones":0,"num_object_copies":1820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20242,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3222}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19798","reported_seq":19910,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:59.622791+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:29:59.622791+0000","last_peered":"2024-09-15T21:29:59.622791+0000","last_clean":"2024-09-15T21:29:59.622791+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:29:59.622791+0000","last_undegraded":"2024-09-15T21:29:59.622791+0000","last_fullsized":"2024-09-15T21:29:59.622791+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3589062,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18079,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20220","reported_seq":28138,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:59.622854+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:29:59.622854+0000","last_peered":"2024-09-15T21:29:59.622854+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:29:59.622854+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3624984,"num_objects":886,"num_object_clones":0,"num_object_copies":1772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":376,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":886,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18320,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3892,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":510}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19447","reported_seq":19524,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:59.638556+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:29:59.638556+0000","last_peered":"2024-09-15T21:29:59.638556+0000","last_clean":"2024-09-15T21:29:59.638556+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:29:59.638556+0000","last_undegraded":"2024-09-15T21:29:59.638556+0000","last_fullsized":"2024-09-15T21:29:59.638556+0000","mapping_epoch":35,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3551232,"num_objects":867,"num_object_clones":0,"num_object_copies":1734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19447,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19628","reported_seq":19767,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:59.149283+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:29:59.149283+0000","last_peered":"2024-09-15T21:29:59.149283+0000","last_clean":"2024-09-15T21:29:59.149283+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:29:59.149283+0000","last_undegraded":"2024-09-15T21:29:59.149283+0000","last_fullsized":"2024-09-15T21:29:59.149283+0000","mapping_epoch":34,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3592675,"num_objects":878,"num_object_clones":0,"num_object_copies":1756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16083,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19708","reported_seq":19851,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:59.190063+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:29:59.190063+0000","last_peered":"2024-09-15T21:29:59.190063+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:29:59.190063+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3797475,"num_objects":928,"num_object_clones":0,"num_object_copies":1856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":928,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16384,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3148},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19798","reported_seq":19941,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:59.169711+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:29:59.169711+0000","last_peered":"2024-09-15T21:29:59.169711+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:29:59.169711+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3572678,"num_objects":874,"num_object_clones":0,"num_object_copies":1748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":874,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16109,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3287},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'19951","reported_seq":20086,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:59.154599+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:29:59.154599+0000","last_peered":"2024-09-15T21:29:59.154599+0000","last_clean":"2024-09-15T21:29:59.154599+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:29:59.154599+0000","last_undegraded":"2024-09-15T21:29:59.154599+0000","last_fullsized":"2024-09-15T21:29:59.154599+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3756032,"num_objects":917,"num_object_clones":0,"num_object_copies":1834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16723,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19670","reported_seq":19803,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:59.625767+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:29:59.625767+0000","last_peered":"2024-09-15T21:29:59.625767+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:29:59.625767+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3453411,"num_objects":844,"num_object_clones":0,"num_object_copies":1688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":844,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17602,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3243}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19525","reported_seq":19662,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:59.203851+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:29:59.203851+0000","last_peered":"2024-09-15T21:29:59.203851+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:29:59.203851+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3919872,"num_objects":957,"num_object_clones":0,"num_object_copies":1914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16198,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3063}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19760","reported_seq":19885,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:59.592313+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:29:59.592313+0000","last_peered":"2024-09-15T21:29:59.592313+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:29:59.592313+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3617251,"num_objects":884,"num_object_clones":0,"num_object_copies":1768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":884,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17619,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3206}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19703","reported_seq":28168,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:59.598296+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:29:59.598296+0000","last_peered":"2024-09-15T21:29:59.598296+0000","last_clean":"2024-09-15T21:29:59.598296+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:29:59.598296+0000","last_undegraded":"2024-09-15T21:29:59.598296+0000","last_fullsized":"2024-09-15T21:29:59.598296+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3510272,"num_objects":857,"num_object_clones":0,"num_object_copies":1714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17396,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19757","reported_seq":19897,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:59.189857+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:29:59.189857+0000","last_peered":"2024-09-15T21:29:59.189857+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:29:59.189857+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3805184,"num_objects":929,"num_object_clones":0,"num_object_copies":1858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16701,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3165}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19680","reported_seq":19817,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:59.202205+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:29:59.202205+0000","last_peered":"2024-09-15T21:29:59.202205+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:29:59.202205+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3727360,"num_objects":910,"num_object_clones":0,"num_object_copies":1820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16589,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3229}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19384","reported_seq":27698,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:29:55.917892+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:29:55.917892+0000","last_peered":"2024-09-15T21:29:55.917892+0000","last_clean":"2024-09-15T21:29:55.917892+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:29:55.917892+0000","last_undegraded":"2024-09-15T21:29:55.917892+0000","last_fullsized":"2024-09-15T21:29:55.917892+0000","mapping_epoch":46,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4112384,"num_objects":1004,"num_object_clones":0,"num_object_copies":2008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15868,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.526495+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:44.526495+0000","last_peered":"2024-09-15T21:27:44.526495+0000","last_clean":"2024-09-15T21:27:44.526495+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:44.526495+0000","last_undegraded":"2024-09-15T21:27:44.526495+0000","last_fullsized":"2024-09-15T21:27:44.526495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'19929","reported_seq":20073,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:29:59.155862+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:29:59.155862+0000","last_peered":"2024-09-15T21:29:59.155862+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:29:59.155862+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3838918,"num_objects":939,"num_object_clones":0,"num_object_copies":1878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":939,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16633,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3208},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":59196150,"num_objects":14462,"num_object_clones":0,"num_object_copies":28924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37333,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14462,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":275993,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12141,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":325296128,"data_stored":325150690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160800,"ondisk_log_size":160800,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":524992,"kb_used_data":160108,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93846848,"statfs":{"total":96636764160,"available":96099172352,"internally_reserved":0,"allocated":163950592,"data_stored":163665334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576312,"kb_used_data":111516,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93795528,"statfs":{"total":96636764160,"available":96046620672,"internally_reserved":0,"allocated":114192384,"data_stored":113907121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475917995},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,6,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513628,"kb_used_data":48768,"kb_used_omap":4,"kb_used_meta":464827,"kb_avail":93858212,"statfs":{"total":96636764160,"available":96110809088,"internally_reserved":0,"allocated":49938432,"data_stored":49620193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475983535},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,8,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":112648192,"data_stored":112612062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49483776,"data_stored":49414670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":163164160,"data_stored":163123958,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:04.576 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:05.136 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:05.136 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:05.161 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-15T21:30:04.870087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":49398537,"num_objects":12072,"num_object_clones":0,"num_object_copies":24144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12072,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":278387,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12291,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160995,"ondisk_log_size":160995,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1595824,"kb_used_data":301300,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281519696,"statfs":{"total":289910292480,"available":288276168704,"internally_reserved":0,"allocated":308531200,"data_stored":307642440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1325517827},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,5,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28811264,"num_objects":-7034,"num_object_clones":0,"num_object_copies":-14068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7034,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":448,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003883"},"pg_stats":[{"pgid":"2.f","version":"47'20395","reported_seq":20477,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.184732+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:04.184732+0000","last_peered":"2024-09-15T21:30:04.184732+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:04.184732+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3100672,"num_objects":757,"num_object_clones":0,"num_object_copies":1514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20395,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3375}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19959","reported_seq":20071,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.581668+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:04.581668+0000","last_peered":"2024-09-15T21:30:04.581668+0000","last_clean":"2024-09-15T21:30:04.581668+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:04.581668+0000","last_undegraded":"2024-09-15T21:30:04.581668+0000","last_fullsized":"2024-09-15T21:30:04.581668+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2929606,"num_objects":717,"num_object_clones":0,"num_object_copies":1434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":717,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18240,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20382","reported_seq":28598,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.651592+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:30:04.651592+0000","last_peered":"2024-09-15T21:30:04.651592+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:30:04.651592+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2961432,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18482,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":497}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19585","reported_seq":19662,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.486539+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:04.486539+0000","last_peered":"2024-09-15T21:30:04.486539+0000","last_clean":"2024-09-15T21:30:04.486539+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:04.486539+0000","last_undegraded":"2024-09-15T21:30:04.486539+0000","last_fullsized":"2024-09-15T21:30:04.486539+0000","mapping_epoch":35,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2985984,"num_objects":729,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19585,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19768","reported_seq":19906,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.202435+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:04.202435+0000","last_peered":"2024-09-15T21:30:04.202435+0000","last_clean":"2024-09-15T21:30:04.202435+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:04.202435+0000","last_undegraded":"2024-09-15T21:30:04.202435+0000","last_fullsized":"2024-09-15T21:30:04.202435+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3019235,"num_objects":738,"num_object_clones":0,"num_object_copies":1476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16223,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19873","reported_seq":20016,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.225433+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:04.225433+0000","last_peered":"2024-09-15T21:30:04.225433+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:04.225433+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3121635,"num_objects":763,"num_object_clones":0,"num_object_copies":1526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16549,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3313},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19946","reported_seq":20089,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.049639+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:04.049639+0000","last_peered":"2024-09-15T21:30:04.049639+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:04.049639+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2966470,"num_objects":726,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16257,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3435},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20087","reported_seq":20222,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.225458+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:04.225458+0000","last_peered":"2024-09-15T21:30:04.225458+0000","last_clean":"2024-09-15T21:30:04.225458+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:04.225458+0000","last_undegraded":"2024-09-15T21:30:04.225458+0000","last_fullsized":"2024-09-15T21:30:04.225458+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3198976,"num_objects":781,"num_object_clones":0,"num_object_copies":1562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16859,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19799","reported_seq":19931,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.562491+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:04.562491+0000","last_peered":"2024-09-15T21:30:04.562491+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:04.562491+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2925027,"num_objects":715,"num_object_clones":0,"num_object_copies":1430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17731,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3372}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19677","reported_seq":19814,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.245361+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:04.245361+0000","last_peered":"2024-09-15T21:30:04.245361+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:04.245361+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3297280,"num_objects":805,"num_object_clones":0,"num_object_copies":1610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16350,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3215}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19917","reported_seq":20042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.642382+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:04.642382+0000","last_peered":"2024-09-15T21:30:04.642382+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:04.642382+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2974179,"num_objects":727,"num_object_clones":0,"num_object_copies":1454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":727,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17776,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19846","reported_seq":28311,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.587078+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:04.587078+0000","last_peered":"2024-09-15T21:30:04.587078+0000","last_clean":"2024-09-15T21:30:04.587078+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:04.587078+0000","last_undegraded":"2024-09-15T21:30:04.587078+0000","last_fullsized":"2024-09-15T21:30:04.587078+0000","mapping_epoch":44,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924544,"num_objects":714,"num_object_clones":0,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17539,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19920","reported_seq":20060,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.239485+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:04.239485+0000","last_peered":"2024-09-15T21:30:04.239485+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:04.239485+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3137536,"num_objects":766,"num_object_clones":0,"num_object_copies":1532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16864,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3328}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19828","reported_seq":19967,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.266902+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:30:04.266902+0000","last_peered":"2024-09-15T21:30:04.266902+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:04.266902+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3121152,"num_objects":762,"num_object_clones":0,"num_object_copies":1524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16737,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3377}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19531","reported_seq":27846,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:01.028770+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:01.028770+0000","last_peered":"2024-09-15T21:30:01.028770+0000","last_clean":"2024-09-15T21:30:01.028770+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:01.028770+0000","last_undegraded":"2024-09-15T21:30:01.028770+0000","last_fullsized":"2024-09-15T21:30:01.028770+0000","mapping_epoch":46,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3510272,"num_objects":857,"num_object_clones":0,"num_object_copies":1714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16015,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.526495+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:44.526495+0000","last_peered":"2024-09-15T21:27:44.526495+0000","last_clean":"2024-09-15T21:27:44.526495+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:44.526495+0000","last_undegraded":"2024-09-15T21:27:44.526495+0000","last_fullsized":"2024-09-15T21:27:44.526495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20079","reported_seq":20221,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.163631+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:04.163631+0000","last_peered":"2024-09-15T21:30:04.163631+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:04.163631+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3224518,"num_objects":789,"num_object_clones":0,"num_object_copies":1578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":789,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16783,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3358},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":49398518,"num_objects":12070,"num_object_clones":0,"num_object_copies":24140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12070,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":278385,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12290,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":305664000,"data_stored":305518562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160992,"ondisk_log_size":160992,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507856,"kb_used_data":155456,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93863984,"statfs":{"total":96636764160,"available":96116719616,"internally_reserved":0,"allocated":159186944,"data_stored":158901686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577520,"kb_used_data":104408,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93794320,"statfs":{"total":96636764160,"available":96045383680,"internally_reserved":0,"allocated":106913792,"data_stored":106628529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484437675},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510448,"kb_used_data":41436,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93861392,"statfs":{"total":96636764160,"available":96114065408,"internally_reserved":0,"allocated":42430464,"data_stored":42112225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":105312256,"data_stored":105276126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":42000384,"data_stored":41931278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":158351360,"data_stored":158311158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:05.162 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:05.728 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:05.728 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:05.751 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-15T21:30:04.870087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":49398537,"num_objects":12072,"num_object_clones":0,"num_object_copies":24144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12072,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":278387,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12291,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160995,"ondisk_log_size":160995,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1595824,"kb_used_data":301300,"kb_used_omap":12,"kb_used_meta":1294451,"kb_avail":281519696,"statfs":{"total":289910292480,"available":288276168704,"internally_reserved":0,"allocated":308531200,"data_stored":307642440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1325517827},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,4,5,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28811264,"num_objects":-7034,"num_object_clones":0,"num_object_copies":-14068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7034,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":448,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003883"},"pg_stats":[{"pgid":"2.f","version":"47'20395","reported_seq":20477,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.184732+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:04.184732+0000","last_peered":"2024-09-15T21:30:04.184732+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:04.184732+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3100672,"num_objects":757,"num_object_clones":0,"num_object_copies":1514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20395,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3375}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19959","reported_seq":20071,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.581668+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:04.581668+0000","last_peered":"2024-09-15T21:30:04.581668+0000","last_clean":"2024-09-15T21:30:04.581668+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:04.581668+0000","last_undegraded":"2024-09-15T21:30:04.581668+0000","last_fullsized":"2024-09-15T21:30:04.581668+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2929606,"num_objects":717,"num_object_clones":0,"num_object_copies":1434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":717,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18240,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20382","reported_seq":28598,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.651592+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:30:04.651592+0000","last_peered":"2024-09-15T21:30:04.651592+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:30:04.651592+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2961432,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18482,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":497}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19585","reported_seq":19662,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.486539+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:04.486539+0000","last_peered":"2024-09-15T21:30:04.486539+0000","last_clean":"2024-09-15T21:30:04.486539+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:04.486539+0000","last_undegraded":"2024-09-15T21:30:04.486539+0000","last_fullsized":"2024-09-15T21:30:04.486539+0000","mapping_epoch":35,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2985984,"num_objects":729,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19585,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19768","reported_seq":19906,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.202435+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:04.202435+0000","last_peered":"2024-09-15T21:30:04.202435+0000","last_clean":"2024-09-15T21:30:04.202435+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:04.202435+0000","last_undegraded":"2024-09-15T21:30:04.202435+0000","last_fullsized":"2024-09-15T21:30:04.202435+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3019235,"num_objects":738,"num_object_clones":0,"num_object_copies":1476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16223,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19873","reported_seq":20016,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.225433+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:04.225433+0000","last_peered":"2024-09-15T21:30:04.225433+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:04.225433+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3121635,"num_objects":763,"num_object_clones":0,"num_object_copies":1526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16549,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3313},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19946","reported_seq":20089,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.049639+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:04.049639+0000","last_peered":"2024-09-15T21:30:04.049639+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:04.049639+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2966470,"num_objects":726,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16257,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3435},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20087","reported_seq":20222,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.225458+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:04.225458+0000","last_peered":"2024-09-15T21:30:04.225458+0000","last_clean":"2024-09-15T21:30:04.225458+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:04.225458+0000","last_undegraded":"2024-09-15T21:30:04.225458+0000","last_fullsized":"2024-09-15T21:30:04.225458+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3198976,"num_objects":781,"num_object_clones":0,"num_object_copies":1562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16859,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19799","reported_seq":19931,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.562491+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:04.562491+0000","last_peered":"2024-09-15T21:30:04.562491+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:04.562491+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2925027,"num_objects":715,"num_object_clones":0,"num_object_copies":1430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17731,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3372}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19677","reported_seq":19814,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.245361+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:04.245361+0000","last_peered":"2024-09-15T21:30:04.245361+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:04.245361+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3297280,"num_objects":805,"num_object_clones":0,"num_object_copies":1610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16350,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3215}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19917","reported_seq":20042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.642382+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:04.642382+0000","last_peered":"2024-09-15T21:30:04.642382+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:04.642382+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2974179,"num_objects":727,"num_object_clones":0,"num_object_copies":1454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":727,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17776,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19846","reported_seq":28311,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.587078+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:04.587078+0000","last_peered":"2024-09-15T21:30:04.587078+0000","last_clean":"2024-09-15T21:30:04.587078+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:04.587078+0000","last_undegraded":"2024-09-15T21:30:04.587078+0000","last_fullsized":"2024-09-15T21:30:04.587078+0000","mapping_epoch":44,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924544,"num_objects":714,"num_object_clones":0,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17539,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19920","reported_seq":20060,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.239485+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:04.239485+0000","last_peered":"2024-09-15T21:30:04.239485+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:04.239485+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3137536,"num_objects":766,"num_object_clones":0,"num_object_copies":1532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16864,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3328}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19828","reported_seq":19967,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.266902+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:30:04.266902+0000","last_peered":"2024-09-15T21:30:04.266902+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:04.266902+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3121152,"num_objects":762,"num_object_clones":0,"num_object_copies":1524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16737,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3377}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19531","reported_seq":27846,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:01.028770+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:01.028770+0000","last_peered":"2024-09-15T21:30:01.028770+0000","last_clean":"2024-09-15T21:30:01.028770+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:01.028770+0000","last_undegraded":"2024-09-15T21:30:01.028770+0000","last_fullsized":"2024-09-15T21:30:01.028770+0000","mapping_epoch":46,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3510272,"num_objects":857,"num_object_clones":0,"num_object_copies":1714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16015,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.526495+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:44.526495+0000","last_peered":"2024-09-15T21:27:44.526495+0000","last_clean":"2024-09-15T21:27:44.526495+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:44.526495+0000","last_undegraded":"2024-09-15T21:27:44.526495+0000","last_fullsized":"2024-09-15T21:27:44.526495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20079","reported_seq":20221,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.163631+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:04.163631+0000","last_peered":"2024-09-15T21:30:04.163631+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:04.163631+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3224518,"num_objects":789,"num_object_clones":0,"num_object_copies":1578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":789,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16783,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3358},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":49398518,"num_objects":12070,"num_object_clones":0,"num_object_copies":24140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12070,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":278385,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12290,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":305664000,"data_stored":305518562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160992,"ondisk_log_size":160992,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507856,"kb_used_data":155456,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93863984,"statfs":{"total":96636764160,"available":96116719616,"internally_reserved":0,"allocated":159186944,"data_stored":158901686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577520,"kb_used_data":104408,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93794320,"statfs":{"total":96636764160,"available":96045383680,"internally_reserved":0,"allocated":106913792,"data_stored":106628529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484437675},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":510448,"kb_used_data":41436,"kb_used_omap":4,"kb_used_meta":468987,"kb_avail":93861392,"statfs":{"total":96636764160,"available":96114065408,"internally_reserved":0,"allocated":42430464,"data_stored":42112225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480243375},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":105312256,"data_stored":105276126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":42000384,"data_stored":41931278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":158351360,"data_stored":158311158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:05.752 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:30:05.752 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20395", 'reported_seq': 20477, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:04.184732+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:04.184732+0000', 'last_peered': '2024-09-15T21:30:04.184732+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:04.184732+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3100672, 'num_objects': 757, 'num_object_clones': 0, 'num_object_copies': 1514, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 757, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20395, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3375}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:05.753 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:30:05.753 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20382", 'reported_seq': 28598, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:04.651592+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:30:04.651592+0000', 'last_peered': '2024-09-15T21:30:04.651592+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:30:04.651592+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2961432, 'num_objects': 724, 'num_object_clones': 0, 'num_object_copies': 1448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 227, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 724, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18482, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4041, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 497}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:05.753 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:30:05.753 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'19873", 'reported_seq': 20016, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:04.225433+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:04.225433+0000', 'last_peered': '2024-09-15T21:30:04.225433+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:04.225433+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3121635, 'num_objects': 763, 'num_object_clones': 0, 'num_object_copies': 1526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 763, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16549, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3313}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:05.753 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:30:05.753 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'19946", 'reported_seq': 20089, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:04.049639+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:04.049639+0000', 'last_peered': '2024-09-15T21:30:04.049639+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:04.049639+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2966470, 'num_objects': 726, 'num_object_clones': 0, 'num_object_copies': 1452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16257, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3435}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:05.753 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:30:05.753 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'19799", 'reported_seq': 19931, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:04.562491+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:30:04.562491+0000', 'last_peered': '2024-09-15T21:30:04.562491+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:30:04.562491+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2925027, 'num_objects': 715, 'num_object_clones': 0, 'num_object_copies': 1430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 715, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17731, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3372}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:05.753 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:05.753 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19677", 'reported_seq': 19814, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:04.245361+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:04.245361+0000', 'last_peered': '2024-09-15T21:30:04.245361+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:04.245361+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3297280, 'num_objects': 805, 'num_object_clones': 0, 'num_object_copies': 1610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 805, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16350, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3215}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:05.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:30:05.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'19917", 'reported_seq': 20042, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:04.642382+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:04.642382+0000', 'last_peered': '2024-09-15T21:30:04.642382+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:04.642382+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2974179, 'num_objects': 727, 'num_object_clones': 0, 'num_object_copies': 1454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 727, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17776, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3363}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:05.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:30:05.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'19920", 'reported_seq': 20060, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:04.239485+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:04.239485+0000', 'last_peered': '2024-09-15T21:30:04.239485+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:04.239485+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3137536, 'num_objects': 766, 'num_object_clones': 0, 'num_object_copies': 1532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 766, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16864, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3328}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:05.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:30:05.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'19828", 'reported_seq': 19967, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:04.266902+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:30:04.266902+0000', 'last_peered': '2024-09-15T21:30:04.266902+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:30:04.266902+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3121152, 'num_objects': 762, 'num_object_clones': 0, 'num_object_copies': 1524, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 762, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16737, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3377}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:05.754 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:30:05.754 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'20079", 'reported_seq': 20221, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:04.163631+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:04.163631+0000', 'last_peered': '2024-09-15T21:30:04.163631+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:04.163631+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3224518, 'num_objects': 789, 'num_object_clones': 0, 'num_object_copies': 1578, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 789, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16783, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3358}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:05.754 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:06.444 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37184/24144 objects degraded (154.009%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":658,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12072,"data_bytes":49398537,"bytes_used":1634123776,"bytes_avail":288276168704,"bytes_total":289910292480,"degraded_objects":37184,"degraded_total":24144,"degraded_ratio":1.5400927766732935,"recovering_objects_per_sec":37,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":448,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":585},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:06.468 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:06.468 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:07.039 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:07.061 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-15T21:30:06.870735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48812809,"num_objects":11929,"num_object_clones":0,"num_object_copies":23858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11929,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":278530,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12291,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161038,"ondisk_log_size":161038,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1593984,"kb_used_data":295304,"kb_used_omap":12,"kb_used_meta":1298611,"kb_avail":281521536,"statfs":{"total":289910292480,"available":288278052864,"internally_reserved":0,"allocated":302391296,"data_stored":301502536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1329777667},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,5,10,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20164608,"num_objects":-4923,"num_object_clones":0,"num_object_copies":-9846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4923,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003712"},"pg_stats":[{"pgid":"2.f","version":"47'20395","reported_seq":20477,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.184732+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:04.184732+0000","last_peered":"2024-09-15T21:30:04.184732+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:04.184732+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3100672,"num_objects":757,"num_object_clones":0,"num_object_copies":1514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20395,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3375}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19959","reported_seq":20071,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.581668+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:04.581668+0000","last_peered":"2024-09-15T21:30:04.581668+0000","last_clean":"2024-09-15T21:30:04.581668+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:04.581668+0000","last_undegraded":"2024-09-15T21:30:04.581668+0000","last_fullsized":"2024-09-15T21:30:04.581668+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2929606,"num_objects":717,"num_object_clones":0,"num_object_copies":1434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":717,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18240,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20382","reported_seq":28598,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.651592+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:30:04.651592+0000","last_peered":"2024-09-15T21:30:04.651592+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:30:04.651592+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2961432,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18482,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":497}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19585","reported_seq":19662,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.486539+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:04.486539+0000","last_peered":"2024-09-15T21:30:04.486539+0000","last_clean":"2024-09-15T21:30:04.486539+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:04.486539+0000","last_undegraded":"2024-09-15T21:30:04.486539+0000","last_fullsized":"2024-09-15T21:30:04.486539+0000","mapping_epoch":35,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2985984,"num_objects":729,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19585,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19768","reported_seq":19906,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.202435+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:04.202435+0000","last_peered":"2024-09-15T21:30:04.202435+0000","last_clean":"2024-09-15T21:30:04.202435+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:04.202435+0000","last_undegraded":"2024-09-15T21:30:04.202435+0000","last_fullsized":"2024-09-15T21:30:04.202435+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3019235,"num_objects":738,"num_object_clones":0,"num_object_copies":1476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16223,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19873","reported_seq":20016,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.225433+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:04.225433+0000","last_peered":"2024-09-15T21:30:04.225433+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:04.225433+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3121635,"num_objects":763,"num_object_clones":0,"num_object_copies":1526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16549,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3313},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19946","reported_seq":20089,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.049639+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:04.049639+0000","last_peered":"2024-09-15T21:30:04.049639+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:04.049639+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2966470,"num_objects":726,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16257,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3435},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20087","reported_seq":20222,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.225458+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:04.225458+0000","last_peered":"2024-09-15T21:30:04.225458+0000","last_clean":"2024-09-15T21:30:04.225458+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:04.225458+0000","last_undegraded":"2024-09-15T21:30:04.225458+0000","last_fullsized":"2024-09-15T21:30:04.225458+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3198976,"num_objects":781,"num_object_clones":0,"num_object_copies":1562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16859,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19799","reported_seq":19931,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.562491+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:04.562491+0000","last_peered":"2024-09-15T21:30:04.562491+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:04.562491+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2925027,"num_objects":715,"num_object_clones":0,"num_object_copies":1430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17731,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3372}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19677","reported_seq":19814,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.245361+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:04.245361+0000","last_peered":"2024-09-15T21:30:04.245361+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:04.245361+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3297280,"num_objects":805,"num_object_clones":0,"num_object_copies":1610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16350,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3215}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19917","reported_seq":20042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.642382+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:04.642382+0000","last_peered":"2024-09-15T21:30:04.642382+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:04.642382+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2974179,"num_objects":727,"num_object_clones":0,"num_object_copies":1454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":727,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17776,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19846","reported_seq":28311,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.587078+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:04.587078+0000","last_peered":"2024-09-15T21:30:04.587078+0000","last_clean":"2024-09-15T21:30:04.587078+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:04.587078+0000","last_undegraded":"2024-09-15T21:30:04.587078+0000","last_fullsized":"2024-09-15T21:30:04.587078+0000","mapping_epoch":44,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924544,"num_objects":714,"num_object_clones":0,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17539,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19920","reported_seq":20060,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.239485+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:04.239485+0000","last_peered":"2024-09-15T21:30:04.239485+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:04.239485+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3137536,"num_objects":766,"num_object_clones":0,"num_object_copies":1532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16864,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3328}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19828","reported_seq":19967,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.266902+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:30:04.266902+0000","last_peered":"2024-09-15T21:30:04.266902+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:04.266902+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3121152,"num_objects":762,"num_object_clones":0,"num_object_copies":1524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16737,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3377}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19674","reported_seq":27990,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:06.080293+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:06.080293+0000","last_peered":"2024-09-15T21:30:06.080293+0000","last_clean":"2024-09-15T21:30:06.080293+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:06.080293+0000","last_undegraded":"2024-09-15T21:30:06.080293+0000","last_fullsized":"2024-09-15T21:30:06.080293+0000","mapping_epoch":46,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924544,"num_objects":714,"num_object_clones":0,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16158,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.526495+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:44.526495+0000","last_peered":"2024-09-15T21:27:44.526495+0000","last_clean":"2024-09-15T21:27:44.526495+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:44.526495+0000","last_undegraded":"2024-09-15T21:27:44.526495+0000","last_fullsized":"2024-09-15T21:27:44.526495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20079","reported_seq":20221,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.163631+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:04.163631+0000","last_peered":"2024-09-15T21:30:04.163631+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:04.163631+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3224518,"num_objects":789,"num_object_clones":0,"num_object_copies":1578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":789,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16783,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3358},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":48812790,"num_objects":11927,"num_object_clones":0,"num_object_copies":23854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11927,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":278528,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12290,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":298274816,"data_stored":298129378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161035,"ondisk_log_size":161035,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507856,"kb_used_data":155456,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93863984,"statfs":{"total":96636764160,"available":96116719616,"internally_reserved":0,"allocated":159186944,"data_stored":158901686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577520,"kb_used_data":104408,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93794320,"statfs":{"total":96636764160,"available":96045383680,"internally_reserved":0,"allocated":106913792,"data_stored":106628529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484437675},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508608,"kb_used_data":35440,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93863232,"statfs":{"total":96636764160,"available":96115949568,"internally_reserved":0,"allocated":36290560,"data_stored":35972321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,6,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":105312256,"data_stored":105276126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":34611200,"data_stored":34542094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":158351360,"data_stored":158311158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:10.063 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:10.605 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:10.608 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:10.626 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-09-15T21:30:08.871513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48812809,"num_objects":11929,"num_object_clones":0,"num_object_copies":23858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11929,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":278530,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12291,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161038,"ondisk_log_size":161038,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1593984,"kb_used_data":295304,"kb_used_omap":12,"kb_used_meta":1298611,"kb_avail":281521536,"statfs":{"total":289910292480,"available":288278052864,"internally_reserved":0,"allocated":302391296,"data_stored":301502536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1329777667},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,5,5,10,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19525632,"num_objects":-4767,"num_object_clones":0,"num_object_copies":-9534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-301,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4767,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003830"},"pg_stats":[{"pgid":"2.f","version":"47'20395","reported_seq":20477,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.184732+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:04.184732+0000","last_peered":"2024-09-15T21:30:04.184732+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:04.184732+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3100672,"num_objects":757,"num_object_clones":0,"num_object_copies":1514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20395,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3375}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'19959","reported_seq":20071,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.581668+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:04.581668+0000","last_peered":"2024-09-15T21:30:04.581668+0000","last_clean":"2024-09-15T21:30:04.581668+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:04.581668+0000","last_undegraded":"2024-09-15T21:30:04.581668+0000","last_fullsized":"2024-09-15T21:30:04.581668+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2929606,"num_objects":717,"num_object_clones":0,"num_object_copies":1434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":717,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18240,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20382","reported_seq":28598,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.651592+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:30:04.651592+0000","last_peered":"2024-09-15T21:30:04.651592+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:30:04.651592+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2961432,"num_objects":724,"num_object_clones":0,"num_object_copies":1448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":227,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":724,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18482,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":497}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19585","reported_seq":19662,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.486539+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:04.486539+0000","last_peered":"2024-09-15T21:30:04.486539+0000","last_clean":"2024-09-15T21:30:04.486539+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:04.486539+0000","last_undegraded":"2024-09-15T21:30:04.486539+0000","last_fullsized":"2024-09-15T21:30:04.486539+0000","mapping_epoch":35,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2985984,"num_objects":729,"num_object_clones":0,"num_object_copies":1458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19585,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19768","reported_seq":19906,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.202435+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:04.202435+0000","last_peered":"2024-09-15T21:30:04.202435+0000","last_clean":"2024-09-15T21:30:04.202435+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:04.202435+0000","last_undegraded":"2024-09-15T21:30:04.202435+0000","last_fullsized":"2024-09-15T21:30:04.202435+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3019235,"num_objects":738,"num_object_clones":0,"num_object_copies":1476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16223,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'19873","reported_seq":20016,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.225433+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:04.225433+0000","last_peered":"2024-09-15T21:30:04.225433+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:04.225433+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3121635,"num_objects":763,"num_object_clones":0,"num_object_copies":1526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16549,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3313},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'19946","reported_seq":20089,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.049639+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:04.049639+0000","last_peered":"2024-09-15T21:30:04.049639+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:04.049639+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2966470,"num_objects":726,"num_object_clones":0,"num_object_copies":1452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16257,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3435},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20087","reported_seq":20222,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.225458+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:04.225458+0000","last_peered":"2024-09-15T21:30:04.225458+0000","last_clean":"2024-09-15T21:30:04.225458+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:04.225458+0000","last_undegraded":"2024-09-15T21:30:04.225458+0000","last_fullsized":"2024-09-15T21:30:04.225458+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3198976,"num_objects":781,"num_object_clones":0,"num_object_copies":1562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16859,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19799","reported_seq":19931,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.562491+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:04.562491+0000","last_peered":"2024-09-15T21:30:04.562491+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:04.562491+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2925027,"num_objects":715,"num_object_clones":0,"num_object_copies":1430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":715,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17731,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3372}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19677","reported_seq":19814,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.245361+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:04.245361+0000","last_peered":"2024-09-15T21:30:04.245361+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:04.245361+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3297280,"num_objects":805,"num_object_clones":0,"num_object_copies":1610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16350,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3215}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'19917","reported_seq":20042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.642382+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:04.642382+0000","last_peered":"2024-09-15T21:30:04.642382+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:04.642382+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2974179,"num_objects":727,"num_object_clones":0,"num_object_copies":1454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":727,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17776,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'19846","reported_seq":28311,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:04.587078+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:04.587078+0000","last_peered":"2024-09-15T21:30:04.587078+0000","last_clean":"2024-09-15T21:30:04.587078+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:04.587078+0000","last_undegraded":"2024-09-15T21:30:04.587078+0000","last_fullsized":"2024-09-15T21:30:04.587078+0000","mapping_epoch":44,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924544,"num_objects":714,"num_object_clones":0,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17539,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'19920","reported_seq":20060,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.239485+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:04.239485+0000","last_peered":"2024-09-15T21:30:04.239485+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:04.239485+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3137536,"num_objects":766,"num_object_clones":0,"num_object_copies":1532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16864,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3328}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19828","reported_seq":19967,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.266902+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:30:04.266902+0000","last_peered":"2024-09-15T21:30:04.266902+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:04.266902+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3121152,"num_objects":762,"num_object_clones":0,"num_object_copies":1524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16737,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3377}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19674","reported_seq":27990,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:06.080293+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:06.080293+0000","last_peered":"2024-09-15T21:30:06.080293+0000","last_clean":"2024-09-15T21:30:06.080293+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:06.080293+0000","last_undegraded":"2024-09-15T21:30:06.080293+0000","last_fullsized":"2024-09-15T21:30:06.080293+0000","mapping_epoch":46,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924544,"num_objects":714,"num_object_clones":0,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16158,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.526495+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:44.526495+0000","last_peered":"2024-09-15T21:27:44.526495+0000","last_clean":"2024-09-15T21:27:44.526495+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:44.526495+0000","last_undegraded":"2024-09-15T21:27:44.526495+0000","last_fullsized":"2024-09-15T21:27:44.526495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20079","reported_seq":20221,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:04.163631+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:04.163631+0000","last_peered":"2024-09-15T21:30:04.163631+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:04.163631+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3224518,"num_objects":789,"num_object_clones":0,"num_object_copies":1578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":789,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16783,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3358},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":48812790,"num_objects":11927,"num_object_clones":0,"num_object_copies":23854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11927,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":278528,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12290,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":298274816,"data_stored":298129378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161035,"ondisk_log_size":161035,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507856,"kb_used_data":155456,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93863984,"statfs":{"total":96636764160,"available":96116719616,"internally_reserved":0,"allocated":159186944,"data_stored":158901686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577520,"kb_used_data":104408,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93794320,"statfs":{"total":96636764160,"available":96045383680,"internally_reserved":0,"allocated":106913792,"data_stored":106628529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484437675},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508608,"kb_used_data":35440,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93863232,"statfs":{"total":96636764160,"available":96115949568,"internally_reserved":0,"allocated":36290560,"data_stored":35972321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,6,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":105312256,"data_stored":105276126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":34611200,"data_stored":34542094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":158351360,"data_stored":158311158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:10.628 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:11.217 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:11.218 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:11.248 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-15T21:30:10.872097+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39715593,"num_objects":9708,"num_object_clones":0,"num_object_copies":19416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9708,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":280751,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12452,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160659,"ondisk_log_size":160659,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1573984,"kb_used_data":283628,"kb_used_omap":12,"kb_used_meta":1290291,"kb_avail":281541536,"statfs":{"total":289910292480,"available":288298532864,"internally_reserved":0,"allocated":290435072,"data_stored":289546312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1321257987},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,6,3,11,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28622848,"num_objects":-6988,"num_object_clones":0,"num_object_copies":-13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-462,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":462,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003686"},"pg_stats":[{"pgid":"2.f","version":"47'20540","reported_seq":20622,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.217750+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:09.217750+0000","last_peered":"2024-09-15T21:30:09.217750+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:09.217750+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2506752,"num_objects":612,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20540,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3520}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20104","reported_seq":20216,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:09.615520+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:09.615520+0000","last_peered":"2024-09-15T21:30:09.615520+0000","last_clean":"2024-09-15T21:30:09.615520+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:09.615520+0000","last_undegraded":"2024-09-15T21:30:09.615520+0000","last_fullsized":"2024-09-15T21:30:09.615520+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2335686,"num_objects":572,"num_object_clones":0,"num_object_copies":1144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":572,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18385,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20521","reported_seq":29059,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.688047+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:30:09.688047+0000","last_peered":"2024-09-15T21:30:09.688047+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:30:09.688047+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2392088,"num_objects":585,"num_object_clones":0,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":66,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18621,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4202,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":519}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19720","reported_seq":19796,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:09.689896+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:09.689896+0000","last_peered":"2024-09-15T21:30:09.689896+0000","last_clean":"2024-09-15T21:30:09.689896+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:09.689896+0000","last_undegraded":"2024-09-15T21:30:09.689896+0000","last_fullsized":"2024-09-15T21:30:09.689896+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2433024,"num_objects":594,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19720,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19915","reported_seq":20054,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:09.248097+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:09.248097+0000","last_peered":"2024-09-15T21:30:09.248097+0000","last_clean":"2024-09-15T21:30:09.248097+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:09.248097+0000","last_undegraded":"2024-09-15T21:30:09.248097+0000","last_fullsized":"2024-09-15T21:30:09.248097+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2417123,"num_objects":591,"num_object_clones":0,"num_object_copies":1182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16370,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20021","reported_seq":20164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.226991+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:09.226991+0000","last_peered":"2024-09-15T21:30:09.226991+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:09.226991+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2515427,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16697,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3461},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20098","reported_seq":20240,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.216523+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:09.216523+0000","last_peered":"2024-09-15T21:30:09.216523+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:09.216523+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2343878,"num_objects":574,"num_object_clones":0,"num_object_copies":1148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16409,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3587},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20236","reported_seq":20371,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:09.182434+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:09.182434+0000","last_peered":"2024-09-15T21:30:09.182434+0000","last_clean":"2024-09-15T21:30:09.182434+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:09.182434+0000","last_undegraded":"2024-09-15T21:30:09.182434+0000","last_fullsized":"2024-09-15T21:30:09.182434+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2588672,"num_objects":632,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17008,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19944","reported_seq":20077,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.546029+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:09.546029+0000","last_peered":"2024-09-15T21:30:09.546029+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:09.546029+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2331107,"num_objects":570,"num_object_clones":0,"num_object_copies":1140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":570,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17876,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3517}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19821","reported_seq":19957,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.224559+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:09.224559+0000","last_peered":"2024-09-15T21:30:09.224559+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:09.224559+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2707456,"num_objects":661,"num_object_clones":0,"num_object_copies":1322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16494,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3359}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20062","reported_seq":20187,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.679094+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:09.679094+0000","last_peered":"2024-09-15T21:30:09.679094+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:09.679094+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2380259,"num_objects":582,"num_object_clones":0,"num_object_copies":1164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":582,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17921,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3508}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20003","reported_seq":28468,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:09.664991+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:09.664991+0000","last_peered":"2024-09-15T21:30:09.664991+0000","last_clean":"2024-09-15T21:30:09.664991+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:09.664991+0000","last_undegraded":"2024-09-15T21:30:09.664991+0000","last_fullsized":"2024-09-15T21:30:09.664991+0000","mapping_epoch":44,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2281472,"num_objects":557,"num_object_clones":0,"num_object_copies":1114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17696,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20081","reported_seq":20221,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.191853+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:09.191853+0000","last_peered":"2024-09-15T21:30:09.191853+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:09.191853+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2478080,"num_objects":605,"num_object_clones":0,"num_object_copies":1210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17025,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3489}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19976","reported_seq":20115,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.137971+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:30:09.137971+0000","last_peered":"2024-09-15T21:30:09.137971+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:09.137971+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2514944,"num_objects":614,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16885,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19674","reported_seq":27990,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:06.080293+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:06.080293+0000","last_peered":"2024-09-15T21:30:06.080293+0000","last_clean":"2024-09-15T21:30:06.080293+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:06.080293+0000","last_undegraded":"2024-09-15T21:30:06.080293+0000","last_fullsized":"2024-09-15T21:30:06.080293+0000","mapping_epoch":46,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2924544,"num_objects":714,"num_object_clones":0,"num_object_copies":1428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16158,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.526495+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:44.526495+0000","last_peered":"2024-09-15T21:27:44.526495+0000","last_clean":"2024-09-15T21:27:44.526495+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:44.526495+0000","last_undegraded":"2024-09-15T21:27:44.526495+0000","last_fullsized":"2024-09-15T21:27:44.526495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20240","reported_seq":20385,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.233367+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:09.233367+0000","last_peered":"2024-09-15T21:30:09.233367+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:09.233367+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2565062,"num_objects":628,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":628,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16944,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":39715574,"num_objects":9706,"num_object_clones":0,"num_object_copies":19412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9706,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":280749,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12451,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286355456,"data_stored":286210018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160656,"ondisk_log_size":160656,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507332,"kb_used_data":150772,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93864508,"statfs":{"total":96636764160,"available":96117256192,"internally_reserved":0,"allocated":154390528,"data_stored":154105270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558044,"kb_used_data":97416,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93813796,"statfs":{"total":96636764160,"available":96065327104,"internally_reserved":0,"allocated":99753984,"data_stored":99468721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471658155},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508608,"kb_used_data":35440,"kb_used_omap":4,"kb_used_meta":473147,"kb_avail":93863232,"statfs":{"total":96636764160,"available":96115949568,"internally_reserved":0,"allocated":36290560,"data_stored":35972321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484503215},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,6,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":98078720,"data_stored":98042590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":34611200,"data_stored":34542094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":153665536,"data_stored":153625334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:11.249 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:30:11.249 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "47'20540", 'reported_seq': 20622, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:09.217750+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:09.217750+0000', 'last_peered': '2024-09-15T21:30:09.217750+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:09.217750+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2506752, 'num_objects': 612, 'num_object_clones': 0, 'num_object_copies': 1224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 612, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20540, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3520}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:11.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:30:11.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20521", 'reported_seq': 29059, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:09.688047+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:30:09.688047+0000', 'last_peered': '2024-09-15T21:30:09.688047+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:30:09.688047+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2392088, 'num_objects': 585, 'num_object_clones': 0, 'num_object_copies': 1170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 66, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 585, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18621, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4202, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 519}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:11.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:30:11.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "47'20021", 'reported_seq': 20164, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:09.226991+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:09.226991+0000', 'last_peered': '2024-09-15T21:30:09.226991+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:09.226991+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2515427, 'num_objects': 615, 'num_object_clones': 0, 'num_object_copies': 1230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 615, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16697, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3461}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:11.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:30:11.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "47'20098", 'reported_seq': 20240, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:09.216523+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:09.216523+0000', 'last_peered': '2024-09-15T21:30:09.216523+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:09.216523+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2343878, 'num_objects': 574, 'num_object_clones': 0, 'num_object_copies': 1148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 574, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16409, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3587}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:11.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:30:11.250 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "47'19944", 'reported_seq': 20077, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:09.546029+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:30:09.546029+0000', 'last_peered': '2024-09-15T21:30:09.546029+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:30:09.546029+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2331107, 'num_objects': 570, 'num_object_clones': 0, 'num_object_copies': 1140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 570, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17876, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3517}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:11.250 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:11.251 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "47'19821", 'reported_seq': 19957, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:09.224559+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:09.224559+0000', 'last_peered': '2024-09-15T21:30:09.224559+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:09.224559+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2707456, 'num_objects': 661, 'num_object_clones': 0, 'num_object_copies': 1322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 661, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16494, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3359}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:11.251 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:30:11.251 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "47'20062", 'reported_seq': 20187, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:09.679094+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:09.679094+0000', 'last_peered': '2024-09-15T21:30:09.679094+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:09.679094+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2380259, 'num_objects': 582, 'num_object_clones': 0, 'num_object_copies': 1164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 582, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17921, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3508}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:11.251 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:30:11.251 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "47'20081", 'reported_seq': 20221, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:09.191853+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:09.191853+0000', 'last_peered': '2024-09-15T21:30:09.191853+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:09.191853+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2478080, 'num_objects': 605, 'num_object_clones': 0, 'num_object_copies': 1210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 605, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17025, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3489}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:11.251 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:30:11.251 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "47'19976", 'reported_seq': 20115, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:09.137971+0000', 'last_change': '2024-09-15T21:21:14.853318+0000', 'last_active': '2024-09-15T21:30:09.137971+0000', 'last_peered': '2024-09-15T21:30:09.137971+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:30:09.137971+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2514944, 'num_objects': 614, 'num_object_clones': 0, 'num_object_copies': 1228, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4142, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 614, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16885, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:11.251 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:30:11.252 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "47'20240", 'reported_seq': 20385, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:09.233367+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:09.233367+0000', 'last_peered': '2024-09-15T21:30:09.233367+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:09.233367+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2565062, 'num_objects': 628, 'num_object_clones': 0, 'num_object_copies': 1256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 628, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16944, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3519}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:11.252 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:11.948 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:11.948 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37023/19416 objects degraded (190.683%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":663,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9708,"data_bytes":39715593,"bytes_used":1611759616,"bytes_avail":288298532864,"bytes_total":289910292480,"degraded_objects":37023,"degraded_total":19416,"degraded_ratio":1.906829419035847,"recovering_objects_per_sec":38,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":462,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":582},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:11.979 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:11.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:12.534 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:12.535 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:12.556 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-09-15T21:30:11.938767+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":39138057,"num_objects":9567,"num_object_clones":0,"num_object_copies":19134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9567,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":280892,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12452,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160600,"ondisk_log_size":160600,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1576424,"kb_used_data":276332,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281539096,"statfs":{"total":289910292480,"available":288296034304,"internally_reserved":0,"allocated":282963968,"data_stored":282075208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1331219459},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,6,5,9,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20058112,"num_objects":-4897,"num_object_clones":0,"num_object_copies":-9794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4897,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.069733"},"pg_stats":[{"pgid":"2.f","version":"47'20540","reported_seq":20622,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.217750+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:09.217750+0000","last_peered":"2024-09-15T21:30:09.217750+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:09.217750+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2506752,"num_objects":612,"num_object_clones":0,"num_object_copies":1224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20540,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3520}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'20104","reported_seq":20216,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:09.615520+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:09.615520+0000","last_peered":"2024-09-15T21:30:09.615520+0000","last_clean":"2024-09-15T21:30:09.615520+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:09.615520+0000","last_undegraded":"2024-09-15T21:30:09.615520+0000","last_fullsized":"2024-09-15T21:30:09.615520+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2335686,"num_objects":572,"num_object_clones":0,"num_object_copies":1144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":572,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18385,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20521","reported_seq":29059,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.688047+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:30:09.688047+0000","last_peered":"2024-09-15T21:30:09.688047+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:30:09.688047+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2392088,"num_objects":585,"num_object_clones":0,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":66,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18621,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4202,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":519}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'19720","reported_seq":19796,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:09.689896+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:09.689896+0000","last_peered":"2024-09-15T21:30:09.689896+0000","last_clean":"2024-09-15T21:30:09.689896+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:09.689896+0000","last_undegraded":"2024-09-15T21:30:09.689896+0000","last_fullsized":"2024-09-15T21:30:09.689896+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2433024,"num_objects":594,"num_object_clones":0,"num_object_copies":1188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19720,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'19915","reported_seq":20054,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:09.248097+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:09.248097+0000","last_peered":"2024-09-15T21:30:09.248097+0000","last_clean":"2024-09-15T21:30:09.248097+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:09.248097+0000","last_undegraded":"2024-09-15T21:30:09.248097+0000","last_fullsized":"2024-09-15T21:30:09.248097+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2417123,"num_objects":591,"num_object_clones":0,"num_object_copies":1182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16370,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'20021","reported_seq":20164,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.226991+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:09.226991+0000","last_peered":"2024-09-15T21:30:09.226991+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:09.226991+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2515427,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16697,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3461},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'20098","reported_seq":20240,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.216523+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:09.216523+0000","last_peered":"2024-09-15T21:30:09.216523+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:09.216523+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2343878,"num_objects":574,"num_object_clones":0,"num_object_copies":1148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16409,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3587},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'20236","reported_seq":20371,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:09.182434+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:09.182434+0000","last_peered":"2024-09-15T21:30:09.182434+0000","last_clean":"2024-09-15T21:30:09.182434+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:09.182434+0000","last_undegraded":"2024-09-15T21:30:09.182434+0000","last_fullsized":"2024-09-15T21:30:09.182434+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2588672,"num_objects":632,"num_object_clones":0,"num_object_copies":1264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17008,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'19944","reported_seq":20077,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.546029+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:09.546029+0000","last_peered":"2024-09-15T21:30:09.546029+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:09.546029+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2331107,"num_objects":570,"num_object_clones":0,"num_object_copies":1140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":570,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17876,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3517}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.367911+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:27:09.367911+0000","last_peered":"2024-09-15T21:27:09.367911+0000","last_clean":"2024-09-15T21:27:09.367911+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:27:09.367911+0000","last_undegraded":"2024-09-15T21:27:09.367911+0000","last_fullsized":"2024-09-15T21:27:09.367911+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'19821","reported_seq":19957,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.224559+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:09.224559+0000","last_peered":"2024-09-15T21:30:09.224559+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:09.224559+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2707456,"num_objects":661,"num_object_clones":0,"num_object_copies":1322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16494,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3359}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.318630+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:27:09.318630+0000","last_peered":"2024-09-15T21:27:09.318630+0000","last_clean":"2024-09-15T21:27:09.318630+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:27:09.318630+0000","last_undegraded":"2024-09-15T21:27:09.318630+0000","last_fullsized":"2024-09-15T21:27:09.318630+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'20062","reported_seq":20187,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.679094+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:09.679094+0000","last_peered":"2024-09-15T21:30:09.679094+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:09.679094+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2380259,"num_objects":582,"num_object_clones":0,"num_object_copies":1164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":582,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17921,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3508}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'20003","reported_seq":28468,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:09.664991+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:09.664991+0000","last_peered":"2024-09-15T21:30:09.664991+0000","last_clean":"2024-09-15T21:30:09.664991+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:09.664991+0000","last_undegraded":"2024-09-15T21:30:09.664991+0000","last_fullsized":"2024-09-15T21:30:09.664991+0000","mapping_epoch":44,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2281472,"num_objects":557,"num_object_clones":0,"num_object_copies":1114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17696,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'20081","reported_seq":20221,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.191853+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:09.191853+0000","last_peered":"2024-09-15T21:30:09.191853+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:09.191853+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2478080,"num_objects":605,"num_object_clones":0,"num_object_copies":1210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17025,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3489}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'19976","reported_seq":20115,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.137971+0000","last_change":"2024-09-15T21:21:14.853318+0000","last_active":"2024-09-15T21:30:09.137971+0000","last_peered":"2024-09-15T21:30:09.137971+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:09.137971+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2514944,"num_objects":614,"num_object_clones":0,"num_object_copies":1228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4142,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16885,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3525}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:09.370063+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:27:09.370063+0000","last_peered":"2024-09-15T21:27:09.370063+0000","last_clean":"2024-09-15T21:27:09.370063+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:27:09.370063+0000","last_undegraded":"2024-09-15T21:27:09.370063+0000","last_fullsized":"2024-09-15T21:27:09.370063+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19815","reported_seq":28131,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:11.035187+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:11.035187+0000","last_peered":"2024-09-15T21:30:11.035187+0000","last_clean":"2024-09-15T21:30:11.035187+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:11.035187+0000","last_undegraded":"2024-09-15T21:30:11.035187+0000","last_fullsized":"2024-09-15T21:30:11.035187+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2347008,"num_objects":573,"num_object_clones":0,"num_object_copies":1146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16299,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.526495+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:44.526495+0000","last_peered":"2024-09-15T21:27:44.526495+0000","last_clean":"2024-09-15T21:27:44.526495+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:44.526495+0000","last_undegraded":"2024-09-15T21:27:44.526495+0000","last_fullsized":"2024-09-15T21:27:44.526495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'20240","reported_seq":20385,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.233367+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:09.233367+0000","last_peered":"2024-09-15T21:30:09.233367+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:09.233367+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2565062,"num_objects":628,"num_object_clones":0,"num_object_copies":1256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":628,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16944,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3519},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":39138038,"num_objects":9565,"num_object_clones":0,"num_object_copies":19130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37023,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9565,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":280890,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12451,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278941696,"data_stored":278796258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160597,"ondisk_log_size":160597,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507332,"kb_used_data":150772,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93864508,"statfs":{"total":96636764160,"available":96117256192,"internally_reserved":0,"allocated":154390528,"data_stored":154105270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558044,"kb_used_data":97416,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93813796,"statfs":{"total":96636764160,"available":96065327104,"internally_reserved":0,"allocated":99753984,"data_stored":99468721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471658155},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511048,"kb_used_data":28144,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93860792,"statfs":{"total":96636764160,"available":96113451008,"internally_reserved":0,"allocated":28819456,"data_stored":28501217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":98078720,"data_stored":98042590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27197440,"data_stored":27128334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":153665536,"data_stored":153625334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:15.558 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:16.131 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:16.131 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:16.154 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-09-15T21:30:14.874631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":31556361,"num_objects":7716,"num_object_clones":0,"num_object_copies":15432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7716,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":282743,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12591,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160751,"ondisk_log_size":160751,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1574416,"kb_used_data":265968,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281541104,"statfs":{"total":289910292480,"available":288298090496,"internally_reserved":0,"allocated":272351232,"data_stored":271443150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1339739139},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,10,4,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17256448,"num_objects":-4213,"num_object_clones":0,"num_object_copies":-8426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4213,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003896"},"pg_stats":[{"pgid":"2.f","version":"49'20691","reported_seq":20777,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.252480+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:14.252480+0000","last_peered":"2024-09-15T21:30:14.252480+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:14.252480+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1888256,"num_objects":461,"num_object_clones":0,"num_object_copies":922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20691,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20221","reported_seq":20337,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.552420+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:14.552420+0000","last_peered":"2024-09-15T21:30:14.552420+0000","last_clean":"2024-09-15T21:30:14.552420+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:14.552420+0000","last_undegraded":"2024-09-15T21:30:14.552420+0000","last_fullsized":"2024-09-15T21:30:14.552420+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1856454,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18502,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20521","reported_seq":29059,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.688047+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:30:09.688047+0000","last_peered":"2024-09-15T21:30:09.688047+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:30:09.688047+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2392088,"num_objects":585,"num_object_clones":0,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":66,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18621,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4202,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":519}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19855","reported_seq":19936,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.632240+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:14.632240+0000","last_peered":"2024-09-15T21:30:14.632240+0000","last_clean":"2024-09-15T21:30:14.632240+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:14.632240+0000","last_undegraded":"2024-09-15T21:30:14.632240+0000","last_fullsized":"2024-09-15T21:30:14.632240+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1880064,"num_objects":459,"num_object_clones":0,"num_object_copies":918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19855,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20036","reported_seq":20177,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.140572+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:14.140572+0000","last_peered":"2024-09-15T21:30:14.140572+0000","last_clean":"2024-09-15T21:30:14.140572+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:14.140572+0000","last_undegraded":"2024-09-15T21:30:14.140572+0000","last_fullsized":"2024-09-15T21:30:14.140572+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1921507,"num_objects":470,"num_object_clones":0,"num_object_copies":940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16491,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20153","reported_seq":20300,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.298934+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:14.298934+0000","last_peered":"2024-09-15T21:30:14.298934+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:14.298934+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1974755,"num_objects":483,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16829,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3593},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20228","reported_seq":20375,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.279020+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:14.279020+0000","last_peered":"2024-09-15T21:30:14.279020+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:14.279020+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1811398,"num_objects":444,"num_object_clones":0,"num_object_copies":888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3717},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20361","reported_seq":20500,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.292172+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:14.292172+0000","last_peered":"2024-09-15T21:30:14.292172+0000","last_clean":"2024-09-15T21:30:14.292172+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:14.292172+0000","last_undegraded":"2024-09-15T21:30:14.292172+0000","last_fullsized":"2024-09-15T21:30:14.292172+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2076672,"num_objects":507,"num_object_clones":0,"num_object_copies":1014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17133,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20066","reported_seq":20203,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.654680+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:14.654680+0000","last_peered":"2024-09-15T21:30:14.654680+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:14.654680+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1831395,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17998,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3639}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19987","reported_seq":20128,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.295541+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:14.295541+0000","last_peered":"2024-09-15T21:30:14.295541+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:14.295541+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2027520,"num_objects":495,"num_object_clones":0,"num_object_copies":990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16660,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966943+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966943+0000","last_peered":"2024-09-15T21:30:12.966943+0000","last_clean":"2024-09-15T21:30:12.966943+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966943+0000","last_undegraded":"2024-09-15T21:30:12.966943+0000","last_fullsized":"2024-09-15T21:30:12.966943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20189","reported_seq":20318,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.656278+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:14.656278+0000","last_peered":"2024-09-15T21:30:14.656278+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:14.656278+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1860067,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18048,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3635}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20145","reported_seq":28614,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.674130+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:14.674130+0000","last_peered":"2024-09-15T21:30:14.674130+0000","last_clean":"2024-09-15T21:30:14.674130+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:14.674130+0000","last_undegraded":"2024-09-15T21:30:14.674130+0000","last_fullsized":"2024-09-15T21:30:14.674130+0000","mapping_epoch":44,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1699840,"num_objects":415,"num_object_clones":0,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17838,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20204","reported_seq":20348,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.340977+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:14.340977+0000","last_peered":"2024-09-15T21:30:14.340977+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:14.340977+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1974272,"num_objects":482,"num_object_clones":0,"num_object_copies":964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17148,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3612}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20102","reported_seq":20524,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.321503+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:14.321503+0000","last_peered":"2024-09-15T21:30:14.321503+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:14.321503+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1998848,"num_objects":488,"num_object_clones":0,"num_object_copies":976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17011,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19815","reported_seq":28131,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:11.035187+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:11.035187+0000","last_peered":"2024-09-15T21:30:11.035187+0000","last_clean":"2024-09-15T21:30:11.035187+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:11.035187+0000","last_undegraded":"2024-09-15T21:30:11.035187+0000","last_fullsized":"2024-09-15T21:30:11.035187+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2347008,"num_objects":573,"num_object_clones":0,"num_object_copies":1146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16299,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.526495+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:44.526495+0000","last_peered":"2024-09-15T21:27:44.526495+0000","last_clean":"2024-09-15T21:27:44.526495+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:44.526495+0000","last_undegraded":"2024-09-15T21:27:44.526495+0000","last_fullsized":"2024-09-15T21:27:44.526495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20374","reported_seq":20523,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.248012+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:14.248012+0000","last_peered":"2024-09-15T21:30:14.248012+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:14.248012+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2016198,"num_objects":494,"num_object_clones":0,"num_object_copies":988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":494,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17078,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3653},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":31556342,"num_objects":7714,"num_object_clones":0,"num_object_copies":15428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7714,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":282741,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12590,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":268111872,"data_stored":267966434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160748,"ondisk_log_size":160748,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507524,"kb_used_data":146788,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93864316,"statfs":{"total":96636764160,"available":96117059584,"internally_reserved":0,"allocated":150310912,"data_stored":150015993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555844,"kb_used_data":91036,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93815996,"statfs":{"total":96636764160,"available":96067579904,"internally_reserved":0,"allocated":93220864,"data_stored":92925940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475917995},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,7,0,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511048,"kb_used_data":28144,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93860792,"statfs":{"total":96636764160,"available":96113451008,"internally_reserved":0,"allocated":28819456,"data_stored":28501217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":91471872,"data_stored":91435742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27197440,"data_stored":27128334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":149442560,"data_stored":149402358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:16.156 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:16.700 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:16.723 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-09-15T21:30:14.874631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":31556361,"num_objects":7716,"num_object_clones":0,"num_object_copies":15432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7716,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":282743,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12591,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160751,"ondisk_log_size":160751,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1574416,"kb_used_data":265968,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281541104,"statfs":{"total":289910292480,"available":288298090496,"internally_reserved":0,"allocated":272351232,"data_stored":271443150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1339739139},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,10,4,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17256448,"num_objects":-4213,"num_object_clones":0,"num_object_copies":-8426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4213,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003896"},"pg_stats":[{"pgid":"2.f","version":"49'20691","reported_seq":20777,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.252480+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:14.252480+0000","last_peered":"2024-09-15T21:30:14.252480+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:14.252480+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1888256,"num_objects":461,"num_object_clones":0,"num_object_copies":922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20691,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20221","reported_seq":20337,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.552420+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:14.552420+0000","last_peered":"2024-09-15T21:30:14.552420+0000","last_clean":"2024-09-15T21:30:14.552420+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:14.552420+0000","last_undegraded":"2024-09-15T21:30:14.552420+0000","last_fullsized":"2024-09-15T21:30:14.552420+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1856454,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18502,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'20521","reported_seq":29059,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:09.688047+0000","last_change":"2024-09-15T21:27:07.376770+0000","last_active":"2024-09-15T21:30:09.688047+0000","last_peered":"2024-09-15T21:30:09.688047+0000","last_clean":"2024-09-15T21:21:12.349985+0000","last_became_active":"2024-09-15T21:21:14.810109+0000","last_became_peered":"2024-09-15T21:21:14.810109+0000","last_unstale":"2024-09-15T21:30:09.688047+0000","last_undegraded":"2024-09-15T21:21:14.398819+0000","last_fullsized":"2024-09-15T21:21:14.351050+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2392088,"num_objects":585,"num_object_clones":0,"num_object_copies":1170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":66,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":585,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18621,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4202,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":519}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19855","reported_seq":19936,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.632240+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:14.632240+0000","last_peered":"2024-09-15T21:30:14.632240+0000","last_clean":"2024-09-15T21:30:14.632240+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:14.632240+0000","last_undegraded":"2024-09-15T21:30:14.632240+0000","last_fullsized":"2024-09-15T21:30:14.632240+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1880064,"num_objects":459,"num_object_clones":0,"num_object_copies":918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19855,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20036","reported_seq":20177,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.140572+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:14.140572+0000","last_peered":"2024-09-15T21:30:14.140572+0000","last_clean":"2024-09-15T21:30:14.140572+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:14.140572+0000","last_undegraded":"2024-09-15T21:30:14.140572+0000","last_fullsized":"2024-09-15T21:30:14.140572+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1921507,"num_objects":470,"num_object_clones":0,"num_object_copies":940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16491,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20153","reported_seq":20300,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.298934+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:14.298934+0000","last_peered":"2024-09-15T21:30:14.298934+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:14.298934+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1974755,"num_objects":483,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16829,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3593},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20228","reported_seq":20375,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.279020+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:14.279020+0000","last_peered":"2024-09-15T21:30:14.279020+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:14.279020+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1811398,"num_objects":444,"num_object_clones":0,"num_object_copies":888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3717},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20361","reported_seq":20500,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.292172+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:14.292172+0000","last_peered":"2024-09-15T21:30:14.292172+0000","last_clean":"2024-09-15T21:30:14.292172+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:14.292172+0000","last_undegraded":"2024-09-15T21:30:14.292172+0000","last_fullsized":"2024-09-15T21:30:14.292172+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2076672,"num_objects":507,"num_object_clones":0,"num_object_copies":1014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17133,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:14.182005+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:27:14.182005+0000","last_peered":"2024-09-15T21:27:14.182005+0000","last_clean":"2024-09-15T21:27:14.182005+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:27:14.182005+0000","last_undegraded":"2024-09-15T21:27:14.182005+0000","last_fullsized":"2024-09-15T21:27:14.182005+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20066","reported_seq":20203,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.654680+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:14.654680+0000","last_peered":"2024-09-15T21:30:14.654680+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:14.654680+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1831395,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17998,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3639}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19987","reported_seq":20128,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.295541+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:14.295541+0000","last_peered":"2024-09-15T21:30:14.295541+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:14.295541+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2027520,"num_objects":495,"num_object_clones":0,"num_object_copies":990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16660,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966943+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966943+0000","last_peered":"2024-09-15T21:30:12.966943+0000","last_clean":"2024-09-15T21:30:12.966943+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966943+0000","last_undegraded":"2024-09-15T21:30:12.966943+0000","last_fullsized":"2024-09-15T21:30:12.966943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20189","reported_seq":20318,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.656278+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:14.656278+0000","last_peered":"2024-09-15T21:30:14.656278+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:14.656278+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1860067,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18048,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3635}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":86,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.394985+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:27:29.394985+0000","last_peered":"2024-09-15T21:27:29.394985+0000","last_clean":"2024-09-15T21:27:29.394985+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:27:29.394985+0000","last_undegraded":"2024-09-15T21:27:29.394985+0000","last_fullsized":"2024-09-15T21:27:29.394985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20145","reported_seq":28614,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.674130+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:14.674130+0000","last_peered":"2024-09-15T21:30:14.674130+0000","last_clean":"2024-09-15T21:30:14.674130+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:14.674130+0000","last_undegraded":"2024-09-15T21:30:14.674130+0000","last_fullsized":"2024-09-15T21:30:14.674130+0000","mapping_epoch":44,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1699840,"num_objects":415,"num_object_clones":0,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17838,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20204","reported_seq":20348,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.340977+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:14.340977+0000","last_peered":"2024-09-15T21:30:14.340977+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:14.340977+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1974272,"num_objects":482,"num_object_clones":0,"num_object_copies":964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17148,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3612}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395010+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:27:29.395010+0000","last_peered":"2024-09-15T21:27:29.395010+0000","last_clean":"2024-09-15T21:27:29.395010+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:27:29.395010+0000","last_undegraded":"2024-09-15T21:27:29.395010+0000","last_fullsized":"2024-09-15T21:27:29.395010+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20102","reported_seq":20524,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.321503+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:14.321503+0000","last_peered":"2024-09-15T21:30:14.321503+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:14.321503+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1998848,"num_objects":488,"num_object_clones":0,"num_object_copies":976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17011,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'19815","reported_seq":28131,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:30:11.035187+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:11.035187+0000","last_peered":"2024-09-15T21:30:11.035187+0000","last_clean":"2024-09-15T21:30:11.035187+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:11.035187+0000","last_undegraded":"2024-09-15T21:30:11.035187+0000","last_fullsized":"2024-09-15T21:30:11.035187+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2347008,"num_objects":573,"num_object_clones":0,"num_object_copies":1146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16299,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":76,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:29.395025+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:27:29.395025+0000","last_peered":"2024-09-15T21:27:29.395025+0000","last_clean":"2024-09-15T21:27:29.395025+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:27:29.395025+0000","last_undegraded":"2024-09-15T21:27:29.395025+0000","last_fullsized":"2024-09-15T21:27:29.395025+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-15T21:27:44.526495+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:27:44.526495+0000","last_peered":"2024-09-15T21:27:44.526495+0000","last_clean":"2024-09-15T21:27:44.526495+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:27:44.526495+0000","last_undegraded":"2024-09-15T21:27:44.526495+0000","last_fullsized":"2024-09-15T21:27:44.526495+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20374","reported_seq":20523,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.248012+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:14.248012+0000","last_peered":"2024-09-15T21:30:14.248012+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:14.248012+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2016198,"num_objects":494,"num_object_clones":0,"num_object_copies":988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":494,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17078,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3653},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":31556342,"num_objects":7714,"num_object_clones":0,"num_object_copies":15428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7714,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":282741,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12590,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":268111872,"data_stored":267966434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160748,"ondisk_log_size":160748,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507524,"kb_used_data":146788,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93864316,"statfs":{"total":96636764160,"available":96117059584,"internally_reserved":0,"allocated":150310912,"data_stored":150015993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555844,"kb_used_data":91036,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93815996,"statfs":{"total":96636764160,"available":96067579904,"internally_reserved":0,"allocated":93220864,"data_stored":92925940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475917995},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,7,0,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511048,"kb_used_data":28144,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93860792,"statfs":{"total":96636764160,"available":96113451008,"internally_reserved":0,"allocated":28819456,"data_stored":28501217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,3,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":91471872,"data_stored":91435742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":27197440,"data_stored":27128334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":149442560,"data_stored":149402358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:16.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:30:16.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20691", 'reported_seq': 20777, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:14.252480+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:14.252480+0000', 'last_peered': '2024-09-15T21:30:14.252480+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:14.252480+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1888256, 'num_objects': 461, 'num_object_clones': 0, 'num_object_copies': 922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 461, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20691, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3671}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:16.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.d is not active+clean 2024-09-15T21:30:16.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.d', 'version': "47'20521", 'reported_seq': 29059, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:09.688047+0000', 'last_change': '2024-09-15T21:27:07.376770+0000', 'last_active': '2024-09-15T21:30:09.688047+0000', 'last_peered': '2024-09-15T21:30:09.688047+0000', 'last_clean': '2024-09-15T21:21:12.349985+0000', 'last_became_active': '2024-09-15T21:21:14.810109+0000', 'last_became_peered': '2024-09-15T21:21:14.810109+0000', 'last_unstale': '2024-09-15T21:30:09.688047+0000', 'last_undegraded': '2024-09-15T21:21:14.398819+0000', 'last_fullsized': '2024-09-15T21:21:14.351050+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1928", 'last_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:44.532058+0000', 'objects_scrubbed': 1896, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 603, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2392088, 'num_objects': 585, 'num_object_clones': 0, 'num_object_copies': 1170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 66, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 585, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18621, 'num_write_kb': 34617, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4202, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 519}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:16.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:30:16.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20153", 'reported_seq': 20300, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:14.298934+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:14.298934+0000', 'last_peered': '2024-09-15T21:30:14.298934+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:14.298934+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1974755, 'num_objects': 483, 'num_object_clones': 0, 'num_object_copies': 966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 483, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16829, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3593}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:16.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:30:16.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20228", 'reported_seq': 20375, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:14.279020+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:14.279020+0000', 'last_peered': '2024-09-15T21:30:14.279020+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:14.279020+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1811398, 'num_objects': 444, 'num_object_clones': 0, 'num_object_copies': 888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 444, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16539, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3717}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:16.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:30:16.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20066", 'reported_seq': 20203, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:14.654680+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:30:14.654680+0000', 'last_peered': '2024-09-15T21:30:14.654680+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:30:14.654680+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1831395, 'num_objects': 448, 'num_object_clones': 0, 'num_object_copies': 896, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 448, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17998, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3639}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:16.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:16.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'19987", 'reported_seq': 20128, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:14.295541+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:14.295541+0000', 'last_peered': '2024-09-15T21:30:14.295541+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:14.295541+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2027520, 'num_objects': 495, 'num_object_clones': 0, 'num_object_copies': 990, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 495, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16660, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3525}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:16.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:30:16.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20189", 'reported_seq': 20318, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:14.656278+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:14.656278+0000', 'last_peered': '2024-09-15T21:30:14.656278+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:14.656278+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1860067, 'num_objects': 455, 'num_object_clones': 0, 'num_object_copies': 910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 455, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18048, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3635}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:16.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:30:16.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20204", 'reported_seq': 20348, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:14.340977+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:14.340977+0000', 'last_peered': '2024-09-15T21:30:14.340977+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:14.340977+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1974272, 'num_objects': 482, 'num_object_clones': 0, 'num_object_copies': 964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 482, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17148, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3612}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:16.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:30:16.726 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "49'20102", 'reported_seq': 20524, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:14.321503+0000', 'last_change': '2024-09-15T21:30:11.380283+0000', 'last_active': '2024-09-15T21:30:14.321503+0000', 'last_peered': '2024-09-15T21:30:14.321503+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:30:14.321503+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1998848, 'num_objects': 488, 'num_object_clones': 0, 'num_object_copies': 976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4003, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 488, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17011, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 139, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:16.726 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:30:16.726 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20374", 'reported_seq': 20523, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:14.248012+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:14.248012+0000', 'last_peered': '2024-09-15T21:30:14.248012+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:14.248012+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2016198, 'num_objects': 494, 'num_object_clones': 0, 'num_object_copies': 988, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 494, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17078, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3653}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:16.726 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:17.422 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:17.423 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36818/14818 objects degraded (248.468%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":669,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":7409,"data_bytes":30298889,"bytes_used":1612722176,"bytes_avail":288297570304,"bytes_total":289910292480,"degraded_objects":36818,"degraded_total":14818,"degraded_ratio":2.4846807936293698,"recovering_objects_per_sec":45,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":366,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":564},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:17.450 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:17.450 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:18.035 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:18.038 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:18.068 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":376,"stamp":"2024-09-15T21:30:16.875274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":30298889,"num_objects":7409,"num_object_clones":0,"num_object_copies":14818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7409,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":283050,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12657,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1574924,"kb_used_data":258152,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281540596,"statfs":{"total":289910292480,"available":288297570304,"internally_reserved":0,"allocated":264347648,"data_stored":263440744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1348258819},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,10,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18513920,"num_objects":-4520,"num_object_clones":0,"num_object_copies":-9040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4520,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":366,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003760"},"pg_stats":[{"pgid":"2.f","version":"49'20691","reported_seq":20777,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.252480+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:14.252480+0000","last_peered":"2024-09-15T21:30:14.252480+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:14.252480+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1888256,"num_objects":461,"num_object_clones":0,"num_object_copies":922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20691,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20221","reported_seq":20337,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.552420+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:14.552420+0000","last_peered":"2024-09-15T21:30:14.552420+0000","last_clean":"2024-09-15T21:30:14.552420+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:14.552420+0000","last_undegraded":"2024-09-15T21:30:14.552420+0000","last_fullsized":"2024-09-15T21:30:14.552420+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1856454,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18502,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20689","reported_seq":29372,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:16.261333+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:16.261333+0000","last_peered":"2024-09-15T21:30:16.261333+0000","last_clean":"2024-09-15T21:30:16.261333+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:16.261333+0000","last_undegraded":"2024-09-15T21:30:16.261333+0000","last_fullsized":"2024-09-15T21:30:16.261333+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703960,"num_objects":417,"num_object_clones":0,"num_object_copies":834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":417,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18789,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19855","reported_seq":19936,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.632240+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:14.632240+0000","last_peered":"2024-09-15T21:30:14.632240+0000","last_clean":"2024-09-15T21:30:14.632240+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:14.632240+0000","last_undegraded":"2024-09-15T21:30:14.632240+0000","last_fullsized":"2024-09-15T21:30:14.632240+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1880064,"num_objects":459,"num_object_clones":0,"num_object_copies":918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19855,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20036","reported_seq":20177,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.140572+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:14.140572+0000","last_peered":"2024-09-15T21:30:14.140572+0000","last_clean":"2024-09-15T21:30:14.140572+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:14.140572+0000","last_undegraded":"2024-09-15T21:30:14.140572+0000","last_fullsized":"2024-09-15T21:30:14.140572+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1921507,"num_objects":470,"num_object_clones":0,"num_object_copies":940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":470,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16491,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20153","reported_seq":20300,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.298934+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:14.298934+0000","last_peered":"2024-09-15T21:30:14.298934+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:14.298934+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1974755,"num_objects":483,"num_object_clones":0,"num_object_copies":966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":483,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16829,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3593},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20228","reported_seq":20375,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.279020+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:14.279020+0000","last_peered":"2024-09-15T21:30:14.279020+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:14.279020+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1811398,"num_objects":444,"num_object_clones":0,"num_object_copies":888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3717},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20361","reported_seq":20500,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.292172+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:14.292172+0000","last_peered":"2024-09-15T21:30:14.292172+0000","last_clean":"2024-09-15T21:30:14.292172+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:14.292172+0000","last_undegraded":"2024-09-15T21:30:14.292172+0000","last_fullsized":"2024-09-15T21:30:14.292172+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2076672,"num_objects":507,"num_object_clones":0,"num_object_copies":1014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17133,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20066","reported_seq":20203,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.654680+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:14.654680+0000","last_peered":"2024-09-15T21:30:14.654680+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:14.654680+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1831395,"num_objects":448,"num_object_clones":0,"num_object_copies":896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":448,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17998,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3639}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19987","reported_seq":20128,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.295541+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:14.295541+0000","last_peered":"2024-09-15T21:30:14.295541+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:14.295541+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2027520,"num_objects":495,"num_object_clones":0,"num_object_copies":990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16660,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966943+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966943+0000","last_peered":"2024-09-15T21:30:12.966943+0000","last_clean":"2024-09-15T21:30:12.966943+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966943+0000","last_undegraded":"2024-09-15T21:30:12.966943+0000","last_fullsized":"2024-09-15T21:30:12.966943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20189","reported_seq":20318,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.656278+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:14.656278+0000","last_peered":"2024-09-15T21:30:14.656278+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:14.656278+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1860067,"num_objects":455,"num_object_clones":0,"num_object_copies":910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":455,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18048,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3635}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20145","reported_seq":28614,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:14.674130+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:14.674130+0000","last_peered":"2024-09-15T21:30:14.674130+0000","last_clean":"2024-09-15T21:30:14.674130+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:14.674130+0000","last_undegraded":"2024-09-15T21:30:14.674130+0000","last_fullsized":"2024-09-15T21:30:14.674130+0000","mapping_epoch":44,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1699840,"num_objects":415,"num_object_clones":0,"num_object_copies":830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17838,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20204","reported_seq":20348,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.340977+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:14.340977+0000","last_peered":"2024-09-15T21:30:14.340977+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:14.340977+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1974272,"num_objects":482,"num_object_clones":0,"num_object_copies":964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17148,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3612}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20102","reported_seq":20524,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.321503+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:14.321503+0000","last_peered":"2024-09-15T21:30:14.321503+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:14.321503+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1998848,"num_objects":488,"num_object_clones":0,"num_object_copies":976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":488,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17011,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19954","reported_seq":28274,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:16.168985+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:16.168985+0000","last_peered":"2024-09-15T21:30:16.168985+0000","last_clean":"2024-09-15T21:30:16.168985+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:16.168985+0000","last_undegraded":"2024-09-15T21:30:16.168985+0000","last_fullsized":"2024-09-15T21:30:16.168985+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1777664,"num_objects":434,"num_object_clones":0,"num_object_copies":868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16438,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20374","reported_seq":20523,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:14.248012+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:14.248012+0000","last_peered":"2024-09-15T21:30:14.248012+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:14.248012+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2016198,"num_objects":494,"num_object_clones":0,"num_object_copies":988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":494,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17078,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3653},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":30298870,"num_objects":7407,"num_object_clones":0,"num_object_copies":14814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7407,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":283048,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12656,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":261287936,"data_stored":261153337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507524,"kb_used_data":146788,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93864316,"statfs":{"total":96636764160,"available":96117059584,"internally_reserved":0,"allocated":150310912,"data_stored":150015993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555844,"kb_used_data":91036,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93815996,"statfs":{"total":96636764160,"available":96067579904,"internally_reserved":0,"allocated":93220864,"data_stored":92925940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":475917995},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,7,0,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511556,"kb_used_data":20328,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93860284,"statfs":{"total":96636764160,"available":96112930816,"internally_reserved":0,"allocated":20815872,"data_stored":20498811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":91471872,"data_stored":91435742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20373504,"data_stored":20315237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":149442560,"data_stored":149402358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:21.070 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:21.649 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:21.650 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:21.673 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-09-15T21:30:20.876622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21713673,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":285146,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12741,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160854,"ondisk_log_size":160854,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1575268,"kb_used_data":246044,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281540252,"statfs":{"total":289910292480,"available":288297218048,"internally_reserved":0,"allocated":251949056,"data_stored":251042152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1361038339},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,6,7,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17424384,"num_objects":-4254,"num_object_clones":0,"num_object_copies":-8508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4254,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.937854"},"pg_stats":[{"pgid":"2.f","version":"49'20839","reported_seq":20923,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.234192+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:19.234192+0000","last_peered":"2024-09-15T21:30:19.234192+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:19.234192+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1282048,"num_objects":313,"num_object_clones":0,"num_object_copies":626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20839,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3819}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20377","reported_seq":20493,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.691169+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:19.691169+0000","last_peered":"2024-09-15T21:30:19.691169+0000","last_clean":"2024-09-15T21:30:19.691169+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:19.691169+0000","last_undegraded":"2024-09-15T21:30:19.691169+0000","last_fullsized":"2024-09-15T21:30:19.691169+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1217478,"num_objects":299,"num_object_clones":0,"num_object_copies":598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":299,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18658,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20689","reported_seq":29372,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:16.261333+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:16.261333+0000","last_peered":"2024-09-15T21:30:16.261333+0000","last_clean":"2024-09-15T21:30:16.261333+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:16.261333+0000","last_undegraded":"2024-09-15T21:30:16.261333+0000","last_fullsized":"2024-09-15T21:30:16.261333+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703960,"num_objects":417,"num_object_clones":0,"num_object_copies":834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":417,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18789,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19992","reported_seq":20072,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.699145+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:19.699145+0000","last_peered":"2024-09-15T21:30:19.699145+0000","last_clean":"2024-09-15T21:30:19.699145+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:19.699145+0000","last_undegraded":"2024-09-15T21:30:19.699145+0000","last_fullsized":"2024-09-15T21:30:19.699145+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1318912,"num_objects":322,"num_object_clones":0,"num_object_copies":644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19992,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20190","reported_seq":20333,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.312454+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:19.312454+0000","last_peered":"2024-09-15T21:30:19.312454+0000","last_clean":"2024-09-15T21:30:19.312454+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:19.312454+0000","last_undegraded":"2024-09-15T21:30:19.312454+0000","last_fullsized":"2024-09-15T21:30:19.312454+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1290723,"num_objects":316,"num_object_clones":0,"num_object_copies":632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16645,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20315","reported_seq":20462,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.329955+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:19.329955+0000","last_peered":"2024-09-15T21:30:19.329955+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:19.329955+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1311203,"num_objects":321,"num_object_clones":0,"num_object_copies":642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":321,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16991,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3755},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20364","reported_seq":20511,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.357823+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:19.357823+0000","last_peered":"2024-09-15T21:30:19.357823+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:19.357823+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1254342,"num_objects":308,"num_object_clones":0,"num_object_copies":616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16675,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3853},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20524","reported_seq":20663,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.337835+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:19.337835+0000","last_peered":"2024-09-15T21:30:19.337835+0000","last_clean":"2024-09-15T21:30:19.337835+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:19.337835+0000","last_undegraded":"2024-09-15T21:30:19.337835+0000","last_fullsized":"2024-09-15T21:30:19.337835+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1409024,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17296,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20215","reported_seq":20350,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.595480+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:19.595480+0000","last_peered":"2024-09-15T21:30:19.595480+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:19.595480+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1221091,"num_objects":299,"num_object_clones":0,"num_object_copies":598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18147,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3788}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20129","reported_seq":20270,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.273603+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:19.273603+0000","last_peered":"2024-09-15T21:30:19.273603+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:19.273603+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1445888,"num_objects":353,"num_object_clones":0,"num_object_copies":706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16802,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3667}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966943+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966943+0000","last_peered":"2024-09-15T21:30:12.966943+0000","last_clean":"2024-09-15T21:30:12.966943+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966943+0000","last_undegraded":"2024-09-15T21:30:12.966943+0000","last_fullsized":"2024-09-15T21:30:12.966943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20330","reported_seq":20459,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.680106+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:19.680106+0000","last_peered":"2024-09-15T21:30:19.680106+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:19.680106+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1282531,"num_objects":314,"num_object_clones":0,"num_object_copies":628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18189,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3776}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20295","reported_seq":28764,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.665738+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:19.665738+0000","last_peered":"2024-09-15T21:30:19.665738+0000","last_clean":"2024-09-15T21:30:19.665738+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:19.665738+0000","last_undegraded":"2024-09-15T21:30:19.665738+0000","last_fullsized":"2024-09-15T21:30:19.665738+0000","mapping_epoch":44,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1085440,"num_objects":265,"num_object_clones":0,"num_object_copies":530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17988,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20351","reported_seq":20491,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.235040+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:19.235040+0000","last_peered":"2024-09-15T21:30:19.235040+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:19.235040+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1372160,"num_objects":335,"num_object_clones":0,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17295,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20253","reported_seq":20843,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.357166+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:19.357166+0000","last_peered":"2024-09-15T21:30:19.357166+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:19.357166+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1380352,"num_objects":337,"num_object_clones":0,"num_object_copies":674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3919,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17162,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3582}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19954","reported_seq":28274,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:16.168985+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:16.168985+0000","last_peered":"2024-09-15T21:30:16.168985+0000","last_clean":"2024-09-15T21:30:16.168985+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:16.168985+0000","last_undegraded":"2024-09-15T21:30:16.168985+0000","last_fullsized":"2024-09-15T21:30:16.168985+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1777664,"num_objects":434,"num_object_clones":0,"num_object_copies":868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16438,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20534","reported_seq":20683,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.229225+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:19.229225+0000","last_peered":"2024-09-15T21:30:19.229225+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:19.229225+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1360838,"num_objects":334,"num_object_clones":0,"num_object_copies":668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17238,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3813},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":21713654,"num_objects":5311,"num_object_clones":0,"num_object_copies":10622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5311,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":285144,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12740,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249069568,"data_stored":248934969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160851,"ondisk_log_size":160851,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506908,"kb_used_data":142020,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93864932,"statfs":{"total":96636764160,"available":96117690368,"internally_reserved":0,"allocated":145428480,"data_stored":145133561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556804,"kb_used_data":83696,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93815036,"statfs":{"total":96636764160,"available":96066596864,"internally_reserved":0,"allocated":85704704,"data_stored":85409780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484437675},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511556,"kb_used_data":20328,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93860284,"statfs":{"total":96636764160,"available":96112930816,"internally_reserved":0,"allocated":20815872,"data_stored":20498811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":84078592,"data_stored":84042462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20373504,"data_stored":20315237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":144617472,"data_stored":144577270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:21.674 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:22.261 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:22.263 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:22.295 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-09-15T21:30:20.876622+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":21713673,"num_objects":5313,"num_object_clones":0,"num_object_copies":10626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5313,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":285146,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12741,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160854,"ondisk_log_size":160854,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1575268,"kb_used_data":246044,"kb_used_omap":12,"kb_used_meta":1329139,"kb_avail":281540252,"statfs":{"total":289910292480,"available":288297218048,"internally_reserved":0,"allocated":251949056,"data_stored":251042152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1361038339},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,6,7,9,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17424384,"num_objects":-4254,"num_object_clones":0,"num_object_copies":-8508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4254,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.937854"},"pg_stats":[{"pgid":"2.f","version":"49'20839","reported_seq":20923,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.234192+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:19.234192+0000","last_peered":"2024-09-15T21:30:19.234192+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:19.234192+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1282048,"num_objects":313,"num_object_clones":0,"num_object_copies":626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20839,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3819}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20377","reported_seq":20493,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.691169+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:19.691169+0000","last_peered":"2024-09-15T21:30:19.691169+0000","last_clean":"2024-09-15T21:30:19.691169+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:19.691169+0000","last_undegraded":"2024-09-15T21:30:19.691169+0000","last_fullsized":"2024-09-15T21:30:19.691169+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1217478,"num_objects":299,"num_object_clones":0,"num_object_copies":598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":299,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18658,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20689","reported_seq":29372,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:16.261333+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:16.261333+0000","last_peered":"2024-09-15T21:30:16.261333+0000","last_clean":"2024-09-15T21:30:16.261333+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:16.261333+0000","last_undegraded":"2024-09-15T21:30:16.261333+0000","last_fullsized":"2024-09-15T21:30:16.261333+0000","mapping_epoch":48,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1703960,"num_objects":417,"num_object_clones":0,"num_object_copies":834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":417,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18789,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19992","reported_seq":20072,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.699145+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:19.699145+0000","last_peered":"2024-09-15T21:30:19.699145+0000","last_clean":"2024-09-15T21:30:19.699145+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:19.699145+0000","last_undegraded":"2024-09-15T21:30:19.699145+0000","last_fullsized":"2024-09-15T21:30:19.699145+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1318912,"num_objects":322,"num_object_clones":0,"num_object_copies":644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19992,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20190","reported_seq":20333,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.312454+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:19.312454+0000","last_peered":"2024-09-15T21:30:19.312454+0000","last_clean":"2024-09-15T21:30:19.312454+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:19.312454+0000","last_undegraded":"2024-09-15T21:30:19.312454+0000","last_fullsized":"2024-09-15T21:30:19.312454+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1290723,"num_objects":316,"num_object_clones":0,"num_object_copies":632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16645,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20315","reported_seq":20462,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.329955+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:19.329955+0000","last_peered":"2024-09-15T21:30:19.329955+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:19.329955+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1311203,"num_objects":321,"num_object_clones":0,"num_object_copies":642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":321,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16991,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3755},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20364","reported_seq":20511,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.357823+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:19.357823+0000","last_peered":"2024-09-15T21:30:19.357823+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:19.357823+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1254342,"num_objects":308,"num_object_clones":0,"num_object_copies":616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16675,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3853},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20524","reported_seq":20663,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.337835+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:19.337835+0000","last_peered":"2024-09-15T21:30:19.337835+0000","last_clean":"2024-09-15T21:30:19.337835+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:19.337835+0000","last_undegraded":"2024-09-15T21:30:19.337835+0000","last_fullsized":"2024-09-15T21:30:19.337835+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1409024,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17296,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20215","reported_seq":20350,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.595480+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:19.595480+0000","last_peered":"2024-09-15T21:30:19.595480+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:19.595480+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1221091,"num_objects":299,"num_object_clones":0,"num_object_copies":598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18147,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3788}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20129","reported_seq":20270,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.273603+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:19.273603+0000","last_peered":"2024-09-15T21:30:19.273603+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:19.273603+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1445888,"num_objects":353,"num_object_clones":0,"num_object_copies":706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16802,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3667}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966943+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966943+0000","last_peered":"2024-09-15T21:30:12.966943+0000","last_clean":"2024-09-15T21:30:12.966943+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966943+0000","last_undegraded":"2024-09-15T21:30:12.966943+0000","last_fullsized":"2024-09-15T21:30:12.966943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20330","reported_seq":20459,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.680106+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:19.680106+0000","last_peered":"2024-09-15T21:30:19.680106+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:19.680106+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1282531,"num_objects":314,"num_object_clones":0,"num_object_copies":628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18189,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3776}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20295","reported_seq":28764,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.665738+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:19.665738+0000","last_peered":"2024-09-15T21:30:19.665738+0000","last_clean":"2024-09-15T21:30:19.665738+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:19.665738+0000","last_undegraded":"2024-09-15T21:30:19.665738+0000","last_fullsized":"2024-09-15T21:30:19.665738+0000","mapping_epoch":44,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1085440,"num_objects":265,"num_object_clones":0,"num_object_copies":530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17988,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20351","reported_seq":20491,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.235040+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:19.235040+0000","last_peered":"2024-09-15T21:30:19.235040+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:19.235040+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1372160,"num_objects":335,"num_object_clones":0,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17295,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20253","reported_seq":20843,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.357166+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:19.357166+0000","last_peered":"2024-09-15T21:30:19.357166+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:19.357166+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1380352,"num_objects":337,"num_object_clones":0,"num_object_copies":674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3919,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17162,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3582}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19954","reported_seq":28274,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:16.168985+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:16.168985+0000","last_peered":"2024-09-15T21:30:16.168985+0000","last_clean":"2024-09-15T21:30:16.168985+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:16.168985+0000","last_undegraded":"2024-09-15T21:30:16.168985+0000","last_fullsized":"2024-09-15T21:30:16.168985+0000","mapping_epoch":46,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1777664,"num_objects":434,"num_object_clones":0,"num_object_copies":868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":434,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16438,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20534","reported_seq":20683,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.229225+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:19.229225+0000","last_peered":"2024-09-15T21:30:19.229225+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:19.229225+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1360838,"num_objects":334,"num_object_clones":0,"num_object_copies":668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17238,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3813},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":21713654,"num_objects":5311,"num_object_clones":0,"num_object_copies":10622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5311,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":285144,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12740,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":249069568,"data_stored":248934969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160851,"ondisk_log_size":160851,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506908,"kb_used_data":142020,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93864932,"statfs":{"total":96636764160,"available":96117690368,"internally_reserved":0,"allocated":145428480,"data_stored":145133561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556804,"kb_used_data":83696,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93815036,"statfs":{"total":96636764160,"available":96066596864,"internally_reserved":0,"allocated":85704704,"data_stored":85409780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484437675},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":511556,"kb_used_data":20328,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93860284,"statfs":{"total":96636764160,"available":96112930816,"internally_reserved":0,"allocated":20815872,"data_stored":20498811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":84078592,"data_stored":84042462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20373504,"data_stored":20315237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":144617472,"data_stored":144577270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:22.296 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:30:22.296 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20839", 'reported_seq': 20923, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:19.234192+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:19.234192+0000', 'last_peered': '2024-09-15T21:30:19.234192+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:19.234192+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1282048, 'num_objects': 313, 'num_object_clones': 0, 'num_object_copies': 626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 313, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20839, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3819}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:22.296 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:30:22.297 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20315", 'reported_seq': 20462, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:19.329955+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:19.329955+0000', 'last_peered': '2024-09-15T21:30:19.329955+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:19.329955+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1311203, 'num_objects': 321, 'num_object_clones': 0, 'num_object_copies': 642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 321, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16991, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3755}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:22.297 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:30:22.297 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20364", 'reported_seq': 20511, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:19.357823+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:19.357823+0000', 'last_peered': '2024-09-15T21:30:19.357823+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:19.357823+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1254342, 'num_objects': 308, 'num_object_clones': 0, 'num_object_copies': 616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 308, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16675, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3853}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:22.297 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:30:22.297 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20215", 'reported_seq': 20350, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:19.595480+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:30:19.595480+0000', 'last_peered': '2024-09-15T21:30:19.595480+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:30:19.595480+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1221091, 'num_objects': 299, 'num_object_clones': 0, 'num_object_copies': 598, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 299, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18147, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3788}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:22.297 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:22.297 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20129", 'reported_seq': 20270, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:19.273603+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:19.273603+0000', 'last_peered': '2024-09-15T21:30:19.273603+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:19.273603+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1445888, 'num_objects': 353, 'num_object_clones': 0, 'num_object_copies': 706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 353, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16802, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3667}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:22.297 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:30:22.297 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20330", 'reported_seq': 20459, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:19.680106+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:19.680106+0000', 'last_peered': '2024-09-15T21:30:19.680106+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:19.680106+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1282531, 'num_objects': 314, 'num_object_clones': 0, 'num_object_copies': 628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 314, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18189, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3776}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:22.297 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:30:22.298 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20351", 'reported_seq': 20491, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:19.235040+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:19.235040+0000', 'last_peered': '2024-09-15T21:30:19.235040+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:19.235040+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1372160, 'num_objects': 335, 'num_object_clones': 0, 'num_object_copies': 670, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 335, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17295, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3759}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:22.298 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:30:22.298 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "49'20253", 'reported_seq': 20843, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:19.357166+0000', 'last_change': '2024-09-15T21:30:11.380283+0000', 'last_active': '2024-09-15T21:30:19.357166+0000', 'last_peered': '2024-09-15T21:30:19.357166+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:30:19.357166+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1380352, 'num_objects': 337, 'num_object_clones': 0, 'num_object_copies': 674, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3919, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 337, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17162, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 223, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3582}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:22.298 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:30:22.298 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20534", 'reported_seq': 20683, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:19.229225+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:19.229225+0000', 'last_peered': '2024-09-15T21:30:19.229225+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:19.229225+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1360838, 'num_objects': 334, 'num_object_clones': 0, 'num_object_copies': 668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 334, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17238, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3813}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:22.298 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:22.992 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:22.992 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36734/10626 objects degraded (345.699%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":674,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5313,"data_bytes":21713673,"bytes_used":1613074432,"bytes_avail":288297218048,"bytes_total":289910292480,"degraded_objects":36734,"degraded_total":10626,"degraded_ratio":3.4569922830792397,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":289,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":475},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:23.011 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:23.011 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:23.574 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:23.598 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-09-15T21:30:22.877258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":20443913,"num_objects":5003,"num_object_clones":0,"num_object_copies":10006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5003,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":285456,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12741,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160764,"ondisk_log_size":160764,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1559596,"kb_used_data":238704,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281555924,"statfs":{"total":289910292480,"available":288313266176,"internally_reserved":0,"allocated":244432896,"data_stored":243525992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1352518659},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,6,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18694144,"num_objects":-4564,"num_object_clones":0,"num_object_copies":-9128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4564,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.004491"},"pg_stats":[{"pgid":"2.f","version":"49'20839","reported_seq":20923,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.234192+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:19.234192+0000","last_peered":"2024-09-15T21:30:19.234192+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:19.234192+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1282048,"num_objects":313,"num_object_clones":0,"num_object_copies":626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20839,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3819}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20377","reported_seq":20493,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.691169+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:19.691169+0000","last_peered":"2024-09-15T21:30:19.691169+0000","last_clean":"2024-09-15T21:30:19.691169+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:19.691169+0000","last_undegraded":"2024-09-15T21:30:19.691169+0000","last_fullsized":"2024-09-15T21:30:19.691169+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1217478,"num_objects":299,"num_object_clones":0,"num_object_copies":598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":299,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18658,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20842","reported_seq":29525,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:21.368671+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:21.368671+0000","last_peered":"2024-09-15T21:30:21.368671+0000","last_clean":"2024-09-15T21:30:21.368671+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:21.368671+0000","last_undegraded":"2024-09-15T21:30:21.368671+0000","last_fullsized":"2024-09-15T21:30:21.368671+0000","mapping_epoch":48,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1077272,"num_objects":264,"num_object_clones":0,"num_object_copies":528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":264,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18942,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'19992","reported_seq":20072,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.699145+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:19.699145+0000","last_peered":"2024-09-15T21:30:19.699145+0000","last_clean":"2024-09-15T21:30:19.699145+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:19.699145+0000","last_undegraded":"2024-09-15T21:30:19.699145+0000","last_fullsized":"2024-09-15T21:30:19.699145+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1318912,"num_objects":322,"num_object_clones":0,"num_object_copies":644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19992,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20190","reported_seq":20333,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.312454+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:19.312454+0000","last_peered":"2024-09-15T21:30:19.312454+0000","last_clean":"2024-09-15T21:30:19.312454+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:19.312454+0000","last_undegraded":"2024-09-15T21:30:19.312454+0000","last_fullsized":"2024-09-15T21:30:19.312454+0000","mapping_epoch":34,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1290723,"num_objects":316,"num_object_clones":0,"num_object_copies":632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16645,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20315","reported_seq":20462,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.329955+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:19.329955+0000","last_peered":"2024-09-15T21:30:19.329955+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:19.329955+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1311203,"num_objects":321,"num_object_clones":0,"num_object_copies":642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":321,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16991,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3755},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20364","reported_seq":20511,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.357823+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:19.357823+0000","last_peered":"2024-09-15T21:30:19.357823+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:19.357823+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1254342,"num_objects":308,"num_object_clones":0,"num_object_copies":616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16675,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3853},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20524","reported_seq":20663,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.337835+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:19.337835+0000","last_peered":"2024-09-15T21:30:19.337835+0000","last_clean":"2024-09-15T21:30:19.337835+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:19.337835+0000","last_undegraded":"2024-09-15T21:30:19.337835+0000","last_fullsized":"2024-09-15T21:30:19.337835+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1409024,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17296,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20215","reported_seq":20350,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.595480+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:19.595480+0000","last_peered":"2024-09-15T21:30:19.595480+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:19.595480+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1221091,"num_objects":299,"num_object_clones":0,"num_object_copies":598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18147,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3788}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20129","reported_seq":20270,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.273603+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:19.273603+0000","last_peered":"2024-09-15T21:30:19.273603+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:19.273603+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1445888,"num_objects":353,"num_object_clones":0,"num_object_copies":706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16802,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3667}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966943+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966943+0000","last_peered":"2024-09-15T21:30:12.966943+0000","last_clean":"2024-09-15T21:30:12.966943+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966943+0000","last_undegraded":"2024-09-15T21:30:12.966943+0000","last_fullsized":"2024-09-15T21:30:12.966943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20330","reported_seq":20459,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.680106+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:19.680106+0000","last_peered":"2024-09-15T21:30:19.680106+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:19.680106+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1282531,"num_objects":314,"num_object_clones":0,"num_object_copies":628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":314,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18189,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3776}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20295","reported_seq":28764,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:19.665738+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:19.665738+0000","last_peered":"2024-09-15T21:30:19.665738+0000","last_clean":"2024-09-15T21:30:19.665738+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:19.665738+0000","last_undegraded":"2024-09-15T21:30:19.665738+0000","last_fullsized":"2024-09-15T21:30:19.665738+0000","mapping_epoch":44,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1085440,"num_objects":265,"num_object_clones":0,"num_object_copies":530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17988,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20351","reported_seq":20491,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.235040+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:19.235040+0000","last_peered":"2024-09-15T21:30:19.235040+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:19.235040+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1372160,"num_objects":335,"num_object_clones":0,"num_object_copies":670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17295,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20253","reported_seq":20843,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.357166+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:19.357166+0000","last_peered":"2024-09-15T21:30:19.357166+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:19.357166+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1380352,"num_objects":337,"num_object_clones":0,"num_object_copies":674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3919,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17162,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3582}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20111","reported_seq":28431,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:21.203182+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:21.203182+0000","last_peered":"2024-09-15T21:30:21.203182+0000","last_clean":"2024-09-15T21:30:21.203182+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:21.203182+0000","last_undegraded":"2024-09-15T21:30:21.203182+0000","last_fullsized":"2024-09-15T21:30:21.203182+0000","mapping_epoch":46,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1134592,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16595,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20534","reported_seq":20683,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:19.229225+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:19.229225+0000","last_peered":"2024-09-15T21:30:19.229225+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:19.229225+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1360838,"num_objects":334,"num_object_clones":0,"num_object_copies":668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":334,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17238,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3813},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":20443894,"num_objects":5001,"num_object_clones":0,"num_object_copies":10002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36734,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5001,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":285454,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12740,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":241524736,"data_stored":241390137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160761,"ondisk_log_size":160761,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506908,"kb_used_data":142020,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93864932,"statfs":{"total":96636764160,"available":96117690368,"internally_reserved":0,"allocated":145428480,"data_stored":145133561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":373616297},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":556804,"kb_used_data":83696,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93815036,"statfs":{"total":96636764160,"available":96066596864,"internally_reserved":0,"allocated":85704704,"data_stored":85409780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":484437675},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495884,"kb_used_data":12988,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93875956,"statfs":{"total":96636764160,"available":96128978944,"internally_reserved":0,"allocated":13299712,"data_stored":12982651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":84078592,"data_stored":84042462,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12828672,"data_stored":12770405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":144617472,"data_stored":144577270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:26.601 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:27.184 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:27.185 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:27.208 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-15T21:30:26.878817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10781449,"num_objects":2644,"num_object_clones":0,"num_object_copies":5288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":287815,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12838,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161023,"ondisk_log_size":161023,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521752,"kb_used_data":221668,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281593768,"statfs":{"total":289910292480,"available":288352018432,"internally_reserved":0,"allocated":226988032,"data_stored":226081128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1331219459},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,7,8,10,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20774912,"num_objects":-5072,"num_object_clones":0,"num_object_copies":-10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5072,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004186"},"pg_stats":[{"pgid":"2.f","version":"49'20986","reported_seq":21072,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.417101+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:24.417101+0000","last_peered":"2024-09-15T21:30:24.417101+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:24.417101+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":679936,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20986,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20505","reported_seq":20621,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.639235+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:24.639235+0000","last_peered":"2024-09-15T21:30:24.639235+0000","last_clean":"2024-09-15T21:30:24.639235+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:24.639235+0000","last_undegraded":"2024-09-15T21:30:24.639235+0000","last_fullsized":"2024-09-15T21:30:24.639235+0000","mapping_epoch":35,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":693190,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18786,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20994","reported_seq":29677,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:26.406712+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:26.406712+0000","last_peered":"2024-09-15T21:30:26.406712+0000","last_clean":"2024-09-15T21:30:26.406712+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:26.406712+0000","last_undegraded":"2024-09-15T21:30:26.406712+0000","last_fullsized":"2024-09-15T21:30:26.406712+0000","mapping_epoch":48,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454680,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19094,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20215,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.626644+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:24.626644+0000","last_peered":"2024-09-15T21:30:24.626644+0000","last_clean":"2024-09-15T21:30:24.626644+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:24.626644+0000","last_undegraded":"2024-09-15T21:30:24.626644+0000","last_fullsized":"2024-09-15T21:30:24.626644+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":737280,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20340","reported_seq":20483,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.378785+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:24.378785+0000","last_peered":"2024-09-15T21:30:24.378785+0000","last_clean":"2024-09-15T21:30:24.378785+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:24.378785+0000","last_undegraded":"2024-09-15T21:30:24.378785+0000","last_fullsized":"2024-09-15T21:30:24.378785+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":676323,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16795,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20474","reported_seq":20621,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.409491+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:24.409491+0000","last_peered":"2024-09-15T21:30:24.409491+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:24.409491+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":659939,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17150,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3914},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20494","reported_seq":20641,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.414000+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:24.414000+0000","last_peered":"2024-09-15T21:30:24.414000+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:24.414000+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":721862,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16805,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3983},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20690","reported_seq":20828,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.391365+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:24.391365+0000","last_peered":"2024-09-15T21:30:24.391365+0000","last_clean":"2024-09-15T21:30:24.391365+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:24.391365+0000","last_undegraded":"2024-09-15T21:30:24.391365+0000","last_fullsized":"2024-09-15T21:30:24.391365+0000","mapping_epoch":34,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":729088,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17462,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20360","reported_seq":20497,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.705079+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:24.705079+0000","last_peered":"2024-09-15T21:30:24.705079+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:24.705079+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":627171,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18292,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20292","reported_seq":20433,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.420685+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:24.420685+0000","last_peered":"2024-09-15T21:30:24.420685+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:24.420685+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":778240,"num_objects":190,"num_object_clones":0,"num_object_copies":380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16965,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3830}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966943+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966943+0000","last_peered":"2024-09-15T21:30:12.966943+0000","last_clean":"2024-09-15T21:30:12.966943+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966943+0000","last_undegraded":"2024-09-15T21:30:12.966943+0000","last_fullsized":"2024-09-15T21:30:12.966943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20488","reported_seq":20617,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.704466+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:24.704466+0000","last_peered":"2024-09-15T21:30:24.704466+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:24.704466+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":635363,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18347,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3934}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20421","reported_seq":28889,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.738990+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:24.738990+0000","last_peered":"2024-09-15T21:30:24.738990+0000","last_clean":"2024-09-15T21:30:24.738990+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:24.738990+0000","last_undegraded":"2024-09-15T21:30:24.738990+0000","last_fullsized":"2024-09-15T21:30:24.738990+0000","mapping_epoch":44,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":569344,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18114,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20503","reported_seq":20646,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.433352+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:24.433352+0000","last_peered":"2024-09-15T21:30:24.433352+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:24.433352+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17447,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3911}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20397","reported_seq":21181,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.444027+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:24.444027+0000","last_peered":"2024-09-15T21:30:24.444027+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:24.444027+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":790528,"num_objects":193,"num_object_clones":0,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3822,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20251","reported_seq":28571,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:26.249260+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:26.249260+0000","last_peered":"2024-09-15T21:30:26.249260+0000","last_clean":"2024-09-15T21:30:26.249260+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:26.249260+0000","last_undegraded":"2024-09-15T21:30:26.249260+0000","last_fullsized":"2024-09-15T21:30:26.249260+0000","mapping_epoch":46,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":561152,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16735,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20691","reported_seq":20840,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.356094+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:24.356094+0000","last_peered":"2024-09-15T21:30:24.356094+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:24.356094+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717766,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17395,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3970},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":10781430,"num_objects":2642,"num_object_clones":0,"num_object_copies":5284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2642,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":287813,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12837,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":222199808,"data_stored":222065209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161020,"ondisk_log_size":161020,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489900,"kb_used_data":137496,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93881940,"statfs":{"total":96636764160,"available":96135106560,"internally_reserved":0,"allocated":140795904,"data_stored":140500985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537424,"kb_used_data":76800,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93834416,"statfs":{"total":96636764160,"available":96086441984,"internally_reserved":0,"allocated":78643200,"data_stored":78348276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471658155},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494428,"kb_used_data":7372,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93877412,"statfs":{"total":96636764160,"available":96130469888,"internally_reserved":0,"allocated":7548928,"data_stored":7231867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498724527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":76734464,"data_stored":76698334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5652480,"data_stored":5594213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139812864,"data_stored":139772662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:27.209 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:27.766 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:27.767 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:27.793 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-15T21:30:26.878817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10781449,"num_objects":2644,"num_object_clones":0,"num_object_copies":5288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":287815,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12838,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161023,"ondisk_log_size":161023,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521752,"kb_used_data":221668,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281593768,"statfs":{"total":289910292480,"available":288352018432,"internally_reserved":0,"allocated":226988032,"data_stored":226081128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1331219459},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,7,8,10,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20774912,"num_objects":-5072,"num_object_clones":0,"num_object_copies":-10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5072,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004186"},"pg_stats":[{"pgid":"2.f","version":"49'20986","reported_seq":21072,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.417101+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:24.417101+0000","last_peered":"2024-09-15T21:30:24.417101+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:24.417101+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":679936,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20986,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20505","reported_seq":20621,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.639235+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:24.639235+0000","last_peered":"2024-09-15T21:30:24.639235+0000","last_clean":"2024-09-15T21:30:24.639235+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:24.639235+0000","last_undegraded":"2024-09-15T21:30:24.639235+0000","last_fullsized":"2024-09-15T21:30:24.639235+0000","mapping_epoch":35,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":693190,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18786,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20994","reported_seq":29677,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:26.406712+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:26.406712+0000","last_peered":"2024-09-15T21:30:26.406712+0000","last_clean":"2024-09-15T21:30:26.406712+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:26.406712+0000","last_undegraded":"2024-09-15T21:30:26.406712+0000","last_fullsized":"2024-09-15T21:30:26.406712+0000","mapping_epoch":48,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454680,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19094,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20215,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.626644+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:24.626644+0000","last_peered":"2024-09-15T21:30:24.626644+0000","last_clean":"2024-09-15T21:30:24.626644+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:24.626644+0000","last_undegraded":"2024-09-15T21:30:24.626644+0000","last_fullsized":"2024-09-15T21:30:24.626644+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":737280,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20340","reported_seq":20483,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.378785+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:24.378785+0000","last_peered":"2024-09-15T21:30:24.378785+0000","last_clean":"2024-09-15T21:30:24.378785+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:24.378785+0000","last_undegraded":"2024-09-15T21:30:24.378785+0000","last_fullsized":"2024-09-15T21:30:24.378785+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":676323,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16795,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20474","reported_seq":20621,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.409491+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:24.409491+0000","last_peered":"2024-09-15T21:30:24.409491+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:24.409491+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":659939,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17150,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3914},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20494","reported_seq":20641,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.414000+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:24.414000+0000","last_peered":"2024-09-15T21:30:24.414000+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:24.414000+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":721862,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16805,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3983},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20690","reported_seq":20828,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.391365+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:24.391365+0000","last_peered":"2024-09-15T21:30:24.391365+0000","last_clean":"2024-09-15T21:30:24.391365+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:24.391365+0000","last_undegraded":"2024-09-15T21:30:24.391365+0000","last_fullsized":"2024-09-15T21:30:24.391365+0000","mapping_epoch":34,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":729088,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17462,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20360","reported_seq":20497,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.705079+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:24.705079+0000","last_peered":"2024-09-15T21:30:24.705079+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:24.705079+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":627171,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18292,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20292","reported_seq":20433,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.420685+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:24.420685+0000","last_peered":"2024-09-15T21:30:24.420685+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:24.420685+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":778240,"num_objects":190,"num_object_clones":0,"num_object_copies":380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16965,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3830}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966943+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966943+0000","last_peered":"2024-09-15T21:30:12.966943+0000","last_clean":"2024-09-15T21:30:12.966943+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966943+0000","last_undegraded":"2024-09-15T21:30:12.966943+0000","last_fullsized":"2024-09-15T21:30:12.966943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20488","reported_seq":20617,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.704466+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:24.704466+0000","last_peered":"2024-09-15T21:30:24.704466+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:24.704466+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":635363,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18347,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3934}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20421","reported_seq":28889,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.738990+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:24.738990+0000","last_peered":"2024-09-15T21:30:24.738990+0000","last_clean":"2024-09-15T21:30:24.738990+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:24.738990+0000","last_undegraded":"2024-09-15T21:30:24.738990+0000","last_fullsized":"2024-09-15T21:30:24.738990+0000","mapping_epoch":44,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":569344,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18114,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20503","reported_seq":20646,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.433352+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:24.433352+0000","last_peered":"2024-09-15T21:30:24.433352+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:24.433352+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17447,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3911}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20397","reported_seq":21181,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.444027+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:24.444027+0000","last_peered":"2024-09-15T21:30:24.444027+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:24.444027+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":790528,"num_objects":193,"num_object_clones":0,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3822,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20251","reported_seq":28571,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:26.249260+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:26.249260+0000","last_peered":"2024-09-15T21:30:26.249260+0000","last_clean":"2024-09-15T21:30:26.249260+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:26.249260+0000","last_undegraded":"2024-09-15T21:30:26.249260+0000","last_fullsized":"2024-09-15T21:30:26.249260+0000","mapping_epoch":46,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":561152,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16735,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20691","reported_seq":20840,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.356094+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:24.356094+0000","last_peered":"2024-09-15T21:30:24.356094+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:24.356094+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717766,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17395,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3970},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":10781430,"num_objects":2642,"num_object_clones":0,"num_object_copies":5284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2642,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":287813,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12837,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":222199808,"data_stored":222065209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161020,"ondisk_log_size":161020,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489900,"kb_used_data":137496,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93881940,"statfs":{"total":96636764160,"available":96135106560,"internally_reserved":0,"allocated":140795904,"data_stored":140500985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537424,"kb_used_data":76800,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93834416,"statfs":{"total":96636764160,"available":96086441984,"internally_reserved":0,"allocated":78643200,"data_stored":78348276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471658155},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494428,"kb_used_data":7372,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93877412,"statfs":{"total":96636764160,"available":96130469888,"internally_reserved":0,"allocated":7548928,"data_stored":7231867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498724527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":76734464,"data_stored":76698334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5652480,"data_stored":5594213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139812864,"data_stored":139772662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:27.794 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:30:27.794 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'20986", 'reported_seq': 21072, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:24.417101+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:24.417101+0000', 'last_peered': '2024-09-15T21:30:24.417101+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:24.417101+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'10900", 'ondisk_log_start': "45'10900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 679936, 'num_objects': 166, 'num_object_clones': 0, 'num_object_copies': 332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 166, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20986, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3966}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:27.794 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:30:27.794 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20474", 'reported_seq': 20621, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:24.409491+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:24.409491+0000', 'last_peered': '2024-09-15T21:30:24.409491+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:24.409491+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 659939, 'num_objects': 162, 'num_object_clones': 0, 'num_object_copies': 324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 162, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17150, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3914}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:27.794 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:30:27.795 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20494", 'reported_seq': 20641, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:24.414000+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:24.414000+0000', 'last_peered': '2024-09-15T21:30:24.414000+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:24.414000+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 721862, 'num_objects': 178, 'num_object_clones': 0, 'num_object_copies': 356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16805, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3983}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:27.795 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:30:27.795 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20360", 'reported_seq': 20497, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:24.705079+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:30:24.705079+0000', 'last_peered': '2024-09-15T21:30:24.705079+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:30:24.705079+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 627171, 'num_objects': 154, 'num_object_clones': 0, 'num_object_copies': 308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 154, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18292, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3933}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:27.795 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:27.795 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20292", 'reported_seq': 20433, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:24.420685+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:24.420685+0000', 'last_peered': '2024-09-15T21:30:24.420685+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:24.420685+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 778240, 'num_objects': 190, 'num_object_clones': 0, 'num_object_copies': 380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 190, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16965, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3830}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:27.795 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:30:27.795 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20488", 'reported_seq': 20617, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:24.704466+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:24.704466+0000', 'last_peered': '2024-09-15T21:30:24.704466+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:24.704466+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 635363, 'num_objects': 156, 'num_object_clones': 0, 'num_object_copies': 312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 156, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18347, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -3934}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:27.795 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:30:27.795 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20503", 'reported_seq': 20646, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:24.433352+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:24.433352+0000', 'last_peered': '2024-09-15T21:30:24.433352+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:24.433352+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 749568, 'num_objects': 183, 'num_object_clones': 0, 'num_object_copies': 366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 183, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17447, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3911}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:27.795 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:30:27.795 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "49'20397", 'reported_seq': 21181, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:24.444027+0000', 'last_change': '2024-09-15T21:30:11.380283+0000', 'last_active': '2024-09-15T21:30:24.444027+0000', 'last_peered': '2024-09-15T21:30:24.444027+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:30:24.444027+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'10300", 'ondisk_log_start': "45'10300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 790528, 'num_objects': 193, 'num_object_clones': 0, 'num_object_copies': 386, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3822, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 193, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17306, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 320, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3629}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:27.796 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:30:27.796 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20691", 'reported_seq': 20840, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:24.356094+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:24.356094+0000', 'last_peered': '2024-09-15T21:30:24.356094+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:24.356094+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 717766, 'num_objects': 177, 'num_object_clones': 0, 'num_object_copies': 354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 177, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17395, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3970}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:27.796 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:28.476 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:28.476 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36637/5288 objects degraded (692.833%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":680,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":2644,"data_bytes":10781449,"bytes_used":1558274048,"bytes_avail":288352018432,"bytes_total":289910292480,"degraded_objects":36637,"degraded_total":5288,"degraded_ratio":6.9283282904689862,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":422},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:28.504 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:28.504 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:29.059 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:29.060 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:29.081 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-15T21:30:28.879649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":10781449,"num_objects":2644,"num_object_clones":0,"num_object_copies":5288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":287815,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12838,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161023,"ondisk_log_size":161023,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521752,"kb_used_data":221668,"kb_used_omap":12,"kb_used_meta":1300019,"kb_avail":281593768,"statfs":{"total":289910292480,"available":288352018432,"internally_reserved":0,"allocated":226988032,"data_stored":226081128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1331219459},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,3,7,8,10,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19517440,"num_objects":-4765,"num_object_clones":0,"num_object_copies":-9530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4765,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004374"},"pg_stats":[{"pgid":"2.f","version":"49'20986","reported_seq":21072,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.417101+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:24.417101+0000","last_peered":"2024-09-15T21:30:24.417101+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:24.417101+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":679936,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20986,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3966}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20505","reported_seq":20621,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.639235+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:24.639235+0000","last_peered":"2024-09-15T21:30:24.639235+0000","last_clean":"2024-09-15T21:30:24.639235+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:24.639235+0000","last_undegraded":"2024-09-15T21:30:24.639235+0000","last_fullsized":"2024-09-15T21:30:24.639235+0000","mapping_epoch":35,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":693190,"num_objects":171,"num_object_clones":0,"num_object_copies":342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":171,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18786,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20994","reported_seq":29677,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:26.406712+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:26.406712+0000","last_peered":"2024-09-15T21:30:26.406712+0000","last_clean":"2024-09-15T21:30:26.406712+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:26.406712+0000","last_undegraded":"2024-09-15T21:30:26.406712+0000","last_fullsized":"2024-09-15T21:30:26.406712+0000","mapping_epoch":48,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454680,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19094,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20215,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.626644+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:24.626644+0000","last_peered":"2024-09-15T21:30:24.626644+0000","last_clean":"2024-09-15T21:30:24.626644+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:24.626644+0000","last_undegraded":"2024-09-15T21:30:24.626644+0000","last_fullsized":"2024-09-15T21:30:24.626644+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":737280,"num_objects":180,"num_object_clones":0,"num_object_copies":360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20340","reported_seq":20483,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.378785+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:24.378785+0000","last_peered":"2024-09-15T21:30:24.378785+0000","last_clean":"2024-09-15T21:30:24.378785+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:24.378785+0000","last_undegraded":"2024-09-15T21:30:24.378785+0000","last_fullsized":"2024-09-15T21:30:24.378785+0000","mapping_epoch":34,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":676323,"num_objects":166,"num_object_clones":0,"num_object_copies":332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16795,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20474","reported_seq":20621,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.409491+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:24.409491+0000","last_peered":"2024-09-15T21:30:24.409491+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:24.409491+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":659939,"num_objects":162,"num_object_clones":0,"num_object_copies":324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":162,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17150,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3914},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20494","reported_seq":20641,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.414000+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:24.414000+0000","last_peered":"2024-09-15T21:30:24.414000+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:24.414000+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":721862,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16805,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3983},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20690","reported_seq":20828,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.391365+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:24.391365+0000","last_peered":"2024-09-15T21:30:24.391365+0000","last_clean":"2024-09-15T21:30:24.391365+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:24.391365+0000","last_undegraded":"2024-09-15T21:30:24.391365+0000","last_fullsized":"2024-09-15T21:30:24.391365+0000","mapping_epoch":34,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":729088,"num_objects":178,"num_object_clones":0,"num_object_copies":356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17462,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20360","reported_seq":20497,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.705079+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:24.705079+0000","last_peered":"2024-09-15T21:30:24.705079+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:24.705079+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":627171,"num_objects":154,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":154,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18292,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3933}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20292","reported_seq":20433,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.420685+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:24.420685+0000","last_peered":"2024-09-15T21:30:24.420685+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:24.420685+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":778240,"num_objects":190,"num_object_clones":0,"num_object_copies":380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16965,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3830}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966943+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966943+0000","last_peered":"2024-09-15T21:30:12.966943+0000","last_clean":"2024-09-15T21:30:12.966943+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966943+0000","last_undegraded":"2024-09-15T21:30:12.966943+0000","last_fullsized":"2024-09-15T21:30:12.966943+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20488","reported_seq":20617,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.704466+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:24.704466+0000","last_peered":"2024-09-15T21:30:24.704466+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:24.704466+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":635363,"num_objects":156,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":156,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18347,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-3934}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20421","reported_seq":28889,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:24.738990+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:24.738990+0000","last_peered":"2024-09-15T21:30:24.738990+0000","last_clean":"2024-09-15T21:30:24.738990+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:24.738990+0000","last_undegraded":"2024-09-15T21:30:24.738990+0000","last_fullsized":"2024-09-15T21:30:24.738990+0000","mapping_epoch":44,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":569344,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18114,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20503","reported_seq":20646,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.433352+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:24.433352+0000","last_peered":"2024-09-15T21:30:24.433352+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:24.433352+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17447,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3911}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20397","reported_seq":21181,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.444027+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:24.444027+0000","last_peered":"2024-09-15T21:30:24.444027+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:24.444027+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":790528,"num_objects":193,"num_object_clones":0,"num_object_copies":386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3822,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17306,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3629}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20251","reported_seq":28571,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:26.249260+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:26.249260+0000","last_peered":"2024-09-15T21:30:26.249260+0000","last_clean":"2024-09-15T21:30:26.249260+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:26.249260+0000","last_undegraded":"2024-09-15T21:30:26.249260+0000","last_fullsized":"2024-09-15T21:30:26.249260+0000","mapping_epoch":46,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":561152,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16735,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20691","reported_seq":20840,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:24.356094+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:24.356094+0000","last_peered":"2024-09-15T21:30:24.356094+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:24.356094+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717766,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17395,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3970},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":10781430,"num_objects":2642,"num_object_clones":0,"num_object_copies":5284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2642,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":287813,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12837,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":222199808,"data_stored":222065209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161020,"ondisk_log_size":161020,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489900,"kb_used_data":137496,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93881940,"statfs":{"total":96636764160,"available":96135106560,"internally_reserved":0,"allocated":140795904,"data_stored":140500985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":360836777},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537424,"kb_used_data":76800,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93834416,"statfs":{"total":96636764160,"available":96086441984,"internally_reserved":0,"allocated":78643200,"data_stored":78348276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":471658155},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,3,3,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494428,"kb_used_data":7372,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93877412,"statfs":{"total":96636764160,"available":96130469888,"internally_reserved":0,"allocated":7548928,"data_stored":7231867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498724527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":76734464,"data_stored":76698334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5652480,"data_stored":5594213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139812864,"data_stored":139772662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:30.358 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Removed 165284 objects 2024-09-15T21:30:30.358 INFO:tasks.lost_unfound.radosbench.client.admin.smithi073.stdout:Clean up completed and total clean up time :358.575 2024-09-15T21:30:32.084 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:32.388 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:32.388 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:32.402 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-15T21:30:30.880282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2413321,"num_objects":601,"num_object_clones":0,"num_object_copies":1202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":601,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":289858,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12932,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521120,"kb_used_data":208552,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281594400,"statfs":{"total":289910292480,"available":288352665600,"internally_reserved":0,"allocated":213557248,"data_stored":212650344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1343998979},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,5,3,8,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27885568,"num_objects":-6808,"num_object_clones":0,"num_object_copies":-13616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-275,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6808,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":275,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004215"},"pg_stats":[{"pgid":"2.f","version":"49'21133","reported_seq":21217,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.506952+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:29.506952+0000","last_peered":"2024-09-15T21:30:29.506952+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:29.506952+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21133,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4113}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20649","reported_seq":20765,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.725898+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:29.725898+0000","last_peered":"2024-09-15T21:30:29.725898+0000","last_clean":"2024-09-15T21:30:29.725898+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:29.725898+0000","last_undegraded":"2024-09-15T21:30:29.725898+0000","last_fullsized":"2024-09-15T21:30:29.725898+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103366,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18930,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20994","reported_seq":29677,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:26.406712+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:26.406712+0000","last_peered":"2024-09-15T21:30:26.406712+0000","last_clean":"2024-09-15T21:30:26.406712+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:26.406712+0000","last_undegraded":"2024-09-15T21:30:26.406712+0000","last_fullsized":"2024-09-15T21:30:26.406712+0000","mapping_epoch":48,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454680,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19094,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20288","reported_seq":20369,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.663280+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:29.663280+0000","last_peered":"2024-09-15T21:30:29.663280+0000","last_clean":"2024-09-15T21:30:29.663280+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:29.663280+0000","last_undegraded":"2024-09-15T21:30:29.663280+0000","last_fullsized":"2024-09-15T21:30:29.663280+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106496,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20288,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20474","reported_seq":20617,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.481393+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:29.481393+0000","last_peered":"2024-09-15T21:30:29.481393+0000","last_clean":"2024-09-15T21:30:29.481393+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:29.481393+0000","last_undegraded":"2024-09-15T21:30:29.481393+0000","last_fullsized":"2024-09-15T21:30:29.481393+0000","mapping_epoch":34,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127459,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16929,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20611","reported_seq":20758,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.399533+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:29.399533+0000","last_peered":"2024-09-15T21:30:29.399533+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:29.399533+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98787,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17287,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4051},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20656","reported_seq":20803,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.469037+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:29.469037+0000","last_peered":"2024-09-15T21:30:29.469037+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:29.469037+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58310,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16967,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20851","reported_seq":20990,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.489985+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:29.489985+0000","last_peered":"2024-09-15T21:30:29.489985+0000","last_clean":"2024-09-15T21:30:29.489985+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:29.489985+0000","last_undegraded":"2024-09-15T21:30:29.489985+0000","last_fullsized":"2024-09-15T21:30:29.489985+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69632,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17623,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20500","reported_seq":20637,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.733576+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:29.733576+0000","last_peered":"2024-09-15T21:30:29.733576+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:29.733576+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53731,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18432,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20450","reported_seq":20592,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.437561+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:29.437561+0000","last_peered":"2024-09-15T21:30:29.437561+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:29.437561+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3988}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20619","reported_seq":20748,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.700983+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:29.700983+0000","last_peered":"2024-09-15T21:30:29.700983+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:29.700983+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98787,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18478,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4065}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20544","reported_seq":29013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.684297+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:29.684297+0000","last_peered":"2024-09-15T21:30:29.684297+0000","last_clean":"2024-09-15T21:30:29.684297+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:29.684297+0000","last_undegraded":"2024-09-15T21:30:29.684297+0000","last_fullsized":"2024-09-15T21:30:29.684297+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18237,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20651","reported_seq":20795,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.459212+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:29.459212+0000","last_peered":"2024-09-15T21:30:29.459212+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:29.459212+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143360,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17595,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4059}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20554","reported_seq":21526,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.507097+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:29.507097+0000","last_peered":"2024-09-15T21:30:29.507097+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:29.507097+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147456,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3728,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17463,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3692}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20251","reported_seq":28571,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:26.249260+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:26.249260+0000","last_peered":"2024-09-15T21:30:26.249260+0000","last_clean":"2024-09-15T21:30:26.249260+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:26.249260+0000","last_undegraded":"2024-09-15T21:30:26.249260+0000","last_fullsized":"2024-09-15T21:30:26.249260+0000","mapping_epoch":46,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":561152,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16735,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20838","reported_seq":20984,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.495234+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:29.495234+0000","last_peered":"2024-09-15T21:30:29.495234+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:29.495234+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115654,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17542,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":2413302,"num_objects":599,"num_object_clones":0,"num_object_copies":1198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":599,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":289856,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12931,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210165760,"data_stored":210031161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489528,"kb_used_data":132960,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93882312,"statfs":{"total":96636764160,"available":96135487488,"internally_reserved":0,"allocated":136151040,"data_stored":135856121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537164,"kb_used_data":68220,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93834676,"statfs":{"total":96636764160,"available":96086708224,"internally_reserved":0,"allocated":69857280,"data_stored":69562356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494428,"kb_used_data":7372,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93877412,"statfs":{"total":96636764160,"available":96130469888,"internally_reserved":0,"allocated":7548928,"data_stored":7231867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498724527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69328896,"data_stored":69292766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5652480,"data_stored":5594213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135184384,"data_stored":135144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:32.403 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:32.709 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:32.710 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:32.722 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-15T21:30:30.880282+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2413321,"num_objects":601,"num_object_clones":0,"num_object_copies":1202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":601,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":289858,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12932,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521120,"kb_used_data":208552,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281594400,"statfs":{"total":289910292480,"available":288352665600,"internally_reserved":0,"allocated":213557248,"data_stored":212650344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1343998979},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,5,3,8,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27885568,"num_objects":-6808,"num_object_clones":0,"num_object_copies":-13616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-275,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6808,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":275,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004215"},"pg_stats":[{"pgid":"2.f","version":"49'21133","reported_seq":21217,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.506952+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:29.506952+0000","last_peered":"2024-09-15T21:30:29.506952+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:29.506952+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21133,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4113}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20649","reported_seq":20765,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.725898+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:29.725898+0000","last_peered":"2024-09-15T21:30:29.725898+0000","last_clean":"2024-09-15T21:30:29.725898+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:29.725898+0000","last_undegraded":"2024-09-15T21:30:29.725898+0000","last_fullsized":"2024-09-15T21:30:29.725898+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103366,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18930,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20994","reported_seq":29677,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:26.406712+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:26.406712+0000","last_peered":"2024-09-15T21:30:26.406712+0000","last_clean":"2024-09-15T21:30:26.406712+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:26.406712+0000","last_undegraded":"2024-09-15T21:30:26.406712+0000","last_fullsized":"2024-09-15T21:30:26.406712+0000","mapping_epoch":48,"log_start":"45'10900","ondisk_log_start":"45'10900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":454680,"num_objects":112,"num_object_clones":0,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19094,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20288","reported_seq":20369,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.663280+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:29.663280+0000","last_peered":"2024-09-15T21:30:29.663280+0000","last_clean":"2024-09-15T21:30:29.663280+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:29.663280+0000","last_undegraded":"2024-09-15T21:30:29.663280+0000","last_fullsized":"2024-09-15T21:30:29.663280+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106496,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20288,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20474","reported_seq":20617,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.481393+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:29.481393+0000","last_peered":"2024-09-15T21:30:29.481393+0000","last_clean":"2024-09-15T21:30:29.481393+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:29.481393+0000","last_undegraded":"2024-09-15T21:30:29.481393+0000","last_fullsized":"2024-09-15T21:30:29.481393+0000","mapping_epoch":34,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127459,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16929,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20611","reported_seq":20758,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.399533+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:29.399533+0000","last_peered":"2024-09-15T21:30:29.399533+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:29.399533+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98787,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17287,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4051},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20656","reported_seq":20803,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.469037+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:29.469037+0000","last_peered":"2024-09-15T21:30:29.469037+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:29.469037+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58310,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16967,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20851","reported_seq":20990,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.489985+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:29.489985+0000","last_peered":"2024-09-15T21:30:29.489985+0000","last_clean":"2024-09-15T21:30:29.489985+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:29.489985+0000","last_undegraded":"2024-09-15T21:30:29.489985+0000","last_fullsized":"2024-09-15T21:30:29.489985+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69632,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17623,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20500","reported_seq":20637,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.733576+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:29.733576+0000","last_peered":"2024-09-15T21:30:29.733576+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:29.733576+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53731,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18432,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20450","reported_seq":20592,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.437561+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:29.437561+0000","last_peered":"2024-09-15T21:30:29.437561+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:29.437561+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3988}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20619","reported_seq":20748,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.700983+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:29.700983+0000","last_peered":"2024-09-15T21:30:29.700983+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:29.700983+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98787,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18478,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4065}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20544","reported_seq":29013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.684297+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:29.684297+0000","last_peered":"2024-09-15T21:30:29.684297+0000","last_clean":"2024-09-15T21:30:29.684297+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:29.684297+0000","last_undegraded":"2024-09-15T21:30:29.684297+0000","last_fullsized":"2024-09-15T21:30:29.684297+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18237,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20651","reported_seq":20795,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.459212+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:29.459212+0000","last_peered":"2024-09-15T21:30:29.459212+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:29.459212+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143360,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17595,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4059}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20554","reported_seq":21526,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.507097+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:29.507097+0000","last_peered":"2024-09-15T21:30:29.507097+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:29.507097+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147456,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3728,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17463,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3692}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20251","reported_seq":28571,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:26.249260+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:26.249260+0000","last_peered":"2024-09-15T21:30:26.249260+0000","last_clean":"2024-09-15T21:30:26.249260+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:26.249260+0000","last_undegraded":"2024-09-15T21:30:26.249260+0000","last_fullsized":"2024-09-15T21:30:26.249260+0000","mapping_epoch":46,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":561152,"num_objects":137,"num_object_clones":0,"num_object_copies":274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16735,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20838","reported_seq":20984,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.495234+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:29.495234+0000","last_peered":"2024-09-15T21:30:29.495234+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:29.495234+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115654,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17542,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":2413302,"num_objects":599,"num_object_clones":0,"num_object_copies":1198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":599,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":289856,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12931,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":210165760,"data_stored":210031161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489528,"kb_used_data":132960,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93882312,"statfs":{"total":96636764160,"available":96135487488,"internally_reserved":0,"allocated":136151040,"data_stored":135856121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537164,"kb_used_data":68220,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93834676,"statfs":{"total":96636764160,"available":96086708224,"internally_reserved":0,"allocated":69857280,"data_stored":69562356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":494428,"kb_used_data":7372,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93877412,"statfs":{"total":96636764160,"available":96130469888,"internally_reserved":0,"allocated":7548928,"data_stored":7231867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498724527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69328896,"data_stored":69292766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5652480,"data_stored":5594213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135184384,"data_stored":135144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:32.723 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:30:32.723 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21133", 'reported_seq': 21217, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:29.506952+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:29.506952+0000', 'last_peered': '2024-09-15T21:30:29.506952+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:29.506952+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 77824, 'num_objects': 19, 'num_object_clones': 0, 'num_object_copies': 38, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 19, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21133, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4113}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:32.723 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:30:32.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20611", 'reported_seq': 20758, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:29.399533+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:29.399533+0000', 'last_peered': '2024-09-15T21:30:29.399533+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:29.399533+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 98787, 'num_objects': 25, 'num_object_clones': 0, 'num_object_copies': 50, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 25, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17287, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4051}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:32.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:30:32.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20656", 'reported_seq': 20803, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:29.469037+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:29.469037+0000', 'last_peered': '2024-09-15T21:30:29.469037+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:29.469037+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 58310, 'num_objects': 16, 'num_object_clones': 0, 'num_object_copies': 32, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 16, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16967, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:32.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:30:32.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20500", 'reported_seq': 20637, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:29.733576+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:30:29.733576+0000', 'last_peered': '2024-09-15T21:30:29.733576+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:30:29.733576+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 53731, 'num_objects': 14, 'num_object_clones': 0, 'num_object_copies': 28, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 14, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18432, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4073}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:32.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:32.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20450", 'reported_seq': 20592, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:29.437561+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:29.437561+0000', 'last_peered': '2024-09-15T21:30:29.437561+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:29.437561+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 131072, 'num_objects': 32, 'num_object_clones': 0, 'num_object_copies': 64, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 32, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17123, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3988}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:32.724 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:30:32.724 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20619", 'reported_seq': 20748, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:29.700983+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:29.700983+0000', 'last_peered': '2024-09-15T21:30:29.700983+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:29.700983+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 98787, 'num_objects': 25, 'num_object_clones': 0, 'num_object_copies': 50, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 25, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18478, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4065}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:32.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:30:32.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20651", 'reported_seq': 20795, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:29.459212+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:29.459212+0000', 'last_peered': '2024-09-15T21:30:29.459212+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:29.459212+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 143360, 'num_objects': 35, 'num_object_clones': 0, 'num_object_copies': 70, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 35, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17595, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4059}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:32.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:30:32.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "49'20554", 'reported_seq': 21526, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:29.507097+0000', 'last_change': '2024-09-15T21:30:11.380283+0000', 'last_active': '2024-09-15T21:30:29.507097+0000', 'last_peered': '2024-09-15T21:30:29.507097+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:30:29.507097+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 147456, 'num_objects': 36, 'num_object_clones': 0, 'num_object_copies': 72, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3728, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 36, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17463, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 414, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3692}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:32.725 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:30:32.725 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20838", 'reported_seq': 20984, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:29.495234+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:29.495234+0000', 'last_peered': '2024-09-15T21:30:29.495234+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:29.495234+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 115654, 'num_objects': 30, 'num_object_clones': 0, 'num_object_copies': 60, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 30, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17542, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4117}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:32.725 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:33.154 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:33.154 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36543/704 objects degraded (5190.767%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":684,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":352,"data_bytes":1397489,"bytes_used":1554812928,"bytes_avail":288355479552,"bytes_total":289910292480,"degraded_objects":36543,"degraded_total":704,"degraded_ratio":51.907670454545453,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":413},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:33.167 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:33.167 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:33.518 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:33.519 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:33.532 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-09-15T21:30:32.880631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1397489,"num_objects":352,"num_object_clones":0,"num_object_copies":704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":352,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290107,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12932,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518372,"kb_used_data":201644,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281597148,"statfs":{"total":289910292480,"available":288355479552,"internally_reserved":0,"allocated":206483456,"data_stored":205580624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1348258819},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20316184,"num_objects":-4961,"num_object_clones":0,"num_object_copies":-9922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-191,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4961,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004009"},"pg_stats":[{"pgid":"2.f","version":"49'21133","reported_seq":21217,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.506952+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:29.506952+0000","last_peered":"2024-09-15T21:30:29.506952+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:29.506952+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":77824,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21133,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4113}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20649","reported_seq":20765,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.725898+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:29.725898+0000","last_peered":"2024-09-15T21:30:29.725898+0000","last_clean":"2024-09-15T21:30:29.725898+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:29.725898+0000","last_undegraded":"2024-09-15T21:30:29.725898+0000","last_fullsized":"2024-09-15T21:30:29.725898+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103366,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18930,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20288","reported_seq":20369,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.663280+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:29.663280+0000","last_peered":"2024-09-15T21:30:29.663280+0000","last_clean":"2024-09-15T21:30:29.663280+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:29.663280+0000","last_undegraded":"2024-09-15T21:30:29.663280+0000","last_fullsized":"2024-09-15T21:30:29.663280+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106496,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20288,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20474","reported_seq":20617,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.481393+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:29.481393+0000","last_peered":"2024-09-15T21:30:29.481393+0000","last_clean":"2024-09-15T21:30:29.481393+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:29.481393+0000","last_undegraded":"2024-09-15T21:30:29.481393+0000","last_fullsized":"2024-09-15T21:30:29.481393+0000","mapping_epoch":34,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":127459,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16929,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20611","reported_seq":20758,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.399533+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:29.399533+0000","last_peered":"2024-09-15T21:30:29.399533+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:29.399533+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98787,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17287,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4051},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20656","reported_seq":20803,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.469037+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:29.469037+0000","last_peered":"2024-09-15T21:30:29.469037+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:29.469037+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58310,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16967,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20851","reported_seq":20990,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.489985+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:29.489985+0000","last_peered":"2024-09-15T21:30:29.489985+0000","last_clean":"2024-09-15T21:30:29.489985+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:29.489985+0000","last_undegraded":"2024-09-15T21:30:29.489985+0000","last_fullsized":"2024-09-15T21:30:29.489985+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69632,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17623,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20500","reported_seq":20637,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.733576+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:29.733576+0000","last_peered":"2024-09-15T21:30:29.733576+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:29.733576+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53731,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18432,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20450","reported_seq":20592,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.437561+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:29.437561+0000","last_peered":"2024-09-15T21:30:29.437561+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:29.437561+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131072,"num_objects":32,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17123,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3988}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20619","reported_seq":20748,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.700983+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:29.700983+0000","last_peered":"2024-09-15T21:30:29.700983+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:29.700983+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98787,"num_objects":25,"num_object_clones":0,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18478,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4065}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20544","reported_seq":29013,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:29.684297+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:29.684297+0000","last_peered":"2024-09-15T21:30:29.684297+0000","last_clean":"2024-09-15T21:30:29.684297+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:29.684297+0000","last_undegraded":"2024-09-15T21:30:29.684297+0000","last_fullsized":"2024-09-15T21:30:29.684297+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65536,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18237,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20651","reported_seq":20795,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.459212+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:29.459212+0000","last_peered":"2024-09-15T21:30:29.459212+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:29.459212+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143360,"num_objects":35,"num_object_clones":0,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17595,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4059}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20554","reported_seq":21526,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.507097+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:29.507097+0000","last_peered":"2024-09-15T21:30:29.507097+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:29.507097+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147456,"num_objects":36,"num_object_clones":0,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3728,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17463,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":414,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3692}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20838","reported_seq":20984,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:29.495234+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:29.495234+0000","last_peered":"2024-09-15T21:30:29.495234+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:29.495234+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":115654,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17542,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4117},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1397470,"num_objects":350,"num_object_clones":0,"num_object_copies":700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36543,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":350,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290105,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12931,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":204574720,"data_stored":204444193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489528,"kb_used_data":132960,"kb_used_omap":4,"kb_used_meta":356539,"kb_avail":93882312,"statfs":{"total":96636764160,"available":96135487488,"internally_reserved":0,"allocated":136151040,"data_stored":135856121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":365096617},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537164,"kb_used_data":68220,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93834676,"statfs":{"total":96636764160,"available":96086708224,"internally_reserved":0,"allocated":69857280,"data_stored":69562356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491680,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880160,"statfs":{"total":96636764160,"available":96133283840,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":69328896,"data_stored":69292766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":135184384,"data_stored":135144182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:36.534 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:36.842 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:36.854 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-15T21:30:34.880935+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13408,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520024,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281595496,"statfs":{"total":289910292480,"available":288353787904,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1352518659},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20439064,"num_objects":-4991,"num_object_clones":0,"num_object_copies":-9982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4991,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4991,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003677"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20772,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.349217+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.349217+0000","last_peered":"2024-09-15T21:30:30.349217+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.349217+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29029,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232485+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232485+0000","last_peered":"2024-09-15T21:30:30.232485+0000","last_clean":"2024-09-15T21:30:30.232485+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232485+0000","last_undegraded":"2024-09-15T21:30:30.232485+0000","last_fullsized":"2024-09-15T21:30:30.232485+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":22514,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:34.547594+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:34.547594+0000","last_peered":"2024-09-15T21:30:34.547594+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:34.547594+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3252,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":890,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3252}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21015,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.349237+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.349237+0000","last_peered":"2024-09-15T21:30:30.349237+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.349237+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13407,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491680,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880160,"statfs":{"total":96636764160,"available":96133283840,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:36.856 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:37.157 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:37.157 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:37.170 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-15T21:30:36.881295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13408,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507544,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281607976,"statfs":{"total":289910292480,"available":288366567424,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1339739139},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11972632,"num_objects":-2924,"num_object_clones":0,"num_object_copies":-5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-570,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2924,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":570,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003243"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20772,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.349217+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.349217+0000","last_peered":"2024-09-15T21:30:30.349217+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.349217+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29029,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232485+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232485+0000","last_peered":"2024-09-15T21:30:30.232485+0000","last_clean":"2024-09-15T21:30:30.232485+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232485+0000","last_undegraded":"2024-09-15T21:30:30.232485+0000","last_fullsized":"2024-09-15T21:30:30.232485+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":22514,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:34.547594+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:34.547594+0000","last_peered":"2024-09-15T21:30:34.547594+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:34.547594+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3252,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":890,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3252}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21015,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.349237+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.349237+0000","last_peered":"2024-09-15T21:30:30.349237+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.349237+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13407,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479200,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892640,"statfs":{"total":96636764160,"available":96146063360,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:37.171 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:30:37.171 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21238, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.288198+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:30.288198+0000', 'last_peered': '2024-09-15T21:30:30.288198+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:30.288198+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:37.172 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:30:37.172 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20782, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.315187+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:30.315187+0000', 'last_peered': '2024-09-15T21:30:30.315187+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:30.315187+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:37.172 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:30:37.172 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20817, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.299988+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:30.299988+0000', 'last_peered': '2024-09-15T21:30:30.299988+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:30.299988+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:37.172 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:30:37.172 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20513", 'reported_seq': 20650, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.249907+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:30:30.249907+0000', 'last_peered': '2024-09-15T21:30:30.249907+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:30:30.249907+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4086}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:37.172 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:37.172 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20624, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.333734+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:30.333734+0000', 'last_peered': '2024-09-15T21:30:30.333734+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:30.333734+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:37.172 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:30:37.172 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20772, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.349217+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:30.349217+0000', 'last_peered': '2024-09-15T21:30:30.349217+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:30.349217+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:37.173 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:30:37.173 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 20830, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.328033+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:30.328033+0000', 'last_peered': '2024-09-15T21:30:30.328033+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:30.328033+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:37.173 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:30:37.173 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "49'20590", 'reported_seq': 22514, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:34.547594+0000', 'last_change': '2024-09-15T21:30:11.380283+0000', 'last_active': '2024-09-15T21:30:34.547594+0000', 'last_peered': '2024-09-15T21:30:34.547594+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:30:34.547594+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3252, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17499, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 890, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3252}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:37.173 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:30:37.173 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21015, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.349237+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:30.349237+0000', 'last_peered': '2024-09-15T21:30:30.349237+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:30.349237+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:37.173 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:37.584 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:37.584 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36067/24 objects degraded (150279.167%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":689,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1543725056,"bytes_avail":288366567424,"bytes_total":289910292480,"degraded_objects":36067,"degraded_total":24,"degraded_ratio":1502.791666666667,"recovering_objects_per_sec":47,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":570,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":243},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:37.597 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:37.598 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:37.911 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:37.911 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:37.924 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-09-15T21:30:36.881295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13408,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507544,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281607976,"statfs":{"total":289910292480,"available":288366567424,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1339739139},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11972632,"num_objects":-2924,"num_object_clones":0,"num_object_copies":-5848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-570,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2924,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":570,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003243"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20772,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.349217+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.349217+0000","last_peered":"2024-09-15T21:30:30.349217+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.349217+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29029,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232485+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232485+0000","last_peered":"2024-09-15T21:30:30.232485+0000","last_clean":"2024-09-15T21:30:30.232485+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232485+0000","last_undegraded":"2024-09-15T21:30:30.232485+0000","last_fullsized":"2024-09-15T21:30:30.232485+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":22514,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:34.547594+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:34.547594+0000","last_peered":"2024-09-15T21:30:34.547594+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:34.547594+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3252,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":890,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3252}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21015,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.349237+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.349237+0000","last_peered":"2024-09-15T21:30:30.349237+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.349237+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36067,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13407,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479200,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892640,"statfs":{"total":96636764160,"available":96146063360,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:40.926 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:41.239 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:41.239 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:41.252 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-15T21:30:40.882080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14020,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507544,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281607976,"statfs":{"total":289910292480,"available":288366567424,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1339739139},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10776600,"num_objects":-2632,"num_object_clones":0,"num_object_copies":-5264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1182,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2632,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002431"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20772,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.349217+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.349217+0000","last_peered":"2024-09-15T21:30:30.349217+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.349217+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29029,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232485+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232485+0000","last_peered":"2024-09-15T21:30:30.232485+0000","last_clean":"2024-09-15T21:30:30.232485+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232485+0000","last_undegraded":"2024-09-15T21:30:30.232485+0000","last_fullsized":"2024-09-15T21:30:30.232485+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":23738,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:39.549728+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:39.549728+0000","last_peered":"2024-09-15T21:30:39.549728+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:39.549728+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2640,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1502,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2640}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21015,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.349237+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.349237+0000","last_peered":"2024-09-15T21:30:30.349237+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.349237+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14019,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479200,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892640,"statfs":{"total":96636764160,"available":96146063360,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:41.253 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:41.561 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:41.561 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:41.574 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-15T21:30:40.882080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14020,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507544,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281607976,"statfs":{"total":289910292480,"available":288366567424,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1339739139},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10776600,"num_objects":-2632,"num_object_clones":0,"num_object_copies":-5264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1182,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2632,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002431"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20772,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.349217+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.349217+0000","last_peered":"2024-09-15T21:30:30.349217+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.349217+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29029,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232485+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232485+0000","last_peered":"2024-09-15T21:30:30.232485+0000","last_clean":"2024-09-15T21:30:30.232485+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232485+0000","last_undegraded":"2024-09-15T21:30:30.232485+0000","last_fullsized":"2024-09-15T21:30:30.232485+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":23738,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:39.549728+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:39.549728+0000","last_peered":"2024-09-15T21:30:39.549728+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:39.549728+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2640,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1502,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2640}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21015,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.349237+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.349237+0000","last_peered":"2024-09-15T21:30:30.349237+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.349237+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14019,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479200,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892640,"statfs":{"total":96636764160,"available":96146063360,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:41.575 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:30:41.575 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21238, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.288198+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:30.288198+0000', 'last_peered': '2024-09-15T21:30:30.288198+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:30.288198+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:41.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:30:41.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20782, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.315187+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:30.315187+0000', 'last_peered': '2024-09-15T21:30:30.315187+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:30.315187+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:41.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:30:41.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20817, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.299988+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:30.299988+0000', 'last_peered': '2024-09-15T21:30:30.299988+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:30.299988+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:41.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:30:41.576 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20513", 'reported_seq': 20650, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.249907+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:30:30.249907+0000', 'last_peered': '2024-09-15T21:30:30.249907+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:30:30.249907+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4086}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:41.576 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:41.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20624, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.333734+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:30.333734+0000', 'last_peered': '2024-09-15T21:30:30.333734+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:30.333734+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:41.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:30:41.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20772, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.349217+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:30.349217+0000', 'last_peered': '2024-09-15T21:30:30.349217+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:30.349217+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:41.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:30:41.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 20830, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.328033+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:30.328033+0000', 'last_peered': '2024-09-15T21:30:30.328033+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:30.328033+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:41.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:30:41.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "49'20590", 'reported_seq': 23738, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:39.549728+0000', 'last_change': '2024-09-15T21:30:11.380283+0000', 'last_active': '2024-09-15T21:30:39.549728+0000', 'last_peered': '2024-09-15T21:30:39.549728+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:30:39.549728+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2640, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17499, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1502, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2640}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:41.577 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:30:41.577 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21015, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.349237+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:30.349237+0000', 'last_peered': '2024-09-15T21:30:30.349237+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:30.349237+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:41.578 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:41.957 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:41.958 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35455/24 objects degraded (147729.167%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":693,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1543725056,"bytes_avail":288366567424,"bytes_total":289910292480,"degraded_objects":35455,"degraded_total":24,"degraded_ratio":1477.291666666667,"recovering_objects_per_sec":98,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1182,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":219},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:41.971 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:41.972 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:42.311 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:42.312 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:42.324 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-09-15T21:30:40.882080+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14020,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507544,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1308339,"kb_avail":281607976,"statfs":{"total":289910292480,"available":288366567424,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1339739139},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10776600,"num_objects":-2632,"num_object_clones":0,"num_object_copies":-5264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1182,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2632,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002431"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958762+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958762+0000","last_peered":"2024-09-15T21:30:12.958762+0000","last_clean":"2024-09-15T21:30:12.958762+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958762+0000","last_undegraded":"2024-09-15T21:30:12.958762+0000","last_fullsized":"2024-09-15T21:30:12.958762+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20772,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.349217+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.349217+0000","last_peered":"2024-09-15T21:30:30.349217+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.349217+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29029,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232485+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232485+0000","last_peered":"2024-09-15T21:30:30.232485+0000","last_clean":"2024-09-15T21:30:30.232485+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232485+0000","last_undegraded":"2024-09-15T21:30:30.232485+0000","last_fullsized":"2024-09-15T21:30:30.232485+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":23738,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:39.549728+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:39.549728+0000","last_peered":"2024-09-15T21:30:39.549728+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:39.549728+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2640,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1502,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2640}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958976+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.958976+0000","last_peered":"2024-09-15T21:30:12.958976+0000","last_clean":"2024-09-15T21:30:12.958976+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.958976+0000","last_undegraded":"2024-09-15T21:30:12.958976+0000","last_fullsized":"2024-09-15T21:30:12.958976+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21015,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.349237+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.349237+0000","last_peered":"2024-09-15T21:30:30.349237+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.349237+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14019,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479200,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892640,"statfs":{"total":96636764160,"available":96146063360,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490204847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:45.326 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:45.630 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:45.630 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:45.642 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-09-15T21:30:44.882858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14725,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511704,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281603816,"statfs":{"total":289910292480,"available":288362307584,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1343998979},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1392640,"num_objects":-340,"num_object_clones":0,"num_object_copies":-680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1793,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":340,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1793,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002226"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958786+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958786+0000","last_peered":"2024-09-15T21:30:12.958786+0000","last_clean":"2024-09-15T21:30:12.958786+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958786+0000","last_undegraded":"2024-09-15T21:30:12.958786+0000","last_fullsized":"2024-09-15T21:30:12.958786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20773,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.356285+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.356285+0000","last_peered":"2024-09-15T21:30:30.356285+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.356285+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29029,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232485+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232485+0000","last_peered":"2024-09-15T21:30:30.232485+0000","last_clean":"2024-09-15T21:30:30.232485+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232485+0000","last_undegraded":"2024-09-15T21:30:30.232485+0000","last_fullsized":"2024-09-15T21:30:30.232485+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":25148,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:44.553684+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:44.553684+0000","last_peered":"2024-09-15T21:30:44.553684+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:44.553684+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.959012+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.959012+0000","last_peered":"2024-09-15T21:30:12.959012+0000","last_clean":"2024-09-15T21:30:12.959012+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.959012+0000","last_undegraded":"2024-09-15T21:30:12.959012+0000","last_fullsized":"2024-09-15T21:30:12.959012+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21016,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.350426+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.350426+0000","last_peered":"2024-09-15T21:30:30.350426+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.350426+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14724,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483360,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888480,"statfs":{"total":96636764160,"available":96141803520,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:45.644 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:45.951 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:45.951 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:45.964 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-09-15T21:30:44.882858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14725,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511704,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281603816,"statfs":{"total":289910292480,"available":288362307584,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1343998979},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1392640,"num_objects":-340,"num_object_clones":0,"num_object_copies":-680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1793,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":340,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1793,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002226"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958786+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958786+0000","last_peered":"2024-09-15T21:30:12.958786+0000","last_clean":"2024-09-15T21:30:12.958786+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958786+0000","last_undegraded":"2024-09-15T21:30:12.958786+0000","last_fullsized":"2024-09-15T21:30:12.958786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20773,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.356285+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.356285+0000","last_peered":"2024-09-15T21:30:30.356285+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.356285+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29029,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232485+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232485+0000","last_peered":"2024-09-15T21:30:30.232485+0000","last_clean":"2024-09-15T21:30:30.232485+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232485+0000","last_undegraded":"2024-09-15T21:30:30.232485+0000","last_fullsized":"2024-09-15T21:30:30.232485+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":25148,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:44.553684+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:44.553684+0000","last_peered":"2024-09-15T21:30:44.553684+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:44.553684+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.959012+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.959012+0000","last_peered":"2024-09-15T21:30:12.959012+0000","last_clean":"2024-09-15T21:30:12.959012+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.959012+0000","last_undegraded":"2024-09-15T21:30:12.959012+0000","last_fullsized":"2024-09-15T21:30:12.959012+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21016,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.350426+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.350426+0000","last_peered":"2024-09-15T21:30:30.350426+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.350426+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14724,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483360,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888480,"statfs":{"total":96636764160,"available":96141803520,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:45.965 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:30:45.965 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21238, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.288198+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:30.288198+0000', 'last_peered': '2024-09-15T21:30:30.288198+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:30.288198+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:45.965 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:30:45.965 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20782, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.315187+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:30.315187+0000', 'last_peered': '2024-09-15T21:30:30.315187+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:30.315187+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:45.965 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:30:45.966 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20817, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.299988+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:30.299988+0000', 'last_peered': '2024-09-15T21:30:30.299988+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:30.299988+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:45.966 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:30:45.966 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20513", 'reported_seq': 20650, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.249907+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:30:30.249907+0000', 'last_peered': '2024-09-15T21:30:30.249907+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:30:30.249907+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4086}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:45.966 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:45.966 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20624, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.333734+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:30.333734+0000', 'last_peered': '2024-09-15T21:30:30.333734+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:30.333734+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:45.966 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:30:45.966 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20773, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.356285+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:30.356285+0000', 'last_peered': '2024-09-15T21:30:30.356285+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:30.356285+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:45.966 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:30:45.966 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 20830, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.328033+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:30.328033+0000', 'last_peered': '2024-09-15T21:30:30.328033+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:30.328033+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:45.966 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:30:45.967 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "49'20590", 'reported_seq': 25148, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:44.553684+0000', 'last_change': '2024-09-15T21:30:11.380283+0000', 'last_active': '2024-09-15T21:30:44.553684+0000', 'last_peered': '2024-09-15T21:30:44.553684+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:30:44.553684+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17499, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2207, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1935}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:45.967 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:30:45.967 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21016, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.350426+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:30.350426+0000', 'last_peered': '2024-09-15T21:30:30.350426+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:30.350426+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:45.967 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:46.379 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:46.379 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34750/24 objects degraded (144791.667%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":698,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1547984896,"bytes_avail":288362307584,"bytes_total":289910292480,"degraded_objects":34750,"degraded_total":24,"degraded_ratio":1447.916666666667,"recovering_objects_per_sec":149,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1793,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":28},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:46.393 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:46.393 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:46.695 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:46.696 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:46.708 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-09-15T21:30:44.882858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14725,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511704,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281603816,"statfs":{"total":289910292480,"available":288362307584,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1343998979},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1392640,"num_objects":-340,"num_object_clones":0,"num_object_copies":-680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1793,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":340,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1793,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002226"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958786+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958786+0000","last_peered":"2024-09-15T21:30:12.958786+0000","last_clean":"2024-09-15T21:30:12.958786+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958786+0000","last_undegraded":"2024-09-15T21:30:12.958786+0000","last_fullsized":"2024-09-15T21:30:12.958786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20773,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.356285+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.356285+0000","last_peered":"2024-09-15T21:30:30.356285+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.356285+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29029,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232485+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232485+0000","last_peered":"2024-09-15T21:30:30.232485+0000","last_clean":"2024-09-15T21:30:30.232485+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232485+0000","last_undegraded":"2024-09-15T21:30:30.232485+0000","last_fullsized":"2024-09-15T21:30:30.232485+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":25148,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:44.553684+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:44.553684+0000","last_peered":"2024-09-15T21:30:44.553684+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:44.553684+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.959012+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.959012+0000","last_peered":"2024-09-15T21:30:12.959012+0000","last_clean":"2024-09-15T21:30:12.959012+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.959012+0000","last_undegraded":"2024-09-15T21:30:12.959012+0000","last_fullsized":"2024-09-15T21:30:12.959012+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21016,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.350426+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.350426+0000","last_peered":"2024-09-15T21:30:30.350426+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.350426+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14724,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483360,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888480,"statfs":{"total":96636764160,"available":96141803520,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:49.711 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:50.013 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:50.014 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:50.026 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-15T21:30:48.883657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14725,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520024,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281595496,"statfs":{"total":289910292480,"available":288353787904,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1352518659},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002361"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958786+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958786+0000","last_peered":"2024-09-15T21:30:12.958786+0000","last_clean":"2024-09-15T21:30:12.958786+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958786+0000","last_undegraded":"2024-09-15T21:30:12.958786+0000","last_fullsized":"2024-09-15T21:30:12.958786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20773,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.356285+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.356285+0000","last_peered":"2024-09-15T21:30:30.356285+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.356285+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29029,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232485+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232485+0000","last_peered":"2024-09-15T21:30:30.232485+0000","last_clean":"2024-09-15T21:30:30.232485+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232485+0000","last_undegraded":"2024-09-15T21:30:30.232485+0000","last_fullsized":"2024-09-15T21:30:30.232485+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":25148,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:44.553684+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:44.553684+0000","last_peered":"2024-09-15T21:30:44.553684+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:44.553684+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.959012+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.959012+0000","last_peered":"2024-09-15T21:30:12.959012+0000","last_clean":"2024-09-15T21:30:12.959012+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.959012+0000","last_undegraded":"2024-09-15T21:30:12.959012+0000","last_fullsized":"2024-09-15T21:30:12.959012+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21016,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.350426+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.350426+0000","last_peered":"2024-09-15T21:30:30.350426+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.350426+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14724,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491680,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880160,"statfs":{"total":96636764160,"available":96133283840,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:50.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:50.342 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:50.342 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:50.354 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-15T21:30:48.883657+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14725,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520024,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281595496,"statfs":{"total":289910292480,"available":288353787904,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1352518659},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1317,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002361"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958786+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958786+0000","last_peered":"2024-09-15T21:30:12.958786+0000","last_clean":"2024-09-15T21:30:12.958786+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958786+0000","last_undegraded":"2024-09-15T21:30:12.958786+0000","last_fullsized":"2024-09-15T21:30:12.958786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20773,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.356285+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.356285+0000","last_peered":"2024-09-15T21:30:30.356285+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.356285+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29029,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232485+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232485+0000","last_peered":"2024-09-15T21:30:30.232485+0000","last_clean":"2024-09-15T21:30:30.232485+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232485+0000","last_undegraded":"2024-09-15T21:30:30.232485+0000","last_fullsized":"2024-09-15T21:30:30.232485+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":25148,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:44.553684+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:44.553684+0000","last_peered":"2024-09-15T21:30:44.553684+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:44.553684+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2207,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.959012+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.959012+0000","last_peered":"2024-09-15T21:30:12.959012+0000","last_clean":"2024-09-15T21:30:12.959012+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.959012+0000","last_undegraded":"2024-09-15T21:30:12.959012+0000","last_fullsized":"2024-09-15T21:30:12.959012+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21016,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.350426+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.350426+0000","last_peered":"2024-09-15T21:30:30.350426+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.350426+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34750,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14724,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491680,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880160,"statfs":{"total":96636764160,"available":96133283840,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:50.355 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:30:50.355 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21238, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.288198+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:30.288198+0000', 'last_peered': '2024-09-15T21:30:30.288198+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:30.288198+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:50.355 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:30:50.355 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20782, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.315187+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:30.315187+0000', 'last_peered': '2024-09-15T21:30:30.315187+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:30.315187+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:50.356 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:30:50.356 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20817, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.299988+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:30.299988+0000', 'last_peered': '2024-09-15T21:30:30.299988+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:30.299988+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:50.356 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:30:50.356 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20513", 'reported_seq': 20650, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.249907+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:30:30.249907+0000', 'last_peered': '2024-09-15T21:30:30.249907+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:30:30.249907+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4086}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:50.356 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:50.357 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20624, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.333734+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:30.333734+0000', 'last_peered': '2024-09-15T21:30:30.333734+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:30.333734+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:50.357 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:30:50.357 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20773, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.356285+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:30.356285+0000', 'last_peered': '2024-09-15T21:30:30.356285+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:30.356285+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:50.357 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:30:50.357 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 20830, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.328033+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:30.328033+0000', 'last_peered': '2024-09-15T21:30:30.328033+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:30.328033+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:50.357 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:30:50.357 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "49'20590", 'reported_seq': 25148, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:44.553684+0000', 'last_change': '2024-09-15T21:30:11.380283+0000', 'last_active': '2024-09-15T21:30:44.553684+0000', 'last_peered': '2024-09-15T21:30:44.553684+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:30:44.553684+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17499, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2207, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1935}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:50.357 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:30:50.357 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21016, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.350426+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:30.350426+0000', 'last_peered': '2024-09-15T21:30:30.350426+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:30.350426+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:50.358 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:50.754 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:50.754 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34750/24 objects degraded (144791.667%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":702,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1556504576,"bytes_avail":288353787904,"bytes_total":289910292480,"degraded_objects":34750,"degraded_total":24,"degraded_ratio":1447.916666666667,"recovering_objects_per_sec":109,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1317,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:50.767 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:50.767 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:51.070 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:51.071 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:51.085 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-09-15T21:30:50.884011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33863,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15612,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520024,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281595496,"statfs":{"total":289910292480,"available":288353787904,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1352518659},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2204,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002322"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958786+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958786+0000","last_peered":"2024-09-15T21:30:12.958786+0000","last_clean":"2024-09-15T21:30:12.958786+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958786+0000","last_undegraded":"2024-09-15T21:30:12.958786+0000","last_fullsized":"2024-09-15T21:30:12.958786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20773,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.356285+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.356285+0000","last_peered":"2024-09-15T21:30:30.356285+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.356285+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29030,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232796+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232796+0000","last_peered":"2024-09-15T21:30:30.232796+0000","last_clean":"2024-09-15T21:30:30.232796+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232796+0000","last_undegraded":"2024-09-15T21:30:30.232796+0000","last_fullsized":"2024-09-15T21:30:30.232796+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":26922,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:49.553535+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:49.553535+0000","last_peered":"2024-09-15T21:30:49.553535+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:49.553535+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3094,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1048}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.959012+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.959012+0000","last_peered":"2024-09-15T21:30:12.959012+0000","last_clean":"2024-09-15T21:30:12.959012+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.959012+0000","last_undegraded":"2024-09-15T21:30:12.959012+0000","last_fullsized":"2024-09-15T21:30:12.959012+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21016,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.350426+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.350426+0000","last_peered":"2024-09-15T21:30:30.350426+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.350426+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33863,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15611,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491680,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880160,"statfs":{"total":96636764160,"available":96133283840,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:54.087 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:54.398 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:54.399 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:54.411 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-15T21:30:54.106719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33863,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15612,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511704,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281603816,"statfs":{"total":289910292480,"available":288362307584,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1343998979},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1592,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.224200"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958786+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958786+0000","last_peered":"2024-09-15T21:30:12.958786+0000","last_clean":"2024-09-15T21:30:12.958786+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958786+0000","last_undegraded":"2024-09-15T21:30:12.958786+0000","last_fullsized":"2024-09-15T21:30:12.958786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20773,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.356285+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.356285+0000","last_peered":"2024-09-15T21:30:30.356285+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.356285+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29030,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232796+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232796+0000","last_peered":"2024-09-15T21:30:30.232796+0000","last_clean":"2024-09-15T21:30:30.232796+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232796+0000","last_undegraded":"2024-09-15T21:30:30.232796+0000","last_fullsized":"2024-09-15T21:30:30.232796+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":26922,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:49.553535+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:49.553535+0000","last_peered":"2024-09-15T21:30:49.553535+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:49.553535+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3094,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1048}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.959012+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.959012+0000","last_peered":"2024-09-15T21:30:12.959012+0000","last_clean":"2024-09-15T21:30:12.959012+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.959012+0000","last_undegraded":"2024-09-15T21:30:12.959012+0000","last_fullsized":"2024-09-15T21:30:12.959012+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21016,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.350426+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.350426+0000","last_peered":"2024-09-15T21:30:30.350426+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.350426+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33863,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15611,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483360,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888480,"statfs":{"total":96636764160,"available":96141803520,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:54.412 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:54.711 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:54.712 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:54.724 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-09-15T21:30:54.106719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33863,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15612,"num_bytes_recovered":32628755,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511704,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281603816,"statfs":{"total":289910292480,"available":288362307584,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1343998979},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1592,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1592,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.224200"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.288198+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:30.288198+0000","last_peered":"2024-09-15T21:30:30.288198+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:30.288198+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20790,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.339001+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:30.339001+0000","last_peered":"2024-09-15T21:30:30.339001+0000","last_clean":"2024-09-15T21:30:30.339001+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:30.339001+0000","last_undegraded":"2024-09-15T21:30:30.339001+0000","last_fullsized":"2024-09-15T21:30:30.339001+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20395,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345370+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:30.345370+0000","last_peered":"2024-09-15T21:30:30.345370+0000","last_clean":"2024-09-15T21:30:30.345370+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:30.345370+0000","last_undegraded":"2024-09-15T21:30:30.345370+0000","last_fullsized":"2024-09-15T21:30:30.345370+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20648,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273906+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:30.273906+0000","last_peered":"2024-09-15T21:30:30.273906+0000","last_clean":"2024-09-15T21:30:30.273906+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:30.273906+0000","last_undegraded":"2024-09-15T21:30:30.273906+0000","last_fullsized":"2024-09-15T21:30:30.273906+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20782,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.315187+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:30.315187+0000","last_peered":"2024-09-15T21:30:30.315187+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:30.315187+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20817,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.299988+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:30.299988+0000","last_peered":"2024-09-15T21:30:30.299988+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:30.299988+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21007,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.311017+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:30.311017+0000","last_peered":"2024-09-15T21:30:30.311017+0000","last_clean":"2024-09-15T21:30:30.311017+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:30.311017+0000","last_undegraded":"2024-09-15T21:30:30.311017+0000","last_fullsized":"2024-09-15T21:30:30.311017+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20650,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.249907+0000","last_change":"2024-09-15T21:21:14.860584+0000","last_active":"2024-09-15T21:30:30.249907+0000","last_peered":"2024-09-15T21:30:30.249907+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:30.249907+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4088,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4086}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.958786+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:12.958786+0000","last_peered":"2024-09-15T21:30:12.958786+0000","last_clean":"2024-09-15T21:30:12.958786+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:12.958786+0000","last_undegraded":"2024-09-15T21:30:12.958786+0000","last_fullsized":"2024-09-15T21:30:12.958786+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20624,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.333734+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:30.333734+0000","last_peered":"2024-09-15T21:30:30.333734+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:30.333734+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966989+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:12.966989+0000","last_peered":"2024-09-15T21:30:12.966989+0000","last_clean":"2024-09-15T21:30:12.966989+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:12.966989+0000","last_undegraded":"2024-09-15T21:30:12.966989+0000","last_fullsized":"2024-09-15T21:30:12.966989+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20773,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.356285+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:30.356285+0000","last_peered":"2024-09-15T21:30:30.356285+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:30.356285+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29030,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.232796+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:30.232796+0000","last_peered":"2024-09-15T21:30:30.232796+0000","last_clean":"2024-09-15T21:30:30.232796+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:30.232796+0000","last_undegraded":"2024-09-15T21:30:30.232796+0000","last_fullsized":"2024-09-15T21:30:30.232796+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20830,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.328033+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:30.328033+0000","last_peered":"2024-09-15T21:30:30.328033+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:30.328033+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":26922,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:49.553535+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:49.553535+0000","last_peered":"2024-09-15T21:30:49.553535+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:49.553535+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3094,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1048}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.959012+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:12.959012+0000","last_peered":"2024-09-15T21:30:12.959012+0000","last_clean":"2024-09-15T21:30:12.959012+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:12.959012+0000","last_undegraded":"2024-09-15T21:30:12.959012+0000","last_fullsized":"2024-09-15T21:30:12.959012+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21016,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:30.350426+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:30.350426+0000","last_peered":"2024-09-15T21:30:30.350426+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:30.350426+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33863,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15611,"num_bytes_recovered":32628736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483360,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888480,"statfs":{"total":96636764160,"available":96141803520,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:54.726 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:30:54.726 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21238, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.288198+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:30.288198+0000', 'last_peered': '2024-09-15T21:30:30.288198+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:30.288198+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:54.726 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:30:54.726 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20782, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.315187+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:30.315187+0000', 'last_peered': '2024-09-15T21:30:30.315187+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:30.315187+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:54.726 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:30:54.726 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20817, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.299988+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:30.299988+0000', 'last_peered': '2024-09-15T21:30:30.299988+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:30.299988+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:54.726 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:30:54.727 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20513", 'reported_seq': 20650, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.249907+0000', 'last_change': '2024-09-15T21:21:14.860584+0000', 'last_active': '2024-09-15T21:30:30.249907+0000', 'last_peered': '2024-09-15T21:30:30.249907+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:30:30.249907+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4088, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4086}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:54.727 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:54.727 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20624, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.333734+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:30.333734+0000', 'last_peered': '2024-09-15T21:30:30.333734+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:30.333734+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:54.727 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:30:54.727 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20773, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.356285+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:30.356285+0000', 'last_peered': '2024-09-15T21:30:30.356285+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:30.356285+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:54.727 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:30:54.727 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 20830, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.328033+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:30.328033+0000', 'last_peered': '2024-09-15T21:30:30.328033+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:30.328033+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:54.727 INFO:tasks.lost_unfound.ceph_manager:PG 2.6 is not active+clean 2024-09-15T21:30:54.727 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.6', 'version': "49'20590", 'reported_seq': 26922, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:49.553535+0000', 'last_change': '2024-09-15T21:30:11.380283+0000', 'last_active': '2024-09-15T21:30:49.553535+0000', 'last_peered': '2024-09-15T21:30:49.553535+0000', 'last_clean': '2024-09-15T21:21:12.322612+0000', 'last_became_active': '2024-09-15T21:21:14.844323+0000', 'last_became_peered': '2024-09-15T21:21:14.844323+0000', 'last_unstale': '2024-09-15T21:30:49.553535+0000', 'last_undegraded': '2024-09-15T21:21:14.457597+0000', 'last_fullsized': '2024-09-15T21:21:14.356378+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3128", 'last_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:56.700158+0000', 'objects_scrubbed': 3083, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 875, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1048, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17499, 'num_write_kb': 28832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3094, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1048}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:54.727 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:30:54.728 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21016, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:30.350426+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:30.350426+0000', 'last_peered': '2024-09-15T21:30:30.350426+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:30.350426+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:54.728 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:55.206 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:55.207 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33777/24 objects degraded (140737.500%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":706,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":2}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1547984896,"bytes_avail":288362307584,"bytes_total":289910292480,"degraded_objects":33777,"degraded_total":24,"degraded_ratio":1407.375,"recovering_objects_per_sec":97,"recovering_bytes_per_sec":48,"recovering_keys_per_sec":0,"num_objects_recovered":973,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:55.223 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:55.223 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:55.548 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:55.549 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:55.561 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-09-15T21:30:55.114802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33777,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15698,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511704,"kb_used_data":199144,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281603816,"statfs":{"total":289910292480,"available":288362307584,"internally_reserved":0,"allocated":203923456,"data_stored":203024696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1343998979},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":973,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.231573"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21240,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.112009+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:54.112009+0000","last_peered":"2024-09-15T21:30:54.112009+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:54.112009+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20792,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108957+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:54.108957+0000","last_peered":"2024-09-15T21:30:54.108957+0000","last_clean":"2024-09-15T21:30:54.108957+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:54.108957+0000","last_undegraded":"2024-09-15T21:30:54.108957+0000","last_fullsized":"2024-09-15T21:30:54.108957+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29789,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.345995+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:30.345995+0000","last_peered":"2024-09-15T21:30:30.345995+0000","last_clean":"2024-09-15T21:30:30.345995+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:30.345995+0000","last_undegraded":"2024-09-15T21:30:30.345995+0000","last_fullsized":"2024-09-15T21:30:30.345995+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20397,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108431+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:54.108431+0000","last_peered":"2024-09-15T21:30:54.108431+0000","last_clean":"2024-09-15T21:30:54.108431+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:54.108431+0000","last_undegraded":"2024-09-15T21:30:54.108431+0000","last_fullsized":"2024-09-15T21:30:54.108431+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20650,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.110807+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:54.110807+0000","last_peered":"2024-09-15T21:30:54.110807+0000","last_clean":"2024-09-15T21:30:54.110807+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:54.110807+0000","last_undegraded":"2024-09-15T21:30:54.110807+0000","last_fullsized":"2024-09-15T21:30:54.110807+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20784,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.108936+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:54.108936+0000","last_peered":"2024-09-15T21:30:54.108936+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:54.108936+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20819,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.108241+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:54.108241+0000","last_peered":"2024-09-15T21:30:54.108241+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:54.108241+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21010,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108877+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:54.108877+0000","last_peered":"2024-09-15T21:30:54.108877+0000","last_clean":"2024-09-15T21:30:54.108877+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:54.108877+0000","last_undegraded":"2024-09-15T21:30:54.108877+0000","last_fullsized":"2024-09-15T21:30:54.108877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964687+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:12.964687+0000","last_peered":"2024-09-15T21:30:12.964687+0000","last_clean":"2024-09-15T21:30:12.964687+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:12.964687+0000","last_undegraded":"2024-09-15T21:30:12.964687+0000","last_fullsized":"2024-09-15T21:30:12.964687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20826,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.748874+0000","last_change":"2024-09-15T21:30:53.869994+0000","last_active":"2024-09-15T21:30:54.748874+0000","last_peered":"2024-09-15T21:30:54.748874+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:54.748874+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4001}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.109115+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:54.109115+0000","last_peered":"2024-09-15T21:30:54.109115+0000","last_clean":"2024-09-15T21:30:54.109115+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:54.109115+0000","last_undegraded":"2024-09-15T21:30:54.109115+0000","last_fullsized":"2024-09-15T21:30:54.109115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20626,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.109515+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:54.109515+0000","last_peered":"2024-09-15T21:30:54.109515+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:54.109515+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108833+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:54.108833+0000","last_peered":"2024-09-15T21:30:54.108833+0000","last_clean":"2024-09-15T21:30:54.108833+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:54.108833+0000","last_undegraded":"2024-09-15T21:30:54.108833+0000","last_fullsized":"2024-09-15T21:30:54.108833+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20775,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.109209+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:54.109209+0000","last_peered":"2024-09-15T21:30:54.109209+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:54.109209+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.964884+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:12.964884+0000","last_peered":"2024-09-15T21:30:12.964884+0000","last_clean":"2024-09-15T21:30:12.964884+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:12.964884+0000","last_undegraded":"2024-09-15T21:30:12.964884+0000","last_fullsized":"2024-09-15T21:30:12.964884+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29032,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108055+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:54.108055+0000","last_peered":"2024-09-15T21:30:54.108055+0000","last_clean":"2024-09-15T21:30:54.108055+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:54.108055+0000","last_undegraded":"2024-09-15T21:30:54.108055+0000","last_fullsized":"2024-09-15T21:30:54.108055+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20832,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.112009+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:54.112009+0000","last_peered":"2024-09-15T21:30:54.112009+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:54.112009+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965057+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:12.965057+0000","last_peered":"2024-09-15T21:30:12.965057+0000","last_clean":"2024-09-15T21:30:12.965057+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:12.965057+0000","last_undegraded":"2024-09-15T21:30:12.965057+0000","last_fullsized":"2024-09-15T21:30:12.965057+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":26922,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:49.553535+0000","last_change":"2024-09-15T21:30:11.380283+0000","last_active":"2024-09-15T21:30:49.553535+0000","last_peered":"2024-09-15T21:30:49.553535+0000","last_clean":"2024-09-15T21:21:12.322612+0000","last_became_active":"2024-09-15T21:21:14.844323+0000","last_became_peered":"2024-09-15T21:21:14.844323+0000","last_unstale":"2024-09-15T21:30:49.553535+0000","last_undegraded":"2024-09-15T21:21:14.457597+0000","last_fullsized":"2024-09-15T21:21:14.356378+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1048,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3094,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1048}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108332+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:54.108332+0000","last_peered":"2024-09-15T21:30:54.108332+0000","last_clean":"2024-09-15T21:30:54.108332+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:54.108332+0000","last_undegraded":"2024-09-15T21:30:54.108332+0000","last_fullsized":"2024-09-15T21:30:54.108332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28708,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:30.273750+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:30.273750+0000","last_peered":"2024-09-15T21:30:30.273750+0000","last_clean":"2024-09-15T21:30:30.273750+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:30.273750+0000","last_undegraded":"2024-09-15T21:30:30.273750+0000","last_fullsized":"2024-09-15T21:30:30.273750+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.966049+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:12.966049+0000","last_peered":"2024-09-15T21:30:12.966049+0000","last_clean":"2024-09-15T21:30:12.966049+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:12.966049+0000","last_undegraded":"2024-09-15T21:30:12.966049+0000","last_fullsized":"2024-09-15T21:30:12.966049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-15T21:30:12.965133+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:12.965133+0000","last_peered":"2024-09-15T21:30:12.965133+0000","last_clean":"2024-09-15T21:30:12.965133+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:12.965133+0000","last_undegraded":"2024-09-15T21:30:12.965133+0000","last_fullsized":"2024-09-15T21:30:12.965133+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21018,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.110902+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:54.110902+0000","last_peered":"2024-09-15T21:30:54.110902+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:54.110902+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33777,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15697,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202686464,"data_stored":202560009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483360,"kb_used_data":464,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888480,"statfs":{"total":96636764160,"available":96141803520,"internally_reserved":0,"allocated":475136,"data_stored":162147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":7245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:58.564 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:58.912 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:58.913 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:58.925 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-15T21:30:58.885707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32729,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16746,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520040,"kb_used_data":199156,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281595480,"statfs":{"total":289910292480,"available":288353771520,"internally_reserved":0,"allocated":203935744,"data_stored":203030886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1352518659},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1134,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001695"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21240,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.112009+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:54.112009+0000","last_peered":"2024-09-15T21:30:54.112009+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:54.112009+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20792,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108957+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:54.108957+0000","last_peered":"2024-09-15T21:30:54.108957+0000","last_clean":"2024-09-15T21:30:54.108957+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:54.108957+0000","last_undegraded":"2024-09-15T21:30:54.108957+0000","last_fullsized":"2024-09-15T21:30:54.108957+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29793,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120815+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:55.120815+0000","last_peered":"2024-09-15T21:30:55.120815+0000","last_clean":"2024-09-15T21:30:55.120815+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:55.120815+0000","last_undegraded":"2024-09-15T21:30:55.120815+0000","last_fullsized":"2024-09-15T21:30:55.120815+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20397,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108431+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:54.108431+0000","last_peered":"2024-09-15T21:30:54.108431+0000","last_clean":"2024-09-15T21:30:54.108431+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:54.108431+0000","last_undegraded":"2024-09-15T21:30:54.108431+0000","last_fullsized":"2024-09-15T21:30:54.108431+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20650,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.110807+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:54.110807+0000","last_peered":"2024-09-15T21:30:54.110807+0000","last_clean":"2024-09-15T21:30:54.110807+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:54.110807+0000","last_undegraded":"2024-09-15T21:30:54.110807+0000","last_fullsized":"2024-09-15T21:30:54.110807+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20784,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.108936+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:54.108936+0000","last_peered":"2024-09-15T21:30:54.108936+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:54.108936+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20819,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.108241+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:54.108241+0000","last_peered":"2024-09-15T21:30:54.108241+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:54.108241+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21010,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108877+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:54.108877+0000","last_peered":"2024-09-15T21:30:54.108877+0000","last_clean":"2024-09-15T21:30:54.108877+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:54.108877+0000","last_undegraded":"2024-09-15T21:30:54.108877+0000","last_fullsized":"2024-09-15T21:30:54.108877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120265+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:55.120265+0000","last_peered":"2024-09-15T21:30:55.120265+0000","last_clean":"2024-09-15T21:30:55.120265+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:55.120265+0000","last_undegraded":"2024-09-15T21:30:55.120265+0000","last_fullsized":"2024-09-15T21:30:55.120265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20826,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.748874+0000","last_change":"2024-09-15T21:30:53.869994+0000","last_active":"2024-09-15T21:30:54.748874+0000","last_peered":"2024-09-15T21:30:54.748874+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:54.748874+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4001}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.109115+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:54.109115+0000","last_peered":"2024-09-15T21:30:54.109115+0000","last_clean":"2024-09-15T21:30:54.109115+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:54.109115+0000","last_undegraded":"2024-09-15T21:30:54.109115+0000","last_fullsized":"2024-09-15T21:30:54.109115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20626,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.109515+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:54.109515+0000","last_peered":"2024-09-15T21:30:54.109515+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:54.109515+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108833+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:54.108833+0000","last_peered":"2024-09-15T21:30:54.108833+0000","last_clean":"2024-09-15T21:30:54.108833+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:54.108833+0000","last_undegraded":"2024-09-15T21:30:54.108833+0000","last_fullsized":"2024-09-15T21:30:54.108833+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20775,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.109209+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:54.109209+0000","last_peered":"2024-09-15T21:30:54.109209+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:54.109209+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:55.120588+0000","last_peered":"2024-09-15T21:30:55.120588+0000","last_clean":"2024-09-15T21:30:55.120588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:55.120588+0000","last_undegraded":"2024-09-15T21:30:55.120588+0000","last_fullsized":"2024-09-15T21:30:55.120588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29032,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108055+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:54.108055+0000","last_peered":"2024-09-15T21:30:54.108055+0000","last_clean":"2024-09-15T21:30:54.108055+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:54.108055+0000","last_undegraded":"2024-09-15T21:30:54.108055+0000","last_fullsized":"2024-09-15T21:30:54.108055+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20832,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.112009+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:54.112009+0000","last_peered":"2024-09-15T21:30:54.112009+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:54.112009+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120705+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:55.120705+0000","last_peered":"2024-09-15T21:30:55.120705+0000","last_clean":"2024-09-15T21:30:55.120705+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:55.120705+0000","last_undegraded":"2024-09-15T21:30:55.120705+0000","last_fullsized":"2024-09-15T21:30:55.120705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21597,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.215742+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:30:55.215742+0000","last_peered":"2024-09-15T21:30:55.215742+0000","last_clean":"2024-09-15T21:30:55.215742+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:30:55.215742+0000","last_undegraded":"2024-09-15T21:30:55.215742+0000","last_fullsized":"2024-09-15T21:30:55.215742+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108332+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:54.108332+0000","last_peered":"2024-09-15T21:30:54.108332+0000","last_clean":"2024-09-15T21:30:54.108332+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:54.108332+0000","last_undegraded":"2024-09-15T21:30:54.108332+0000","last_fullsized":"2024-09-15T21:30:54.108332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28712,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120847+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:55.120847+0000","last_peered":"2024-09-15T21:30:55.120847+0000","last_clean":"2024-09-15T21:30:55.120847+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:55.120847+0000","last_undegraded":"2024-09-15T21:30:55.120847+0000","last_fullsized":"2024-09-15T21:30:55.120847+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120641+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:55.120641+0000","last_peered":"2024-09-15T21:30:55.120641+0000","last_clean":"2024-09-15T21:30:55.120641+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:55.120641+0000","last_undegraded":"2024-09-15T21:30:55.120641+0000","last_fullsized":"2024-09-15T21:30:55.120641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120915+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:55.120915+0000","last_peered":"2024-09-15T21:30:55.120915+0000","last_clean":"2024-09-15T21:30:55.120915+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:55.120915+0000","last_undegraded":"2024-09-15T21:30:55.120915+0000","last_fullsized":"2024-09-15T21:30:55.120915+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21018,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.110902+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:54.110902+0000","last_peered":"2024-09-15T21:30:54.110902+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:54.110902+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32729,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16745,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202682368,"data_stored":202559526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491696,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880144,"statfs":{"total":96636764160,"available":96133267456,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:58.927 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:30:59.308 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:59.309 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:30:59.321 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-15T21:30:58.885707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32729,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16746,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520040,"kb_used_data":199156,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281595480,"statfs":{"total":289910292480,"available":288353771520,"internally_reserved":0,"allocated":203935744,"data_stored":203030886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1352518659},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1134,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001695"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21240,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.112009+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:54.112009+0000","last_peered":"2024-09-15T21:30:54.112009+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:54.112009+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20792,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108957+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:54.108957+0000","last_peered":"2024-09-15T21:30:54.108957+0000","last_clean":"2024-09-15T21:30:54.108957+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:54.108957+0000","last_undegraded":"2024-09-15T21:30:54.108957+0000","last_fullsized":"2024-09-15T21:30:54.108957+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29793,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120815+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:55.120815+0000","last_peered":"2024-09-15T21:30:55.120815+0000","last_clean":"2024-09-15T21:30:55.120815+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:55.120815+0000","last_undegraded":"2024-09-15T21:30:55.120815+0000","last_fullsized":"2024-09-15T21:30:55.120815+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20397,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108431+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:54.108431+0000","last_peered":"2024-09-15T21:30:54.108431+0000","last_clean":"2024-09-15T21:30:54.108431+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:54.108431+0000","last_undegraded":"2024-09-15T21:30:54.108431+0000","last_fullsized":"2024-09-15T21:30:54.108431+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20650,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.110807+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:54.110807+0000","last_peered":"2024-09-15T21:30:54.110807+0000","last_clean":"2024-09-15T21:30:54.110807+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:54.110807+0000","last_undegraded":"2024-09-15T21:30:54.110807+0000","last_fullsized":"2024-09-15T21:30:54.110807+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20784,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.108936+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:54.108936+0000","last_peered":"2024-09-15T21:30:54.108936+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:54.108936+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20819,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.108241+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:54.108241+0000","last_peered":"2024-09-15T21:30:54.108241+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:54.108241+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21010,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108877+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:54.108877+0000","last_peered":"2024-09-15T21:30:54.108877+0000","last_clean":"2024-09-15T21:30:54.108877+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:54.108877+0000","last_undegraded":"2024-09-15T21:30:54.108877+0000","last_fullsized":"2024-09-15T21:30:54.108877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120265+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:55.120265+0000","last_peered":"2024-09-15T21:30:55.120265+0000","last_clean":"2024-09-15T21:30:55.120265+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:55.120265+0000","last_undegraded":"2024-09-15T21:30:55.120265+0000","last_fullsized":"2024-09-15T21:30:55.120265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20826,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.748874+0000","last_change":"2024-09-15T21:30:53.869994+0000","last_active":"2024-09-15T21:30:54.748874+0000","last_peered":"2024-09-15T21:30:54.748874+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:54.748874+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4001}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.109115+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:54.109115+0000","last_peered":"2024-09-15T21:30:54.109115+0000","last_clean":"2024-09-15T21:30:54.109115+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:54.109115+0000","last_undegraded":"2024-09-15T21:30:54.109115+0000","last_fullsized":"2024-09-15T21:30:54.109115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20626,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.109515+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:54.109515+0000","last_peered":"2024-09-15T21:30:54.109515+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:54.109515+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108833+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:54.108833+0000","last_peered":"2024-09-15T21:30:54.108833+0000","last_clean":"2024-09-15T21:30:54.108833+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:54.108833+0000","last_undegraded":"2024-09-15T21:30:54.108833+0000","last_fullsized":"2024-09-15T21:30:54.108833+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20775,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.109209+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:54.109209+0000","last_peered":"2024-09-15T21:30:54.109209+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:54.109209+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:55.120588+0000","last_peered":"2024-09-15T21:30:55.120588+0000","last_clean":"2024-09-15T21:30:55.120588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:55.120588+0000","last_undegraded":"2024-09-15T21:30:55.120588+0000","last_fullsized":"2024-09-15T21:30:55.120588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29032,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108055+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:54.108055+0000","last_peered":"2024-09-15T21:30:54.108055+0000","last_clean":"2024-09-15T21:30:54.108055+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:54.108055+0000","last_undegraded":"2024-09-15T21:30:54.108055+0000","last_fullsized":"2024-09-15T21:30:54.108055+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20832,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.112009+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:54.112009+0000","last_peered":"2024-09-15T21:30:54.112009+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:54.112009+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120705+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:55.120705+0000","last_peered":"2024-09-15T21:30:55.120705+0000","last_clean":"2024-09-15T21:30:55.120705+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:55.120705+0000","last_undegraded":"2024-09-15T21:30:55.120705+0000","last_fullsized":"2024-09-15T21:30:55.120705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21597,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.215742+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:30:55.215742+0000","last_peered":"2024-09-15T21:30:55.215742+0000","last_clean":"2024-09-15T21:30:55.215742+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:30:55.215742+0000","last_undegraded":"2024-09-15T21:30:55.215742+0000","last_fullsized":"2024-09-15T21:30:55.215742+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108332+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:54.108332+0000","last_peered":"2024-09-15T21:30:54.108332+0000","last_clean":"2024-09-15T21:30:54.108332+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:54.108332+0000","last_undegraded":"2024-09-15T21:30:54.108332+0000","last_fullsized":"2024-09-15T21:30:54.108332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28712,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120847+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:55.120847+0000","last_peered":"2024-09-15T21:30:55.120847+0000","last_clean":"2024-09-15T21:30:55.120847+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:55.120847+0000","last_undegraded":"2024-09-15T21:30:55.120847+0000","last_fullsized":"2024-09-15T21:30:55.120847+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120641+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:55.120641+0000","last_peered":"2024-09-15T21:30:55.120641+0000","last_clean":"2024-09-15T21:30:55.120641+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:55.120641+0000","last_undegraded":"2024-09-15T21:30:55.120641+0000","last_fullsized":"2024-09-15T21:30:55.120641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120915+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:55.120915+0000","last_peered":"2024-09-15T21:30:55.120915+0000","last_clean":"2024-09-15T21:30:55.120915+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:55.120915+0000","last_undegraded":"2024-09-15T21:30:55.120915+0000","last_fullsized":"2024-09-15T21:30:55.120915+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21018,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.110902+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:54.110902+0000","last_peered":"2024-09-15T21:30:54.110902+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:54.110902+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32729,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16745,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202682368,"data_stored":202559526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491696,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880144,"statfs":{"total":96636764160,"available":96133267456,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:30:59.323 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:30:59.323 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21240, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:54.112009+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:54.112009+0000', 'last_peered': '2024-09-15T21:30:54.112009+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:54.112009+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:59.323 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:30:59.323 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20784, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:54.108936+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:54.108936+0000', 'last_peered': '2024-09-15T21:30:54.108936+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:54.108936+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:59.323 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:30:59.323 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20819, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:54.108241+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:54.108241+0000', 'last_peered': '2024-09-15T21:30:54.108241+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:54.108241+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:59.323 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:30:59.323 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20513", 'reported_seq': 20826, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:54.748874+0000', 'last_change': '2024-09-15T21:30:53.869994+0000', 'last_active': '2024-09-15T21:30:54.748874+0000', 'last_peered': '2024-09-15T21:30:54.748874+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:30:54.748874+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4002, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 86, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4001}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:59.324 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:30:59.324 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20626, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:54.109515+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:54.109515+0000', 'last_peered': '2024-09-15T21:30:54.109515+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:54.109515+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:59.324 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:30:59.324 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20775, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:54.109209+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:54.109209+0000', 'last_peered': '2024-09-15T21:30:54.109209+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:54.109209+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:30:59.324 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:30:59.324 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 20832, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:54.112009+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:54.112009+0000', 'last_peered': '2024-09-15T21:30:54.112009+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:54.112009+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:59.324 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:30:59.324 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21018, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:54.110902+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:54.110902+0000', 'last_peered': '2024-09-15T21:30:54.110902+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:54.110902+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:30:59.325 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:30:59.812 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:30:59.812 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32729/24 objects degraded (136370.833%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":711,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1556520960,"bytes_avail":288353771520,"bytes_total":289910292480,"degraded_objects":32729,"degraded_total":24,"degraded_ratio":1363.708333333333,"recovering_objects_per_sec":141,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":1134,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:30:59.829 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:30:59.829 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:00.174 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:00.174 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:00.187 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-15T21:30:58.885707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32729,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16746,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520040,"kb_used_data":199156,"kb_used_omap":12,"kb_used_meta":1320819,"kb_avail":281595480,"statfs":{"total":289910292480,"available":288353771520,"internally_reserved":0,"allocated":203935744,"data_stored":203030886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1352518659},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1134,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1134,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001695"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21240,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.112009+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:54.112009+0000","last_peered":"2024-09-15T21:30:54.112009+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:54.112009+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20792,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108957+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:54.108957+0000","last_peered":"2024-09-15T21:30:54.108957+0000","last_clean":"2024-09-15T21:30:54.108957+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:54.108957+0000","last_undegraded":"2024-09-15T21:30:54.108957+0000","last_fullsized":"2024-09-15T21:30:54.108957+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29793,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120815+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:55.120815+0000","last_peered":"2024-09-15T21:30:55.120815+0000","last_clean":"2024-09-15T21:30:55.120815+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:55.120815+0000","last_undegraded":"2024-09-15T21:30:55.120815+0000","last_fullsized":"2024-09-15T21:30:55.120815+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20397,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108431+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:54.108431+0000","last_peered":"2024-09-15T21:30:54.108431+0000","last_clean":"2024-09-15T21:30:54.108431+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:54.108431+0000","last_undegraded":"2024-09-15T21:30:54.108431+0000","last_fullsized":"2024-09-15T21:30:54.108431+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20650,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.110807+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:54.110807+0000","last_peered":"2024-09-15T21:30:54.110807+0000","last_clean":"2024-09-15T21:30:54.110807+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:54.110807+0000","last_undegraded":"2024-09-15T21:30:54.110807+0000","last_fullsized":"2024-09-15T21:30:54.110807+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20784,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.108936+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:54.108936+0000","last_peered":"2024-09-15T21:30:54.108936+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:54.108936+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20819,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.108241+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:54.108241+0000","last_peered":"2024-09-15T21:30:54.108241+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:54.108241+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21010,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108877+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:54.108877+0000","last_peered":"2024-09-15T21:30:54.108877+0000","last_clean":"2024-09-15T21:30:54.108877+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:54.108877+0000","last_undegraded":"2024-09-15T21:30:54.108877+0000","last_fullsized":"2024-09-15T21:30:54.108877+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120265+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:55.120265+0000","last_peered":"2024-09-15T21:30:55.120265+0000","last_clean":"2024-09-15T21:30:55.120265+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:55.120265+0000","last_undegraded":"2024-09-15T21:30:55.120265+0000","last_fullsized":"2024-09-15T21:30:55.120265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20826,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.748874+0000","last_change":"2024-09-15T21:30:53.869994+0000","last_active":"2024-09-15T21:30:54.748874+0000","last_peered":"2024-09-15T21:30:54.748874+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:54.748874+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4001}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.109115+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:54.109115+0000","last_peered":"2024-09-15T21:30:54.109115+0000","last_clean":"2024-09-15T21:30:54.109115+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:54.109115+0000","last_undegraded":"2024-09-15T21:30:54.109115+0000","last_fullsized":"2024-09-15T21:30:54.109115+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20626,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.109515+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:54.109515+0000","last_peered":"2024-09-15T21:30:54.109515+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:54.109515+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":114,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108833+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:54.108833+0000","last_peered":"2024-09-15T21:30:54.108833+0000","last_clean":"2024-09-15T21:30:54.108833+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:54.108833+0000","last_undegraded":"2024-09-15T21:30:54.108833+0000","last_fullsized":"2024-09-15T21:30:54.108833+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20775,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.109209+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:54.109209+0000","last_peered":"2024-09-15T21:30:54.109209+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:54.109209+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:55.120588+0000","last_peered":"2024-09-15T21:30:55.120588+0000","last_clean":"2024-09-15T21:30:55.120588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:55.120588+0000","last_undegraded":"2024-09-15T21:30:55.120588+0000","last_fullsized":"2024-09-15T21:30:55.120588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29032,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108055+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:54.108055+0000","last_peered":"2024-09-15T21:30:54.108055+0000","last_clean":"2024-09-15T21:30:54.108055+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:54.108055+0000","last_undegraded":"2024-09-15T21:30:54.108055+0000","last_fullsized":"2024-09-15T21:30:54.108055+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20832,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.112009+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:54.112009+0000","last_peered":"2024-09-15T21:30:54.112009+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:54.112009+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120705+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:55.120705+0000","last_peered":"2024-09-15T21:30:55.120705+0000","last_clean":"2024-09-15T21:30:55.120705+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:55.120705+0000","last_undegraded":"2024-09-15T21:30:55.120705+0000","last_fullsized":"2024-09-15T21:30:55.120705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21597,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.215742+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:30:55.215742+0000","last_peered":"2024-09-15T21:30:55.215742+0000","last_clean":"2024-09-15T21:30:55.215742+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:30:55.215742+0000","last_undegraded":"2024-09-15T21:30:55.215742+0000","last_fullsized":"2024-09-15T21:30:55.215742+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-09-15T21:30:54.108332+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:54.108332+0000","last_peered":"2024-09-15T21:30:54.108332+0000","last_clean":"2024-09-15T21:30:54.108332+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:54.108332+0000","last_undegraded":"2024-09-15T21:30:54.108332+0000","last_fullsized":"2024-09-15T21:30:54.108332+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28712,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120847+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:55.120847+0000","last_peered":"2024-09-15T21:30:55.120847+0000","last_clean":"2024-09-15T21:30:55.120847+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:55.120847+0000","last_undegraded":"2024-09-15T21:30:55.120847+0000","last_fullsized":"2024-09-15T21:30:55.120847+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120641+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:55.120641+0000","last_peered":"2024-09-15T21:30:55.120641+0000","last_clean":"2024-09-15T21:30:55.120641+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:55.120641+0000","last_undegraded":"2024-09-15T21:30:55.120641+0000","last_fullsized":"2024-09-15T21:30:55.120641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120915+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:55.120915+0000","last_peered":"2024-09-15T21:30:55.120915+0000","last_clean":"2024-09-15T21:30:55.120915+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:55.120915+0000","last_undegraded":"2024-09-15T21:30:55.120915+0000","last_fullsized":"2024-09-15T21:30:55.120915+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21018,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:54.110902+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:54.110902+0000","last_peered":"2024-09-15T21:30:54.110902+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:54.110902+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32729,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16745,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202682368,"data_stored":202559526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492548,"kb_used_data":131828,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879292,"statfs":{"total":96636764160,"available":96132395008,"internally_reserved":0,"allocated":134991872,"data_stored":134701025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535796,"kb_used_data":66852,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836044,"statfs":{"total":96636764160,"available":96088109056,"internally_reserved":0,"allocated":68456448,"data_stored":68161524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491696,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880144,"statfs":{"total":96636764160,"available":96133267456,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502984367},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:03.189 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:03.533 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:03.534 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:03.546 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-15T21:31:02.886365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17574,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511764,"kb_used_data":199188,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281603756,"statfs":{"total":289910292480,"available":288362246144,"internally_reserved":0,"allocated":203968512,"data_stored":203044232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1343998979},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1962,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.779645"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21242,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.120153+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:55.120153+0000","last_peered":"2024-09-15T21:30:55.120153+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:55.120153+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132782+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:55.132782+0000","last_peered":"2024-09-15T21:30:55.132782+0000","last_clean":"2024-09-15T21:30:55.132782+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:55.132782+0000","last_undegraded":"2024-09-15T21:30:55.132782+0000","last_fullsized":"2024-09-15T21:30:55.132782+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29793,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120815+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:55.120815+0000","last_peered":"2024-09-15T21:30:55.120815+0000","last_clean":"2024-09-15T21:30:55.120815+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:55.120815+0000","last_undegraded":"2024-09-15T21:30:55.120815+0000","last_fullsized":"2024-09-15T21:30:55.120815+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132287+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:55.132287+0000","last_peered":"2024-09-15T21:30:55.132287+0000","last_clean":"2024-09-15T21:30:55.132287+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:55.132287+0000","last_undegraded":"2024-09-15T21:30:55.132287+0000","last_fullsized":"2024-09-15T21:30:55.132287+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20652,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118351+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:55.118351+0000","last_peered":"2024-09-15T21:30:55.118351+0000","last_clean":"2024-09-15T21:30:55.118351+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:55.118351+0000","last_undegraded":"2024-09-15T21:30:55.118351+0000","last_fullsized":"2024-09-15T21:30:55.118351+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20786,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118093+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:55.118093+0000","last_peered":"2024-09-15T21:30:55.118093+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:55.118093+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118034+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:55.118034+0000","last_peered":"2024-09-15T21:30:55.118034+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:55.118034+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21012,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.117918+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:55.117918+0000","last_peered":"2024-09-15T21:30:55.117918+0000","last_clean":"2024-09-15T21:30:55.117918+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:55.117918+0000","last_undegraded":"2024-09-15T21:30:55.117918+0000","last_fullsized":"2024-09-15T21:30:55.117918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120265+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:55.120265+0000","last_peered":"2024-09-15T21:30:55.120265+0000","last_clean":"2024-09-15T21:30:55.120265+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:55.120265+0000","last_undegraded":"2024-09-15T21:30:55.120265+0000","last_fullsized":"2024-09-15T21:30:55.120265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":22484,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:59.759131+0000","last_change":"2024-09-15T21:30:53.869994+0000","last_active":"2024-09-15T21:30:59.759131+0000","last_peered":"2024-09-15T21:30:59.759131+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:59.759131+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":914,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3173}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132075+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:55.132075+0000","last_peered":"2024-09-15T21:30:55.132075+0000","last_clean":"2024-09-15T21:30:55.132075+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:55.132075+0000","last_undegraded":"2024-09-15T21:30:55.132075+0000","last_fullsized":"2024-09-15T21:30:55.132075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20628,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118135+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:55.118135+0000","last_peered":"2024-09-15T21:30:55.118135+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:55.118135+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118023+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:55.118023+0000","last_peered":"2024-09-15T21:30:55.118023+0000","last_clean":"2024-09-15T21:30:55.118023+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:55.118023+0000","last_undegraded":"2024-09-15T21:30:55.118023+0000","last_fullsized":"2024-09-15T21:30:55.118023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.132083+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:55.132083+0000","last_peered":"2024-09-15T21:30:55.132083+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:55.132083+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:55.120588+0000","last_peered":"2024-09-15T21:30:55.120588+0000","last_clean":"2024-09-15T21:30:55.120588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:55.120588+0000","last_undegraded":"2024-09-15T21:30:55.120588+0000","last_fullsized":"2024-09-15T21:30:55.120588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29034,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.131847+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:55.131847+0000","last_peered":"2024-09-15T21:30:55.131847+0000","last_clean":"2024-09-15T21:30:55.131847+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:55.131847+0000","last_undegraded":"2024-09-15T21:30:55.131847+0000","last_fullsized":"2024-09-15T21:30:55.131847+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20834,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118486+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:55.118486+0000","last_peered":"2024-09-15T21:30:55.118486+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:55.118486+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120705+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:55.120705+0000","last_peered":"2024-09-15T21:30:55.120705+0000","last_clean":"2024-09-15T21:30:55.120705+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:55.120705+0000","last_undegraded":"2024-09-15T21:30:55.120705+0000","last_fullsized":"2024-09-15T21:30:55.120705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21597,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.215742+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:30:55.215742+0000","last_peered":"2024-09-15T21:30:55.215742+0000","last_clean":"2024-09-15T21:30:55.215742+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:30:55.215742+0000","last_undegraded":"2024-09-15T21:30:55.215742+0000","last_fullsized":"2024-09-15T21:30:55.215742+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132523+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:55.132523+0000","last_peered":"2024-09-15T21:30:55.132523+0000","last_clean":"2024-09-15T21:30:55.132523+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:55.132523+0000","last_undegraded":"2024-09-15T21:30:55.132523+0000","last_fullsized":"2024-09-15T21:30:55.132523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28712,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120847+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:55.120847+0000","last_peered":"2024-09-15T21:30:55.120847+0000","last_clean":"2024-09-15T21:30:55.120847+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:55.120847+0000","last_undegraded":"2024-09-15T21:30:55.120847+0000","last_fullsized":"2024-09-15T21:30:55.120847+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120641+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:55.120641+0000","last_peered":"2024-09-15T21:30:55.120641+0000","last_clean":"2024-09-15T21:30:55.120641+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:55.120641+0000","last_undegraded":"2024-09-15T21:30:55.120641+0000","last_fullsized":"2024-09-15T21:30:55.120641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120915+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:55.120915+0000","last_peered":"2024-09-15T21:30:55.120915+0000","last_clean":"2024-09-15T21:30:55.120915+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:55.120915+0000","last_undegraded":"2024-09-15T21:30:55.120915+0000","last_fullsized":"2024-09-15T21:30:55.120915+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21020,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118144+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:55.118144+0000","last_peered":"2024-09-15T21:30:55.118144+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:55.118144+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17573,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202682368,"data_stored":202559526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492572,"kb_used_data":131844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879268,"statfs":{"total":96636764160,"available":96132370432,"internally_reserved":0,"allocated":135008256,"data_stored":134707698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535820,"kb_used_data":66868,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836020,"statfs":{"total":96636764160,"available":96088084480,"internally_reserved":0,"allocated":68472832,"data_stored":68168197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483372,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888468,"statfs":{"total":96636764160,"available":96141791232,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:03.548 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:03.933 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:03.934 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:03.947 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-15T21:31:02.886365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17574,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511764,"kb_used_data":199188,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281603756,"statfs":{"total":289910292480,"available":288362246144,"internally_reserved":0,"allocated":203968512,"data_stored":203044232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1343998979},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1962,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.779645"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21242,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.120153+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:55.120153+0000","last_peered":"2024-09-15T21:30:55.120153+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:55.120153+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132782+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:55.132782+0000","last_peered":"2024-09-15T21:30:55.132782+0000","last_clean":"2024-09-15T21:30:55.132782+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:55.132782+0000","last_undegraded":"2024-09-15T21:30:55.132782+0000","last_fullsized":"2024-09-15T21:30:55.132782+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29793,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120815+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:55.120815+0000","last_peered":"2024-09-15T21:30:55.120815+0000","last_clean":"2024-09-15T21:30:55.120815+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:55.120815+0000","last_undegraded":"2024-09-15T21:30:55.120815+0000","last_fullsized":"2024-09-15T21:30:55.120815+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132287+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:55.132287+0000","last_peered":"2024-09-15T21:30:55.132287+0000","last_clean":"2024-09-15T21:30:55.132287+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:55.132287+0000","last_undegraded":"2024-09-15T21:30:55.132287+0000","last_fullsized":"2024-09-15T21:30:55.132287+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20652,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118351+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:55.118351+0000","last_peered":"2024-09-15T21:30:55.118351+0000","last_clean":"2024-09-15T21:30:55.118351+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:55.118351+0000","last_undegraded":"2024-09-15T21:30:55.118351+0000","last_fullsized":"2024-09-15T21:30:55.118351+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20786,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118093+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:55.118093+0000","last_peered":"2024-09-15T21:30:55.118093+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:55.118093+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118034+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:55.118034+0000","last_peered":"2024-09-15T21:30:55.118034+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:55.118034+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21012,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.117918+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:55.117918+0000","last_peered":"2024-09-15T21:30:55.117918+0000","last_clean":"2024-09-15T21:30:55.117918+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:55.117918+0000","last_undegraded":"2024-09-15T21:30:55.117918+0000","last_fullsized":"2024-09-15T21:30:55.117918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120265+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:55.120265+0000","last_peered":"2024-09-15T21:30:55.120265+0000","last_clean":"2024-09-15T21:30:55.120265+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:55.120265+0000","last_undegraded":"2024-09-15T21:30:55.120265+0000","last_fullsized":"2024-09-15T21:30:55.120265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":22484,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:59.759131+0000","last_change":"2024-09-15T21:30:53.869994+0000","last_active":"2024-09-15T21:30:59.759131+0000","last_peered":"2024-09-15T21:30:59.759131+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:59.759131+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":914,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3173}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132075+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:55.132075+0000","last_peered":"2024-09-15T21:30:55.132075+0000","last_clean":"2024-09-15T21:30:55.132075+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:55.132075+0000","last_undegraded":"2024-09-15T21:30:55.132075+0000","last_fullsized":"2024-09-15T21:30:55.132075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20628,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118135+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:55.118135+0000","last_peered":"2024-09-15T21:30:55.118135+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:55.118135+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118023+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:55.118023+0000","last_peered":"2024-09-15T21:30:55.118023+0000","last_clean":"2024-09-15T21:30:55.118023+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:55.118023+0000","last_undegraded":"2024-09-15T21:30:55.118023+0000","last_fullsized":"2024-09-15T21:30:55.118023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.132083+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:55.132083+0000","last_peered":"2024-09-15T21:30:55.132083+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:55.132083+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:55.120588+0000","last_peered":"2024-09-15T21:30:55.120588+0000","last_clean":"2024-09-15T21:30:55.120588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:55.120588+0000","last_undegraded":"2024-09-15T21:30:55.120588+0000","last_fullsized":"2024-09-15T21:30:55.120588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29034,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.131847+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:55.131847+0000","last_peered":"2024-09-15T21:30:55.131847+0000","last_clean":"2024-09-15T21:30:55.131847+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:55.131847+0000","last_undegraded":"2024-09-15T21:30:55.131847+0000","last_fullsized":"2024-09-15T21:30:55.131847+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20834,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118486+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:55.118486+0000","last_peered":"2024-09-15T21:30:55.118486+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:55.118486+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120705+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:55.120705+0000","last_peered":"2024-09-15T21:30:55.120705+0000","last_clean":"2024-09-15T21:30:55.120705+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:55.120705+0000","last_undegraded":"2024-09-15T21:30:55.120705+0000","last_fullsized":"2024-09-15T21:30:55.120705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21597,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.215742+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:30:55.215742+0000","last_peered":"2024-09-15T21:30:55.215742+0000","last_clean":"2024-09-15T21:30:55.215742+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:30:55.215742+0000","last_undegraded":"2024-09-15T21:30:55.215742+0000","last_fullsized":"2024-09-15T21:30:55.215742+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132523+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:55.132523+0000","last_peered":"2024-09-15T21:30:55.132523+0000","last_clean":"2024-09-15T21:30:55.132523+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:55.132523+0000","last_undegraded":"2024-09-15T21:30:55.132523+0000","last_fullsized":"2024-09-15T21:30:55.132523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28712,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120847+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:55.120847+0000","last_peered":"2024-09-15T21:30:55.120847+0000","last_clean":"2024-09-15T21:30:55.120847+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:55.120847+0000","last_undegraded":"2024-09-15T21:30:55.120847+0000","last_fullsized":"2024-09-15T21:30:55.120847+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120641+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:55.120641+0000","last_peered":"2024-09-15T21:30:55.120641+0000","last_clean":"2024-09-15T21:30:55.120641+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:55.120641+0000","last_undegraded":"2024-09-15T21:30:55.120641+0000","last_fullsized":"2024-09-15T21:30:55.120641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120915+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:55.120915+0000","last_peered":"2024-09-15T21:30:55.120915+0000","last_clean":"2024-09-15T21:30:55.120915+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:55.120915+0000","last_undegraded":"2024-09-15T21:30:55.120915+0000","last_fullsized":"2024-09-15T21:30:55.120915+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21020,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118144+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:55.118144+0000","last_peered":"2024-09-15T21:30:55.118144+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:55.118144+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17573,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202682368,"data_stored":202559526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492572,"kb_used_data":131844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879268,"statfs":{"total":96636764160,"available":96132370432,"internally_reserved":0,"allocated":135008256,"data_stored":134707698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535820,"kb_used_data":66868,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836020,"statfs":{"total":96636764160,"available":96088084480,"internally_reserved":0,"allocated":68472832,"data_stored":68168197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483372,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888468,"statfs":{"total":96636764160,"available":96141791232,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:03.948 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:31:03.948 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21242, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.120153+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:55.120153+0000', 'last_peered': '2024-09-15T21:30:55.120153+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:55.120153+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:03.948 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:31:03.948 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20786, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118093+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:55.118093+0000', 'last_peered': '2024-09-15T21:30:55.118093+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:55.118093+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:03.948 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:31:03.948 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20821, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118034+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:55.118034+0000', 'last_peered': '2024-09-15T21:30:55.118034+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:55.118034+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:03.949 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:31:03.949 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20513", 'reported_seq': 22484, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:59.759131+0000', 'last_change': '2024-09-15T21:30:53.869994+0000', 'last_active': '2024-09-15T21:30:59.759131+0000', 'last_peered': '2024-09-15T21:30:59.759131+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:30:59.759131+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 914, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3173}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:03.949 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:31:03.949 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20628, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118135+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:55.118135+0000', 'last_peered': '2024-09-15T21:30:55.118135+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:55.118135+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:03.949 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:31:03.949 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20777, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.132083+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:55.132083+0000', 'last_peered': '2024-09-15T21:30:55.132083+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:55.132083+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:03.949 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:31:03.949 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 20834, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118486+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:55.118486+0000', 'last_peered': '2024-09-15T21:30:55.118486+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:55.118486+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:03.949 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:31:03.949 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21020, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118144+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:55.118144+0000', 'last_peered': '2024-09-15T21:30:55.118144+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:55.118144+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:03.950 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:31:04.392 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:04.392 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31901/24 objects degraded (132920.833%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":716,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1548046336,"bytes_avail":288362246144,"bytes_total":289910292480,"degraded_objects":31901,"degraded_total":24,"degraded_ratio":1329.208333333333,"recovering_objects_per_sec":223,"recovering_bytes_per_sec":55,"recovering_keys_per_sec":0,"num_objects_recovered":1962,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:31:04.405 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:31:04.406 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:04.765 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:04.766 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:04.778 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-15T21:31:02.886365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17574,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511764,"kb_used_data":199188,"kb_used_omap":12,"kb_used_meta":1312499,"kb_avail":281603756,"statfs":{"total":289910292480,"available":288362246144,"internally_reserved":0,"allocated":203968512,"data_stored":203044232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1343998979},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1962,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.779645"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21242,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.120153+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:55.120153+0000","last_peered":"2024-09-15T21:30:55.120153+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:55.120153+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132782+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:55.132782+0000","last_peered":"2024-09-15T21:30:55.132782+0000","last_clean":"2024-09-15T21:30:55.132782+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:55.132782+0000","last_undegraded":"2024-09-15T21:30:55.132782+0000","last_fullsized":"2024-09-15T21:30:55.132782+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29793,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120815+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:55.120815+0000","last_peered":"2024-09-15T21:30:55.120815+0000","last_clean":"2024-09-15T21:30:55.120815+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:55.120815+0000","last_undegraded":"2024-09-15T21:30:55.120815+0000","last_fullsized":"2024-09-15T21:30:55.120815+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132287+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:55.132287+0000","last_peered":"2024-09-15T21:30:55.132287+0000","last_clean":"2024-09-15T21:30:55.132287+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:55.132287+0000","last_undegraded":"2024-09-15T21:30:55.132287+0000","last_fullsized":"2024-09-15T21:30:55.132287+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20652,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118351+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:55.118351+0000","last_peered":"2024-09-15T21:30:55.118351+0000","last_clean":"2024-09-15T21:30:55.118351+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:55.118351+0000","last_undegraded":"2024-09-15T21:30:55.118351+0000","last_fullsized":"2024-09-15T21:30:55.118351+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20786,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118093+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:55.118093+0000","last_peered":"2024-09-15T21:30:55.118093+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:55.118093+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118034+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:55.118034+0000","last_peered":"2024-09-15T21:30:55.118034+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:55.118034+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21012,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.117918+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:55.117918+0000","last_peered":"2024-09-15T21:30:55.117918+0000","last_clean":"2024-09-15T21:30:55.117918+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:55.117918+0000","last_undegraded":"2024-09-15T21:30:55.117918+0000","last_fullsized":"2024-09-15T21:30:55.117918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120265+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:55.120265+0000","last_peered":"2024-09-15T21:30:55.120265+0000","last_clean":"2024-09-15T21:30:55.120265+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:55.120265+0000","last_undegraded":"2024-09-15T21:30:55.120265+0000","last_fullsized":"2024-09-15T21:30:55.120265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":22484,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:59.759131+0000","last_change":"2024-09-15T21:30:53.869994+0000","last_active":"2024-09-15T21:30:59.759131+0000","last_peered":"2024-09-15T21:30:59.759131+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:30:59.759131+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":914,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3173}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132075+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:55.132075+0000","last_peered":"2024-09-15T21:30:55.132075+0000","last_clean":"2024-09-15T21:30:55.132075+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:55.132075+0000","last_undegraded":"2024-09-15T21:30:55.132075+0000","last_fullsized":"2024-09-15T21:30:55.132075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20628,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118135+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:55.118135+0000","last_peered":"2024-09-15T21:30:55.118135+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:55.118135+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118023+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:55.118023+0000","last_peered":"2024-09-15T21:30:55.118023+0000","last_clean":"2024-09-15T21:30:55.118023+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:55.118023+0000","last_undegraded":"2024-09-15T21:30:55.118023+0000","last_fullsized":"2024-09-15T21:30:55.118023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.132083+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:55.132083+0000","last_peered":"2024-09-15T21:30:55.132083+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:55.132083+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:55.120588+0000","last_peered":"2024-09-15T21:30:55.120588+0000","last_clean":"2024-09-15T21:30:55.120588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:55.120588+0000","last_undegraded":"2024-09-15T21:30:55.120588+0000","last_fullsized":"2024-09-15T21:30:55.120588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29034,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.131847+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:55.131847+0000","last_peered":"2024-09-15T21:30:55.131847+0000","last_clean":"2024-09-15T21:30:55.131847+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:55.131847+0000","last_undegraded":"2024-09-15T21:30:55.131847+0000","last_fullsized":"2024-09-15T21:30:55.131847+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20834,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118486+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:55.118486+0000","last_peered":"2024-09-15T21:30:55.118486+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:55.118486+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120705+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:55.120705+0000","last_peered":"2024-09-15T21:30:55.120705+0000","last_clean":"2024-09-15T21:30:55.120705+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:55.120705+0000","last_undegraded":"2024-09-15T21:30:55.120705+0000","last_fullsized":"2024-09-15T21:30:55.120705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21597,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.215742+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:30:55.215742+0000","last_peered":"2024-09-15T21:30:55.215742+0000","last_clean":"2024-09-15T21:30:55.215742+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:30:55.215742+0000","last_undegraded":"2024-09-15T21:30:55.215742+0000","last_fullsized":"2024-09-15T21:30:55.215742+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132523+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:55.132523+0000","last_peered":"2024-09-15T21:30:55.132523+0000","last_clean":"2024-09-15T21:30:55.132523+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:55.132523+0000","last_undegraded":"2024-09-15T21:30:55.132523+0000","last_fullsized":"2024-09-15T21:30:55.132523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28712,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120847+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:55.120847+0000","last_peered":"2024-09-15T21:30:55.120847+0000","last_clean":"2024-09-15T21:30:55.120847+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:55.120847+0000","last_undegraded":"2024-09-15T21:30:55.120847+0000","last_fullsized":"2024-09-15T21:30:55.120847+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120641+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:55.120641+0000","last_peered":"2024-09-15T21:30:55.120641+0000","last_clean":"2024-09-15T21:30:55.120641+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:55.120641+0000","last_undegraded":"2024-09-15T21:30:55.120641+0000","last_fullsized":"2024-09-15T21:30:55.120641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120915+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:55.120915+0000","last_peered":"2024-09-15T21:30:55.120915+0000","last_clean":"2024-09-15T21:30:55.120915+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:55.120915+0000","last_undegraded":"2024-09-15T21:30:55.120915+0000","last_fullsized":"2024-09-15T21:30:55.120915+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21020,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118144+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:55.118144+0000","last_peered":"2024-09-15T21:30:55.118144+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:55.118144+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31901,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17573,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202682368,"data_stored":202559526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492572,"kb_used_data":131844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879268,"statfs":{"total":96636764160,"available":96132370432,"internally_reserved":0,"allocated":135008256,"data_stored":134707698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535820,"kb_used_data":66868,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836020,"statfs":{"total":96636764160,"available":96088084480,"internally_reserved":0,"allocated":68472832,"data_stored":68168197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483372,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888468,"statfs":{"total":96636764160,"available":96141791232,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494464687},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:07.781 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:08.110 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:08.111 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:08.123 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-09-15T21:31:06.887115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18591,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515924,"kb_used_data":199188,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281599596,"statfs":{"total":289910292480,"available":288357986304,"internally_reserved":0,"allocated":203968512,"data_stored":203044232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1348258819},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2893,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.772313"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21242,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.120153+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:55.120153+0000","last_peered":"2024-09-15T21:30:55.120153+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:55.120153+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132782+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:55.132782+0000","last_peered":"2024-09-15T21:30:55.132782+0000","last_clean":"2024-09-15T21:30:55.132782+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:55.132782+0000","last_undegraded":"2024-09-15T21:30:55.132782+0000","last_fullsized":"2024-09-15T21:30:55.132782+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29793,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120815+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:55.120815+0000","last_peered":"2024-09-15T21:30:55.120815+0000","last_clean":"2024-09-15T21:30:55.120815+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:55.120815+0000","last_undegraded":"2024-09-15T21:30:55.120815+0000","last_fullsized":"2024-09-15T21:30:55.120815+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132287+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:55.132287+0000","last_peered":"2024-09-15T21:30:55.132287+0000","last_clean":"2024-09-15T21:30:55.132287+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:55.132287+0000","last_undegraded":"2024-09-15T21:30:55.132287+0000","last_fullsized":"2024-09-15T21:30:55.132287+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20652,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118351+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:55.118351+0000","last_peered":"2024-09-15T21:30:55.118351+0000","last_clean":"2024-09-15T21:30:55.118351+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:55.118351+0000","last_undegraded":"2024-09-15T21:30:55.118351+0000","last_fullsized":"2024-09-15T21:30:55.118351+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20786,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118093+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:55.118093+0000","last_peered":"2024-09-15T21:30:55.118093+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:55.118093+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118034+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:55.118034+0000","last_peered":"2024-09-15T21:30:55.118034+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:55.118034+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21012,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.117918+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:55.117918+0000","last_peered":"2024-09-15T21:30:55.117918+0000","last_clean":"2024-09-15T21:30:55.117918+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:55.117918+0000","last_undegraded":"2024-09-15T21:30:55.117918+0000","last_fullsized":"2024-09-15T21:30:55.117918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120265+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:55.120265+0000","last_peered":"2024-09-15T21:30:55.120265+0000","last_clean":"2024-09-15T21:30:55.120265+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:55.120265+0000","last_undegraded":"2024-09-15T21:30:55.120265+0000","last_fullsized":"2024-09-15T21:30:55.120265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":24518,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:04.760751+0000","last_change":"2024-09-15T21:30:53.869994+0000","last_active":"2024-09-15T21:31:04.760751+0000","last_peered":"2024-09-15T21:31:04.760751+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:31:04.760751+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1931,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2156}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132075+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:55.132075+0000","last_peered":"2024-09-15T21:30:55.132075+0000","last_clean":"2024-09-15T21:30:55.132075+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:55.132075+0000","last_undegraded":"2024-09-15T21:30:55.132075+0000","last_fullsized":"2024-09-15T21:30:55.132075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20628,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118135+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:55.118135+0000","last_peered":"2024-09-15T21:30:55.118135+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:55.118135+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118023+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:55.118023+0000","last_peered":"2024-09-15T21:30:55.118023+0000","last_clean":"2024-09-15T21:30:55.118023+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:55.118023+0000","last_undegraded":"2024-09-15T21:30:55.118023+0000","last_fullsized":"2024-09-15T21:30:55.118023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.132083+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:55.132083+0000","last_peered":"2024-09-15T21:30:55.132083+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:55.132083+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:55.120588+0000","last_peered":"2024-09-15T21:30:55.120588+0000","last_clean":"2024-09-15T21:30:55.120588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:55.120588+0000","last_undegraded":"2024-09-15T21:30:55.120588+0000","last_fullsized":"2024-09-15T21:30:55.120588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29034,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.131847+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:55.131847+0000","last_peered":"2024-09-15T21:30:55.131847+0000","last_clean":"2024-09-15T21:30:55.131847+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:55.131847+0000","last_undegraded":"2024-09-15T21:30:55.131847+0000","last_fullsized":"2024-09-15T21:30:55.131847+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20834,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118486+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:55.118486+0000","last_peered":"2024-09-15T21:30:55.118486+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:55.118486+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120705+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:55.120705+0000","last_peered":"2024-09-15T21:30:55.120705+0000","last_clean":"2024-09-15T21:30:55.120705+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:55.120705+0000","last_undegraded":"2024-09-15T21:30:55.120705+0000","last_fullsized":"2024-09-15T21:30:55.120705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21597,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.215742+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:30:55.215742+0000","last_peered":"2024-09-15T21:30:55.215742+0000","last_clean":"2024-09-15T21:30:55.215742+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:30:55.215742+0000","last_undegraded":"2024-09-15T21:30:55.215742+0000","last_fullsized":"2024-09-15T21:30:55.215742+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132523+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:55.132523+0000","last_peered":"2024-09-15T21:30:55.132523+0000","last_clean":"2024-09-15T21:30:55.132523+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:55.132523+0000","last_undegraded":"2024-09-15T21:30:55.132523+0000","last_fullsized":"2024-09-15T21:30:55.132523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28712,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120847+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:55.120847+0000","last_peered":"2024-09-15T21:30:55.120847+0000","last_clean":"2024-09-15T21:30:55.120847+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:55.120847+0000","last_undegraded":"2024-09-15T21:30:55.120847+0000","last_fullsized":"2024-09-15T21:30:55.120847+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120641+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:55.120641+0000","last_peered":"2024-09-15T21:30:55.120641+0000","last_clean":"2024-09-15T21:30:55.120641+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:55.120641+0000","last_undegraded":"2024-09-15T21:30:55.120641+0000","last_fullsized":"2024-09-15T21:30:55.120641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120915+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:55.120915+0000","last_peered":"2024-09-15T21:30:55.120915+0000","last_clean":"2024-09-15T21:30:55.120915+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:55.120915+0000","last_undegraded":"2024-09-15T21:30:55.120915+0000","last_fullsized":"2024-09-15T21:30:55.120915+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21020,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118144+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:55.118144+0000","last_peered":"2024-09-15T21:30:55.118144+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:55.118144+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18590,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202682368,"data_stored":202559526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492572,"kb_used_data":131844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879268,"statfs":{"total":96636764160,"available":96132370432,"internally_reserved":0,"allocated":135008256,"data_stored":134707698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535820,"kb_used_data":66868,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836020,"statfs":{"total":96636764160,"available":96088084480,"internally_reserved":0,"allocated":68472832,"data_stored":68168197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487532,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884308,"statfs":{"total":96636764160,"available":96137531392,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498724527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:08.124 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:08.436 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:08.437 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:08.449 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-09-15T21:31:06.887115+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18591,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515924,"kb_used_data":199188,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281599596,"statfs":{"total":289910292480,"available":288357986304,"internally_reserved":0,"allocated":203968512,"data_stored":203044232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1348258819},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2893,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.772313"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21242,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.120153+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:55.120153+0000","last_peered":"2024-09-15T21:30:55.120153+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:55.120153+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132782+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:55.132782+0000","last_peered":"2024-09-15T21:30:55.132782+0000","last_clean":"2024-09-15T21:30:55.132782+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:55.132782+0000","last_undegraded":"2024-09-15T21:30:55.132782+0000","last_fullsized":"2024-09-15T21:30:55.132782+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29793,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120815+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:55.120815+0000","last_peered":"2024-09-15T21:30:55.120815+0000","last_clean":"2024-09-15T21:30:55.120815+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:55.120815+0000","last_undegraded":"2024-09-15T21:30:55.120815+0000","last_fullsized":"2024-09-15T21:30:55.120815+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132287+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:55.132287+0000","last_peered":"2024-09-15T21:30:55.132287+0000","last_clean":"2024-09-15T21:30:55.132287+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:55.132287+0000","last_undegraded":"2024-09-15T21:30:55.132287+0000","last_fullsized":"2024-09-15T21:30:55.132287+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20652,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118351+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:55.118351+0000","last_peered":"2024-09-15T21:30:55.118351+0000","last_clean":"2024-09-15T21:30:55.118351+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:55.118351+0000","last_undegraded":"2024-09-15T21:30:55.118351+0000","last_fullsized":"2024-09-15T21:30:55.118351+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20786,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118093+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:55.118093+0000","last_peered":"2024-09-15T21:30:55.118093+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:55.118093+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118034+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:55.118034+0000","last_peered":"2024-09-15T21:30:55.118034+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:55.118034+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21012,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.117918+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:55.117918+0000","last_peered":"2024-09-15T21:30:55.117918+0000","last_clean":"2024-09-15T21:30:55.117918+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:55.117918+0000","last_undegraded":"2024-09-15T21:30:55.117918+0000","last_fullsized":"2024-09-15T21:30:55.117918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120265+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:55.120265+0000","last_peered":"2024-09-15T21:30:55.120265+0000","last_clean":"2024-09-15T21:30:55.120265+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:55.120265+0000","last_undegraded":"2024-09-15T21:30:55.120265+0000","last_fullsized":"2024-09-15T21:30:55.120265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":24518,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:04.760751+0000","last_change":"2024-09-15T21:30:53.869994+0000","last_active":"2024-09-15T21:31:04.760751+0000","last_peered":"2024-09-15T21:31:04.760751+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:31:04.760751+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1931,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2156}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132075+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:55.132075+0000","last_peered":"2024-09-15T21:30:55.132075+0000","last_clean":"2024-09-15T21:30:55.132075+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:55.132075+0000","last_undegraded":"2024-09-15T21:30:55.132075+0000","last_fullsized":"2024-09-15T21:30:55.132075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20628,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118135+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:55.118135+0000","last_peered":"2024-09-15T21:30:55.118135+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:55.118135+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118023+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:55.118023+0000","last_peered":"2024-09-15T21:30:55.118023+0000","last_clean":"2024-09-15T21:30:55.118023+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:55.118023+0000","last_undegraded":"2024-09-15T21:30:55.118023+0000","last_fullsized":"2024-09-15T21:30:55.118023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.132083+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:55.132083+0000","last_peered":"2024-09-15T21:30:55.132083+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:55.132083+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:55.120588+0000","last_peered":"2024-09-15T21:30:55.120588+0000","last_clean":"2024-09-15T21:30:55.120588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:55.120588+0000","last_undegraded":"2024-09-15T21:30:55.120588+0000","last_fullsized":"2024-09-15T21:30:55.120588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29034,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.131847+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:55.131847+0000","last_peered":"2024-09-15T21:30:55.131847+0000","last_clean":"2024-09-15T21:30:55.131847+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:55.131847+0000","last_undegraded":"2024-09-15T21:30:55.131847+0000","last_fullsized":"2024-09-15T21:30:55.131847+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20834,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118486+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:55.118486+0000","last_peered":"2024-09-15T21:30:55.118486+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:55.118486+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120705+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:55.120705+0000","last_peered":"2024-09-15T21:30:55.120705+0000","last_clean":"2024-09-15T21:30:55.120705+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:55.120705+0000","last_undegraded":"2024-09-15T21:30:55.120705+0000","last_fullsized":"2024-09-15T21:30:55.120705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21597,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.215742+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:30:55.215742+0000","last_peered":"2024-09-15T21:30:55.215742+0000","last_clean":"2024-09-15T21:30:55.215742+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:30:55.215742+0000","last_undegraded":"2024-09-15T21:30:55.215742+0000","last_fullsized":"2024-09-15T21:30:55.215742+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132523+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:55.132523+0000","last_peered":"2024-09-15T21:30:55.132523+0000","last_clean":"2024-09-15T21:30:55.132523+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:55.132523+0000","last_undegraded":"2024-09-15T21:30:55.132523+0000","last_fullsized":"2024-09-15T21:30:55.132523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28712,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120847+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:55.120847+0000","last_peered":"2024-09-15T21:30:55.120847+0000","last_clean":"2024-09-15T21:30:55.120847+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:55.120847+0000","last_undegraded":"2024-09-15T21:30:55.120847+0000","last_fullsized":"2024-09-15T21:30:55.120847+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120641+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:55.120641+0000","last_peered":"2024-09-15T21:30:55.120641+0000","last_clean":"2024-09-15T21:30:55.120641+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:55.120641+0000","last_undegraded":"2024-09-15T21:30:55.120641+0000","last_fullsized":"2024-09-15T21:30:55.120641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120915+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:55.120915+0000","last_peered":"2024-09-15T21:30:55.120915+0000","last_clean":"2024-09-15T21:30:55.120915+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:55.120915+0000","last_undegraded":"2024-09-15T21:30:55.120915+0000","last_fullsized":"2024-09-15T21:30:55.120915+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21020,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118144+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:55.118144+0000","last_peered":"2024-09-15T21:30:55.118144+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:55.118144+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18590,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202682368,"data_stored":202559526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492572,"kb_used_data":131844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879268,"statfs":{"total":96636764160,"available":96132370432,"internally_reserved":0,"allocated":135008256,"data_stored":134707698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535820,"kb_used_data":66868,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836020,"statfs":{"total":96636764160,"available":96088084480,"internally_reserved":0,"allocated":68472832,"data_stored":68168197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487532,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884308,"statfs":{"total":96636764160,"available":96137531392,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498724527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:08.450 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:31:08.451 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21242, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.120153+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:55.120153+0000', 'last_peered': '2024-09-15T21:30:55.120153+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:55.120153+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:08.451 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:31:08.451 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20786, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118093+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:55.118093+0000', 'last_peered': '2024-09-15T21:30:55.118093+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:55.118093+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:08.451 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:31:08.451 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20821, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118034+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:55.118034+0000', 'last_peered': '2024-09-15T21:30:55.118034+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:55.118034+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:08.451 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:31:08.451 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20513", 'reported_seq': 24518, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:04.760751+0000', 'last_change': '2024-09-15T21:30:53.869994+0000', 'last_active': '2024-09-15T21:31:04.760751+0000', 'last_peered': '2024-09-15T21:31:04.760751+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:31:04.760751+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2157, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1931, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2156}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:08.451 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:31:08.451 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20628, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118135+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:55.118135+0000', 'last_peered': '2024-09-15T21:30:55.118135+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:55.118135+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:08.452 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:31:08.452 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20777, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.132083+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:55.132083+0000', 'last_peered': '2024-09-15T21:30:55.132083+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:55.132083+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:08.452 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:31:08.452 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 20834, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118486+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:55.118486+0000', 'last_peered': '2024-09-15T21:30:55.118486+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:55.118486+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:08.452 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:31:08.452 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21020, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118144+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:55.118144+0000', 'last_peered': '2024-09-15T21:30:55.118144+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:55.118144+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:08.452 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:31:08.830 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:08.831 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30884/24 objects degraded (128683.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":720,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1552306176,"bytes_avail":288357986304,"bytes_total":289910292480,"degraded_objects":30884,"degraded_total":24,"degraded_ratio":1286.8333333333333,"recovering_objects_per_sec":245,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2893,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:31:08.844 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:31:08.844 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:09.159 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:09.159 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:09.172 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-09-15T21:31:08.887443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18591,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515924,"kb_used_data":199188,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281599596,"statfs":{"total":289910292480,"available":288357986304,"internally_reserved":0,"allocated":203968512,"data_stored":203044232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1348258819},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1845,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1845,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002186"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21242,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.120153+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:55.120153+0000","last_peered":"2024-09-15T21:30:55.120153+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:55.120153+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132782+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:55.132782+0000","last_peered":"2024-09-15T21:30:55.132782+0000","last_clean":"2024-09-15T21:30:55.132782+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:55.132782+0000","last_undegraded":"2024-09-15T21:30:55.132782+0000","last_fullsized":"2024-09-15T21:30:55.132782+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29793,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120815+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:55.120815+0000","last_peered":"2024-09-15T21:30:55.120815+0000","last_clean":"2024-09-15T21:30:55.120815+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:55.120815+0000","last_undegraded":"2024-09-15T21:30:55.120815+0000","last_fullsized":"2024-09-15T21:30:55.120815+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132287+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:55.132287+0000","last_peered":"2024-09-15T21:30:55.132287+0000","last_clean":"2024-09-15T21:30:55.132287+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:55.132287+0000","last_undegraded":"2024-09-15T21:30:55.132287+0000","last_fullsized":"2024-09-15T21:30:55.132287+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20652,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118351+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:55.118351+0000","last_peered":"2024-09-15T21:30:55.118351+0000","last_clean":"2024-09-15T21:30:55.118351+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:55.118351+0000","last_undegraded":"2024-09-15T21:30:55.118351+0000","last_fullsized":"2024-09-15T21:30:55.118351+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20786,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118093+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:55.118093+0000","last_peered":"2024-09-15T21:30:55.118093+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:55.118093+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118034+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:55.118034+0000","last_peered":"2024-09-15T21:30:55.118034+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:55.118034+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21012,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.117918+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:55.117918+0000","last_peered":"2024-09-15T21:30:55.117918+0000","last_clean":"2024-09-15T21:30:55.117918+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:55.117918+0000","last_undegraded":"2024-09-15T21:30:55.117918+0000","last_fullsized":"2024-09-15T21:30:55.117918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120265+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:55.120265+0000","last_peered":"2024-09-15T21:30:55.120265+0000","last_clean":"2024-09-15T21:30:55.120265+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:55.120265+0000","last_undegraded":"2024-09-15T21:30:55.120265+0000","last_fullsized":"2024-09-15T21:30:55.120265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":24518,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:04.760751+0000","last_change":"2024-09-15T21:30:53.869994+0000","last_active":"2024-09-15T21:31:04.760751+0000","last_peered":"2024-09-15T21:31:04.760751+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:31:04.760751+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1931,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2156}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132075+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:55.132075+0000","last_peered":"2024-09-15T21:30:55.132075+0000","last_clean":"2024-09-15T21:30:55.132075+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:55.132075+0000","last_undegraded":"2024-09-15T21:30:55.132075+0000","last_fullsized":"2024-09-15T21:30:55.132075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20628,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118135+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:55.118135+0000","last_peered":"2024-09-15T21:30:55.118135+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:55.118135+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118023+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:55.118023+0000","last_peered":"2024-09-15T21:30:55.118023+0000","last_clean":"2024-09-15T21:30:55.118023+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:55.118023+0000","last_undegraded":"2024-09-15T21:30:55.118023+0000","last_fullsized":"2024-09-15T21:30:55.118023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.132083+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:55.132083+0000","last_peered":"2024-09-15T21:30:55.132083+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:55.132083+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:55.120588+0000","last_peered":"2024-09-15T21:30:55.120588+0000","last_clean":"2024-09-15T21:30:55.120588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:55.120588+0000","last_undegraded":"2024-09-15T21:30:55.120588+0000","last_fullsized":"2024-09-15T21:30:55.120588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29034,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.131847+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:55.131847+0000","last_peered":"2024-09-15T21:30:55.131847+0000","last_clean":"2024-09-15T21:30:55.131847+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:55.131847+0000","last_undegraded":"2024-09-15T21:30:55.131847+0000","last_fullsized":"2024-09-15T21:30:55.131847+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20834,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118486+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:55.118486+0000","last_peered":"2024-09-15T21:30:55.118486+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:55.118486+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120705+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:55.120705+0000","last_peered":"2024-09-15T21:30:55.120705+0000","last_clean":"2024-09-15T21:30:55.120705+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:55.120705+0000","last_undegraded":"2024-09-15T21:30:55.120705+0000","last_fullsized":"2024-09-15T21:30:55.120705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21597,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.215742+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:30:55.215742+0000","last_peered":"2024-09-15T21:30:55.215742+0000","last_clean":"2024-09-15T21:30:55.215742+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:30:55.215742+0000","last_undegraded":"2024-09-15T21:30:55.215742+0000","last_fullsized":"2024-09-15T21:30:55.215742+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132523+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:55.132523+0000","last_peered":"2024-09-15T21:30:55.132523+0000","last_clean":"2024-09-15T21:30:55.132523+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:55.132523+0000","last_undegraded":"2024-09-15T21:30:55.132523+0000","last_fullsized":"2024-09-15T21:30:55.132523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28712,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120847+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:55.120847+0000","last_peered":"2024-09-15T21:30:55.120847+0000","last_clean":"2024-09-15T21:30:55.120847+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:55.120847+0000","last_undegraded":"2024-09-15T21:30:55.120847+0000","last_fullsized":"2024-09-15T21:30:55.120847+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120641+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:55.120641+0000","last_peered":"2024-09-15T21:30:55.120641+0000","last_clean":"2024-09-15T21:30:55.120641+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:55.120641+0000","last_undegraded":"2024-09-15T21:30:55.120641+0000","last_fullsized":"2024-09-15T21:30:55.120641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120915+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:55.120915+0000","last_peered":"2024-09-15T21:30:55.120915+0000","last_clean":"2024-09-15T21:30:55.120915+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:55.120915+0000","last_undegraded":"2024-09-15T21:30:55.120915+0000","last_fullsized":"2024-09-15T21:30:55.120915+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21020,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118144+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:55.118144+0000","last_peered":"2024-09-15T21:30:55.118144+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:55.118144+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18590,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202682368,"data_stored":202559526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492572,"kb_used_data":131844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879268,"statfs":{"total":96636764160,"available":96132370432,"internally_reserved":0,"allocated":135008256,"data_stored":134707698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535820,"kb_used_data":66868,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836020,"statfs":{"total":96636764160,"available":96088084480,"internally_reserved":0,"allocated":68472832,"data_stored":68168197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487532,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884308,"statfs":{"total":96636764160,"available":96137531392,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498724527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:12.174 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:12.479 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:12.479 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:12.491 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-15T21:31:10.887776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19423,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515924,"kb_used_data":199188,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281599596,"statfs":{"total":289910292480,"available":288357986304,"internally_reserved":0,"allocated":203968512,"data_stored":203044232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1348258819},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2677,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002069"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21242,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.120153+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:55.120153+0000","last_peered":"2024-09-15T21:30:55.120153+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:55.120153+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132782+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:55.132782+0000","last_peered":"2024-09-15T21:30:55.132782+0000","last_clean":"2024-09-15T21:30:55.132782+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:55.132782+0000","last_undegraded":"2024-09-15T21:30:55.132782+0000","last_fullsized":"2024-09-15T21:30:55.132782+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29793,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120815+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:55.120815+0000","last_peered":"2024-09-15T21:30:55.120815+0000","last_clean":"2024-09-15T21:30:55.120815+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:55.120815+0000","last_undegraded":"2024-09-15T21:30:55.120815+0000","last_fullsized":"2024-09-15T21:30:55.120815+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132287+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:55.132287+0000","last_peered":"2024-09-15T21:30:55.132287+0000","last_clean":"2024-09-15T21:30:55.132287+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:55.132287+0000","last_undegraded":"2024-09-15T21:30:55.132287+0000","last_fullsized":"2024-09-15T21:30:55.132287+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20653,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118386+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:55.118386+0000","last_peered":"2024-09-15T21:30:55.118386+0000","last_clean":"2024-09-15T21:30:55.118386+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:55.118386+0000","last_undegraded":"2024-09-15T21:30:55.118386+0000","last_fullsized":"2024-09-15T21:30:55.118386+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20786,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118093+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:55.118093+0000","last_peered":"2024-09-15T21:30:55.118093+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:55.118093+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118034+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:55.118034+0000","last_peered":"2024-09-15T21:30:55.118034+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:55.118034+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21012,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.117918+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:55.117918+0000","last_peered":"2024-09-15T21:30:55.117918+0000","last_clean":"2024-09-15T21:30:55.117918+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:55.117918+0000","last_undegraded":"2024-09-15T21:30:55.117918+0000","last_fullsized":"2024-09-15T21:30:55.117918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120265+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:55.120265+0000","last_peered":"2024-09-15T21:30:55.120265+0000","last_clean":"2024-09-15T21:30:55.120265+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:55.120265+0000","last_undegraded":"2024-09-15T21:30:55.120265+0000","last_fullsized":"2024-09-15T21:30:55.120265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":26182,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:09.763068+0000","last_change":"2024-09-15T21:30:53.869994+0000","last_active":"2024-09-15T21:31:09.763068+0000","last_peered":"2024-09-15T21:31:09.763068+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:31:09.763068+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2763,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1324}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132075+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:55.132075+0000","last_peered":"2024-09-15T21:30:55.132075+0000","last_clean":"2024-09-15T21:30:55.132075+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:55.132075+0000","last_undegraded":"2024-09-15T21:30:55.132075+0000","last_fullsized":"2024-09-15T21:30:55.132075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20628,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118135+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:55.118135+0000","last_peered":"2024-09-15T21:30:55.118135+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:55.118135+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118023+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:55.118023+0000","last_peered":"2024-09-15T21:30:55.118023+0000","last_clean":"2024-09-15T21:30:55.118023+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:55.118023+0000","last_undegraded":"2024-09-15T21:30:55.118023+0000","last_fullsized":"2024-09-15T21:30:55.118023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.132083+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:55.132083+0000","last_peered":"2024-09-15T21:30:55.132083+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:55.132083+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:55.120588+0000","last_peered":"2024-09-15T21:30:55.120588+0000","last_clean":"2024-09-15T21:30:55.120588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:55.120588+0000","last_undegraded":"2024-09-15T21:30:55.120588+0000","last_fullsized":"2024-09-15T21:30:55.120588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29034,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.131847+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:55.131847+0000","last_peered":"2024-09-15T21:30:55.131847+0000","last_clean":"2024-09-15T21:30:55.131847+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:55.131847+0000","last_undegraded":"2024-09-15T21:30:55.131847+0000","last_fullsized":"2024-09-15T21:30:55.131847+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20835,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.119500+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:55.119500+0000","last_peered":"2024-09-15T21:30:55.119500+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:55.119500+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120705+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:55.120705+0000","last_peered":"2024-09-15T21:30:55.120705+0000","last_clean":"2024-09-15T21:30:55.120705+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:55.120705+0000","last_undegraded":"2024-09-15T21:30:55.120705+0000","last_fullsized":"2024-09-15T21:30:55.120705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21597,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.215742+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:30:55.215742+0000","last_peered":"2024-09-15T21:30:55.215742+0000","last_clean":"2024-09-15T21:30:55.215742+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:30:55.215742+0000","last_undegraded":"2024-09-15T21:30:55.215742+0000","last_fullsized":"2024-09-15T21:30:55.215742+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132523+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:55.132523+0000","last_peered":"2024-09-15T21:30:55.132523+0000","last_clean":"2024-09-15T21:30:55.132523+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:55.132523+0000","last_undegraded":"2024-09-15T21:30:55.132523+0000","last_fullsized":"2024-09-15T21:30:55.132523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28712,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120847+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:55.120847+0000","last_peered":"2024-09-15T21:30:55.120847+0000","last_clean":"2024-09-15T21:30:55.120847+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:55.120847+0000","last_undegraded":"2024-09-15T21:30:55.120847+0000","last_fullsized":"2024-09-15T21:30:55.120847+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120641+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:55.120641+0000","last_peered":"2024-09-15T21:30:55.120641+0000","last_clean":"2024-09-15T21:30:55.120641+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:55.120641+0000","last_undegraded":"2024-09-15T21:30:55.120641+0000","last_fullsized":"2024-09-15T21:30:55.120641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120915+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:55.120915+0000","last_peered":"2024-09-15T21:30:55.120915+0000","last_clean":"2024-09-15T21:30:55.120915+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:55.120915+0000","last_undegraded":"2024-09-15T21:30:55.120915+0000","last_fullsized":"2024-09-15T21:30:55.120915+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21020,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118144+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:55.118144+0000","last_peered":"2024-09-15T21:30:55.118144+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:55.118144+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19422,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202682368,"data_stored":202559526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492572,"kb_used_data":131844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879268,"statfs":{"total":96636764160,"available":96132370432,"internally_reserved":0,"allocated":135008256,"data_stored":134707698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535820,"kb_used_data":66868,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836020,"statfs":{"total":96636764160,"available":96088084480,"internally_reserved":0,"allocated":68472832,"data_stored":68168197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487532,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884308,"statfs":{"total":96636764160,"available":96137531392,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498724527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:12.493 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:12.873 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:12.885 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-15T21:31:10.887776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19423,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515924,"kb_used_data":199188,"kb_used_omap":12,"kb_used_meta":1316659,"kb_avail":281599596,"statfs":{"total":289910292480,"available":288357986304,"internally_reserved":0,"allocated":203968512,"data_stored":203044232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1348258819},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2677,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2677,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002069"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21242,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.120153+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:55.120153+0000","last_peered":"2024-09-15T21:30:55.120153+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:55.120153+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132782+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:55.132782+0000","last_peered":"2024-09-15T21:30:55.132782+0000","last_clean":"2024-09-15T21:30:55.132782+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:55.132782+0000","last_undegraded":"2024-09-15T21:30:55.132782+0000","last_fullsized":"2024-09-15T21:30:55.132782+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29793,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120815+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:55.120815+0000","last_peered":"2024-09-15T21:30:55.120815+0000","last_clean":"2024-09-15T21:30:55.120815+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:55.120815+0000","last_undegraded":"2024-09-15T21:30:55.120815+0000","last_fullsized":"2024-09-15T21:30:55.120815+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132287+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:55.132287+0000","last_peered":"2024-09-15T21:30:55.132287+0000","last_clean":"2024-09-15T21:30:55.132287+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:55.132287+0000","last_undegraded":"2024-09-15T21:30:55.132287+0000","last_fullsized":"2024-09-15T21:30:55.132287+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20653,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118386+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:55.118386+0000","last_peered":"2024-09-15T21:30:55.118386+0000","last_clean":"2024-09-15T21:30:55.118386+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:55.118386+0000","last_undegraded":"2024-09-15T21:30:55.118386+0000","last_fullsized":"2024-09-15T21:30:55.118386+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20786,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118093+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:55.118093+0000","last_peered":"2024-09-15T21:30:55.118093+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:55.118093+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118034+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:55.118034+0000","last_peered":"2024-09-15T21:30:55.118034+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:55.118034+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21012,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.117918+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:55.117918+0000","last_peered":"2024-09-15T21:30:55.117918+0000","last_clean":"2024-09-15T21:30:55.117918+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:55.117918+0000","last_undegraded":"2024-09-15T21:30:55.117918+0000","last_fullsized":"2024-09-15T21:30:55.117918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120265+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:55.120265+0000","last_peered":"2024-09-15T21:30:55.120265+0000","last_clean":"2024-09-15T21:30:55.120265+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:55.120265+0000","last_undegraded":"2024-09-15T21:30:55.120265+0000","last_fullsized":"2024-09-15T21:30:55.120265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":26182,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:09.763068+0000","last_change":"2024-09-15T21:30:53.869994+0000","last_active":"2024-09-15T21:31:09.763068+0000","last_peered":"2024-09-15T21:31:09.763068+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:31:09.763068+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2763,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1324}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132075+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:55.132075+0000","last_peered":"2024-09-15T21:30:55.132075+0000","last_clean":"2024-09-15T21:30:55.132075+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:55.132075+0000","last_undegraded":"2024-09-15T21:30:55.132075+0000","last_fullsized":"2024-09-15T21:30:55.132075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20628,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118135+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:55.118135+0000","last_peered":"2024-09-15T21:30:55.118135+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:55.118135+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118023+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:55.118023+0000","last_peered":"2024-09-15T21:30:55.118023+0000","last_clean":"2024-09-15T21:30:55.118023+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:55.118023+0000","last_undegraded":"2024-09-15T21:30:55.118023+0000","last_fullsized":"2024-09-15T21:30:55.118023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.132083+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:55.132083+0000","last_peered":"2024-09-15T21:30:55.132083+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:55.132083+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:55.120588+0000","last_peered":"2024-09-15T21:30:55.120588+0000","last_clean":"2024-09-15T21:30:55.120588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:55.120588+0000","last_undegraded":"2024-09-15T21:30:55.120588+0000","last_fullsized":"2024-09-15T21:30:55.120588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29034,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.131847+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:55.131847+0000","last_peered":"2024-09-15T21:30:55.131847+0000","last_clean":"2024-09-15T21:30:55.131847+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:55.131847+0000","last_undegraded":"2024-09-15T21:30:55.131847+0000","last_fullsized":"2024-09-15T21:30:55.131847+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20835,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.119500+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:55.119500+0000","last_peered":"2024-09-15T21:30:55.119500+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:55.119500+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120705+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:55.120705+0000","last_peered":"2024-09-15T21:30:55.120705+0000","last_clean":"2024-09-15T21:30:55.120705+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:55.120705+0000","last_undegraded":"2024-09-15T21:30:55.120705+0000","last_fullsized":"2024-09-15T21:30:55.120705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21597,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.215742+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:30:55.215742+0000","last_peered":"2024-09-15T21:30:55.215742+0000","last_clean":"2024-09-15T21:30:55.215742+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:30:55.215742+0000","last_undegraded":"2024-09-15T21:30:55.215742+0000","last_fullsized":"2024-09-15T21:30:55.215742+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132523+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:55.132523+0000","last_peered":"2024-09-15T21:30:55.132523+0000","last_clean":"2024-09-15T21:30:55.132523+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:55.132523+0000","last_undegraded":"2024-09-15T21:30:55.132523+0000","last_fullsized":"2024-09-15T21:30:55.132523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28712,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120847+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:55.120847+0000","last_peered":"2024-09-15T21:30:55.120847+0000","last_clean":"2024-09-15T21:30:55.120847+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:55.120847+0000","last_undegraded":"2024-09-15T21:30:55.120847+0000","last_fullsized":"2024-09-15T21:30:55.120847+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120641+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:55.120641+0000","last_peered":"2024-09-15T21:30:55.120641+0000","last_clean":"2024-09-15T21:30:55.120641+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:55.120641+0000","last_undegraded":"2024-09-15T21:30:55.120641+0000","last_fullsized":"2024-09-15T21:30:55.120641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120915+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:55.120915+0000","last_peered":"2024-09-15T21:30:55.120915+0000","last_clean":"2024-09-15T21:30:55.120915+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:55.120915+0000","last_undegraded":"2024-09-15T21:30:55.120915+0000","last_fullsized":"2024-09-15T21:30:55.120915+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21020,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118144+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:55.118144+0000","last_peered":"2024-09-15T21:30:55.118144+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:55.118144+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19422,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202682368,"data_stored":202559526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492572,"kb_used_data":131844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879268,"statfs":{"total":96636764160,"available":96132370432,"internally_reserved":0,"allocated":135008256,"data_stored":134707698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535820,"kb_used_data":66868,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836020,"statfs":{"total":96636764160,"available":96088084480,"internally_reserved":0,"allocated":68472832,"data_stored":68168197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487532,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884308,"statfs":{"total":96636764160,"available":96137531392,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498724527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:12.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:31:12.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21242, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.120153+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:55.120153+0000', 'last_peered': '2024-09-15T21:30:55.120153+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:55.120153+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:12.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:31:12.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20786, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118093+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:55.118093+0000', 'last_peered': '2024-09-15T21:30:55.118093+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:55.118093+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:12.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:31:12.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20821, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118034+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:55.118034+0000', 'last_peered': '2024-09-15T21:30:55.118034+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:55.118034+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:12.887 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:31:12.887 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20513", 'reported_seq': 26182, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:09.763068+0000', 'last_change': '2024-09-15T21:30:53.869994+0000', 'last_active': '2024-09-15T21:31:09.763068+0000', 'last_peered': '2024-09-15T21:31:09.763068+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:31:09.763068+0000', 'last_undegraded': '2024-09-15T21:21:14.394994+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 42, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1325, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2763, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1324}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:12.888 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:31:12.888 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20628, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118135+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:55.118135+0000', 'last_peered': '2024-09-15T21:30:55.118135+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:55.118135+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:12.888 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:31:12.888 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20777, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.132083+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:55.132083+0000', 'last_peered': '2024-09-15T21:30:55.132083+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:55.132083+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:12.888 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:31:12.888 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 20835, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.119500+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:55.119500+0000', 'last_peered': '2024-09-15T21:30:55.119500+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:55.119500+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:12.888 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:31:12.888 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21020, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118144+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:55.118144+0000', 'last_peered': '2024-09-15T21:30:55.118144+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:55.118144+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:12.888 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:31:13.280 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:13.280 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30052/24 objects degraded (125216.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":725,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1562267648,"bytes_avail":288348024832,"bytes_total":289910292480,"degraded_objects":30052,"degraded_total":24,"degraded_ratio":1252.1666666666667,"recovering_objects_per_sec":154,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1849,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:31:13.294 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:31:13.295 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:13.594 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:13.595 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:13.608 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-09-15T21:31:12.888133+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19423,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525652,"kb_used_data":199188,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281589868,"statfs":{"total":289910292480,"available":288348024832,"internally_reserved":0,"allocated":203968512,"data_stored":203044232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1358220291},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1849,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1849,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002112"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21242,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.120153+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:55.120153+0000","last_peered":"2024-09-15T21:30:55.120153+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:55.120153+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132782+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:55.132782+0000","last_peered":"2024-09-15T21:30:55.132782+0000","last_clean":"2024-09-15T21:30:55.132782+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:55.132782+0000","last_undegraded":"2024-09-15T21:30:55.132782+0000","last_fullsized":"2024-09-15T21:30:55.132782+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29793,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120815+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:30:55.120815+0000","last_peered":"2024-09-15T21:30:55.120815+0000","last_clean":"2024-09-15T21:30:55.120815+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:30:55.120815+0000","last_undegraded":"2024-09-15T21:30:55.120815+0000","last_fullsized":"2024-09-15T21:30:55.120815+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132287+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:55.132287+0000","last_peered":"2024-09-15T21:30:55.132287+0000","last_clean":"2024-09-15T21:30:55.132287+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:55.132287+0000","last_undegraded":"2024-09-15T21:30:55.132287+0000","last_fullsized":"2024-09-15T21:30:55.132287+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20653,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118386+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:55.118386+0000","last_peered":"2024-09-15T21:30:55.118386+0000","last_clean":"2024-09-15T21:30:55.118386+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:55.118386+0000","last_undegraded":"2024-09-15T21:30:55.118386+0000","last_fullsized":"2024-09-15T21:30:55.118386+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20786,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118093+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:55.118093+0000","last_peered":"2024-09-15T21:30:55.118093+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:55.118093+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118034+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:55.118034+0000","last_peered":"2024-09-15T21:30:55.118034+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:55.118034+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21012,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.117918+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:55.117918+0000","last_peered":"2024-09-15T21:30:55.117918+0000","last_clean":"2024-09-15T21:30:55.117918+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:55.117918+0000","last_undegraded":"2024-09-15T21:30:55.117918+0000","last_fullsized":"2024-09-15T21:30:55.117918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120265+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:30:55.120265+0000","last_peered":"2024-09-15T21:30:55.120265+0000","last_clean":"2024-09-15T21:30:55.120265+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:30:55.120265+0000","last_undegraded":"2024-09-15T21:30:55.120265+0000","last_fullsized":"2024-09-15T21:30:55.120265+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":26182,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:09.763068+0000","last_change":"2024-09-15T21:30:53.869994+0000","last_active":"2024-09-15T21:31:09.763068+0000","last_peered":"2024-09-15T21:31:09.763068+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:31:09.763068+0000","last_undegraded":"2024-09-15T21:21:14.394994+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":42,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2763,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1324}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132075+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:55.132075+0000","last_peered":"2024-09-15T21:30:55.132075+0000","last_clean":"2024-09-15T21:30:55.132075+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:55.132075+0000","last_undegraded":"2024-09-15T21:30:55.132075+0000","last_fullsized":"2024-09-15T21:30:55.132075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20628,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118135+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:55.118135+0000","last_peered":"2024-09-15T21:30:55.118135+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:55.118135+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118023+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:55.118023+0000","last_peered":"2024-09-15T21:30:55.118023+0000","last_clean":"2024-09-15T21:30:55.118023+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:55.118023+0000","last_undegraded":"2024-09-15T21:30:55.118023+0000","last_fullsized":"2024-09-15T21:30:55.118023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.132083+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:55.132083+0000","last_peered":"2024-09-15T21:30:55.132083+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:55.132083+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120588+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:30:55.120588+0000","last_peered":"2024-09-15T21:30:55.120588+0000","last_clean":"2024-09-15T21:30:55.120588+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:30:55.120588+0000","last_undegraded":"2024-09-15T21:30:55.120588+0000","last_fullsized":"2024-09-15T21:30:55.120588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29034,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.131847+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:55.131847+0000","last_peered":"2024-09-15T21:30:55.131847+0000","last_clean":"2024-09-15T21:30:55.131847+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:55.131847+0000","last_undegraded":"2024-09-15T21:30:55.131847+0000","last_fullsized":"2024-09-15T21:30:55.131847+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20835,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.119500+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:55.119500+0000","last_peered":"2024-09-15T21:30:55.119500+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:55.119500+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120705+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:30:55.120705+0000","last_peered":"2024-09-15T21:30:55.120705+0000","last_clean":"2024-09-15T21:30:55.120705+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:30:55.120705+0000","last_undegraded":"2024-09-15T21:30:55.120705+0000","last_fullsized":"2024-09-15T21:30:55.120705+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21597,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.215742+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:30:55.215742+0000","last_peered":"2024-09-15T21:30:55.215742+0000","last_clean":"2024-09-15T21:30:55.215742+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:30:55.215742+0000","last_undegraded":"2024-09-15T21:30:55.215742+0000","last_fullsized":"2024-09-15T21:30:55.215742+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132523+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:55.132523+0000","last_peered":"2024-09-15T21:30:55.132523+0000","last_clean":"2024-09-15T21:30:55.132523+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:55.132523+0000","last_undegraded":"2024-09-15T21:30:55.132523+0000","last_fullsized":"2024-09-15T21:30:55.132523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28712,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120847+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:30:55.120847+0000","last_peered":"2024-09-15T21:30:55.120847+0000","last_clean":"2024-09-15T21:30:55.120847+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:30:55.120847+0000","last_undegraded":"2024-09-15T21:30:55.120847+0000","last_fullsized":"2024-09-15T21:30:55.120847+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120641+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:30:55.120641+0000","last_peered":"2024-09-15T21:30:55.120641+0000","last_clean":"2024-09-15T21:30:55.120641+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:30:55.120641+0000","last_undegraded":"2024-09-15T21:30:55.120641+0000","last_fullsized":"2024-09-15T21:30:55.120641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.120915+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:30:55.120915+0000","last_peered":"2024-09-15T21:30:55.120915+0000","last_clean":"2024-09-15T21:30:55.120915+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:30:55.120915+0000","last_undegraded":"2024-09-15T21:30:55.120915+0000","last_fullsized":"2024-09-15T21:30:55.120915+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21020,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118144+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:55.118144+0000","last_peered":"2024-09-15T21:30:55.118144+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:55.118144+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30052,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19422,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202682368,"data_stored":202559526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492572,"kb_used_data":131844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879268,"statfs":{"total":96636764160,"available":96132370432,"internally_reserved":0,"allocated":135008256,"data_stored":134707698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535820,"kb_used_data":66868,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836020,"statfs":{"total":96636764160,"available":96088084480,"internally_reserved":0,"allocated":68472832,"data_stored":68168197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497260,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874580,"statfs":{"total":96636764160,"available":96127569920,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":6762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:16.610 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:16.924 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:16.924 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:16.938 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-15T21:31:16.888879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28727,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20748,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1533972,"kb_used_data":199188,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281581548,"statfs":{"total":289910292480,"available":288339505152,"internally_reserved":0,"allocated":203968512,"data_stored":203044232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1366739971},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001764"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21242,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.120153+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:55.120153+0000","last_peered":"2024-09-15T21:30:55.120153+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:55.120153+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132782+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:55.132782+0000","last_peered":"2024-09-15T21:30:55.132782+0000","last_clean":"2024-09-15T21:30:55.132782+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:55.132782+0000","last_undegraded":"2024-09-15T21:30:55.132782+0000","last_fullsized":"2024-09-15T21:30:55.132782+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29795,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157852+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:16.157852+0000","last_peered":"2024-09-15T21:31:16.157852+0000","last_clean":"2024-09-15T21:31:16.157852+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:16.157852+0000","last_undegraded":"2024-09-15T21:31:16.157852+0000","last_fullsized":"2024-09-15T21:31:16.157852+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132287+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:55.132287+0000","last_peered":"2024-09-15T21:30:55.132287+0000","last_clean":"2024-09-15T21:30:55.132287+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:55.132287+0000","last_undegraded":"2024-09-15T21:30:55.132287+0000","last_fullsized":"2024-09-15T21:30:55.132287+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20653,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118386+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:55.118386+0000","last_peered":"2024-09-15T21:30:55.118386+0000","last_clean":"2024-09-15T21:30:55.118386+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:55.118386+0000","last_undegraded":"2024-09-15T21:30:55.118386+0000","last_fullsized":"2024-09-15T21:30:55.118386+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20786,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118093+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:55.118093+0000","last_peered":"2024-09-15T21:30:55.118093+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:55.118093+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118034+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:55.118034+0000","last_peered":"2024-09-15T21:30:55.118034+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:55.118034+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21012,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.117918+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:55.117918+0000","last_peered":"2024-09-15T21:30:55.117918+0000","last_clean":"2024-09-15T21:30:55.117918+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:55.117918+0000","last_undegraded":"2024-09-15T21:30:55.117918+0000","last_fullsized":"2024-09-15T21:30:55.117918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157660+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:16.157660+0000","last_peered":"2024-09-15T21:31:16.157660+0000","last_clean":"2024-09-15T21:31:16.157660+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:16.157660+0000","last_undegraded":"2024-09-15T21:31:16.157660+0000","last_fullsized":"2024-09-15T21:31:16.157660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20654,"reported_epoch":52,"state":"peering","last_fresh":"2024-09-15T21:31:15.546118+0000","last_change":"2024-09-15T21:31:16.435123+0000","last_active":"2024-09-15T21:31:16.435123+0000","last_peered":"2024-09-15T21:31:15.546118+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:31:15.546118+0000","last_undegraded":"2024-09-15T21:31:15.546118+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132075+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:55.132075+0000","last_peered":"2024-09-15T21:30:55.132075+0000","last_clean":"2024-09-15T21:30:55.132075+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:55.132075+0000","last_undegraded":"2024-09-15T21:30:55.132075+0000","last_fullsized":"2024-09-15T21:30:55.132075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20628,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118135+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:55.118135+0000","last_peered":"2024-09-15T21:30:55.118135+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:55.118135+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118023+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:55.118023+0000","last_peered":"2024-09-15T21:30:55.118023+0000","last_clean":"2024-09-15T21:30:55.118023+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:55.118023+0000","last_undegraded":"2024-09-15T21:30:55.118023+0000","last_fullsized":"2024-09-15T21:30:55.118023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.132083+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:55.132083+0000","last_peered":"2024-09-15T21:30:55.132083+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:55.132083+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:16.157778+0000","last_peered":"2024-09-15T21:31:16.157778+0000","last_clean":"2024-09-15T21:31:16.157778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:16.157778+0000","last_undegraded":"2024-09-15T21:31:16.157778+0000","last_fullsized":"2024-09-15T21:31:16.157778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29034,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.131847+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:55.131847+0000","last_peered":"2024-09-15T21:30:55.131847+0000","last_clean":"2024-09-15T21:30:55.131847+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:55.131847+0000","last_undegraded":"2024-09-15T21:30:55.131847+0000","last_fullsized":"2024-09-15T21:30:55.131847+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20835,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.119500+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:55.119500+0000","last_peered":"2024-09-15T21:30:55.119500+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:55.119500+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157753+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:16.157753+0000","last_peered":"2024-09-15T21:31:16.157753+0000","last_clean":"2024-09-15T21:31:16.157753+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:16.157753+0000","last_undegraded":"2024-09-15T21:31:16.157753+0000","last_fullsized":"2024-09-15T21:31:16.157753+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21599,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157867+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:16.157867+0000","last_peered":"2024-09-15T21:31:16.157867+0000","last_clean":"2024-09-15T21:31:16.157867+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:16.157867+0000","last_undegraded":"2024-09-15T21:31:16.157867+0000","last_fullsized":"2024-09-15T21:31:16.157867+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132523+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:55.132523+0000","last_peered":"2024-09-15T21:30:55.132523+0000","last_clean":"2024-09-15T21:30:55.132523+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:55.132523+0000","last_undegraded":"2024-09-15T21:30:55.132523+0000","last_fullsized":"2024-09-15T21:30:55.132523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28714,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157947+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:16.157947+0000","last_peered":"2024-09-15T21:31:16.157947+0000","last_clean":"2024-09-15T21:31:16.157947+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:16.157947+0000","last_undegraded":"2024-09-15T21:31:16.157947+0000","last_fullsized":"2024-09-15T21:31:16.157947+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157809+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:16.157809+0000","last_peered":"2024-09-15T21:31:16.157809+0000","last_clean":"2024-09-15T21:31:16.157809+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:16.157809+0000","last_undegraded":"2024-09-15T21:31:16.157809+0000","last_fullsized":"2024-09-15T21:31:16.157809+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.158033+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:16.158033+0000","last_peered":"2024-09-15T21:31:16.158033+0000","last_clean":"2024-09-15T21:31:16.158033+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:16.158033+0000","last_undegraded":"2024-09-15T21:31:16.158033+0000","last_fullsized":"2024-09-15T21:31:16.158033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21020,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118144+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:55.118144+0000","last_peered":"2024-09-15T21:30:55.118144+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:55.118144+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28727,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20747,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492572,"kb_used_data":131844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879268,"statfs":{"total":96636764160,"available":96132370432,"internally_reserved":0,"allocated":135008256,"data_stored":134707698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535820,"kb_used_data":66868,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836020,"statfs":{"total":96636764160,"available":96088084480,"internally_reserved":0,"allocated":68472832,"data_stored":68168197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505580,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866260,"statfs":{"total":96636764160,"available":96119050240,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:16.939 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:17.256 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:17.268 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-15T21:31:17.160525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28727,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20748,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1533972,"kb_used_data":199188,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281581548,"statfs":{"total":289910292480,"available":288339505152,"internally_reserved":0,"allocated":203968512,"data_stored":203044232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1366739971},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.273081"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21242,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.120153+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:55.120153+0000","last_peered":"2024-09-15T21:30:55.120153+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:55.120153+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132782+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:55.132782+0000","last_peered":"2024-09-15T21:30:55.132782+0000","last_clean":"2024-09-15T21:30:55.132782+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:55.132782+0000","last_undegraded":"2024-09-15T21:30:55.132782+0000","last_fullsized":"2024-09-15T21:30:55.132782+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29795,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157852+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:16.157852+0000","last_peered":"2024-09-15T21:31:16.157852+0000","last_clean":"2024-09-15T21:31:16.157852+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:16.157852+0000","last_undegraded":"2024-09-15T21:31:16.157852+0000","last_fullsized":"2024-09-15T21:31:16.157852+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132287+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:55.132287+0000","last_peered":"2024-09-15T21:30:55.132287+0000","last_clean":"2024-09-15T21:30:55.132287+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:55.132287+0000","last_undegraded":"2024-09-15T21:30:55.132287+0000","last_fullsized":"2024-09-15T21:30:55.132287+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20653,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118386+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:55.118386+0000","last_peered":"2024-09-15T21:30:55.118386+0000","last_clean":"2024-09-15T21:30:55.118386+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:55.118386+0000","last_undegraded":"2024-09-15T21:30:55.118386+0000","last_fullsized":"2024-09-15T21:30:55.118386+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20786,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118093+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:55.118093+0000","last_peered":"2024-09-15T21:30:55.118093+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:55.118093+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118034+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:55.118034+0000","last_peered":"2024-09-15T21:30:55.118034+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:55.118034+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21012,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.117918+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:55.117918+0000","last_peered":"2024-09-15T21:30:55.117918+0000","last_clean":"2024-09-15T21:30:55.117918+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:55.117918+0000","last_undegraded":"2024-09-15T21:30:55.117918+0000","last_fullsized":"2024-09-15T21:30:55.117918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157660+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:16.157660+0000","last_peered":"2024-09-15T21:31:16.157660+0000","last_clean":"2024-09-15T21:31:16.157660+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:16.157660+0000","last_undegraded":"2024-09-15T21:31:16.157660+0000","last_fullsized":"2024-09-15T21:31:16.157660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20654,"reported_epoch":52,"state":"peering","last_fresh":"2024-09-15T21:31:15.546118+0000","last_change":"2024-09-15T21:31:16.435123+0000","last_active":"2024-09-15T21:31:16.435123+0000","last_peered":"2024-09-15T21:31:15.546118+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:31:15.546118+0000","last_undegraded":"2024-09-15T21:31:15.546118+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132075+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:55.132075+0000","last_peered":"2024-09-15T21:30:55.132075+0000","last_clean":"2024-09-15T21:30:55.132075+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:55.132075+0000","last_undegraded":"2024-09-15T21:30:55.132075+0000","last_fullsized":"2024-09-15T21:30:55.132075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20628,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118135+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:55.118135+0000","last_peered":"2024-09-15T21:30:55.118135+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:55.118135+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118023+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:55.118023+0000","last_peered":"2024-09-15T21:30:55.118023+0000","last_clean":"2024-09-15T21:30:55.118023+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:55.118023+0000","last_undegraded":"2024-09-15T21:30:55.118023+0000","last_fullsized":"2024-09-15T21:30:55.118023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.132083+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:55.132083+0000","last_peered":"2024-09-15T21:30:55.132083+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:55.132083+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:16.157778+0000","last_peered":"2024-09-15T21:31:16.157778+0000","last_clean":"2024-09-15T21:31:16.157778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:16.157778+0000","last_undegraded":"2024-09-15T21:31:16.157778+0000","last_fullsized":"2024-09-15T21:31:16.157778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29034,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.131847+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:55.131847+0000","last_peered":"2024-09-15T21:30:55.131847+0000","last_clean":"2024-09-15T21:30:55.131847+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:55.131847+0000","last_undegraded":"2024-09-15T21:30:55.131847+0000","last_fullsized":"2024-09-15T21:30:55.131847+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20835,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.119500+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:55.119500+0000","last_peered":"2024-09-15T21:30:55.119500+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:55.119500+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157753+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:16.157753+0000","last_peered":"2024-09-15T21:31:16.157753+0000","last_clean":"2024-09-15T21:31:16.157753+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:16.157753+0000","last_undegraded":"2024-09-15T21:31:16.157753+0000","last_fullsized":"2024-09-15T21:31:16.157753+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21599,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157867+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:16.157867+0000","last_peered":"2024-09-15T21:31:16.157867+0000","last_clean":"2024-09-15T21:31:16.157867+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:16.157867+0000","last_undegraded":"2024-09-15T21:31:16.157867+0000","last_fullsized":"2024-09-15T21:31:16.157867+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132523+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:55.132523+0000","last_peered":"2024-09-15T21:30:55.132523+0000","last_clean":"2024-09-15T21:30:55.132523+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:55.132523+0000","last_undegraded":"2024-09-15T21:30:55.132523+0000","last_fullsized":"2024-09-15T21:30:55.132523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28714,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157947+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:16.157947+0000","last_peered":"2024-09-15T21:31:16.157947+0000","last_clean":"2024-09-15T21:31:16.157947+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:16.157947+0000","last_undegraded":"2024-09-15T21:31:16.157947+0000","last_fullsized":"2024-09-15T21:31:16.157947+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157809+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:16.157809+0000","last_peered":"2024-09-15T21:31:16.157809+0000","last_clean":"2024-09-15T21:31:16.157809+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:16.157809+0000","last_undegraded":"2024-09-15T21:31:16.157809+0000","last_fullsized":"2024-09-15T21:31:16.157809+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.158033+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:16.158033+0000","last_peered":"2024-09-15T21:31:16.158033+0000","last_clean":"2024-09-15T21:31:16.158033+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:16.158033+0000","last_undegraded":"2024-09-15T21:31:16.158033+0000","last_fullsized":"2024-09-15T21:31:16.158033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21020,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118144+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:55.118144+0000","last_peered":"2024-09-15T21:30:55.118144+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:55.118144+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28727,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20747,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492572,"kb_used_data":131844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879268,"statfs":{"total":96636764160,"available":96132370432,"internally_reserved":0,"allocated":135008256,"data_stored":134707698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535820,"kb_used_data":66868,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836020,"statfs":{"total":96636764160,"available":96088084480,"internally_reserved":0,"allocated":68472832,"data_stored":68168197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505580,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866260,"statfs":{"total":96636764160,"available":96119050240,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:17.269 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:31:17.270 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21242, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.120153+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:30:55.120153+0000', 'last_peered': '2024-09-15T21:30:55.120153+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:30:55.120153+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:17.270 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:31:17.270 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20786, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118093+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:30:55.118093+0000', 'last_peered': '2024-09-15T21:30:55.118093+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:30:55.118093+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:17.270 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:31:17.270 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20821, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118034+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:30:55.118034+0000', 'last_peered': '2024-09-15T21:30:55.118034+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:30:55.118034+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:17.270 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:31:17.270 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20513", 'reported_seq': 20654, 'reported_epoch': 52, 'state': 'peering', 'last_fresh': '2024-09-15T21:31:15.546118+0000', 'last_change': '2024-09-15T21:31:16.435123+0000', 'last_active': '2024-09-15T21:31:16.435123+0000', 'last_peered': '2024-09-15T21:31:15.546118+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:31:15.546118+0000', 'last_undegraded': '2024-09-15T21:31:15.546118+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 52, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4088, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-15T21:31:17.270 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:31:17.270 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20628, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118135+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:30:55.118135+0000', 'last_peered': '2024-09-15T21:30:55.118135+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:30:55.118135+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:17.270 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:31:17.271 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20777, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.132083+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:30:55.132083+0000', 'last_peered': '2024-09-15T21:30:55.132083+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:30:55.132083+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:17.271 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:31:17.271 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 20835, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.119500+0000', 'last_change': '2024-09-15T21:21:14.882392+0000', 'last_active': '2024-09-15T21:30:55.119500+0000', 'last_peered': '2024-09-15T21:30:55.119500+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:30:55.119500+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4095, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:17.271 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:31:17.271 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21020, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:30:55.118144+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:30:55.118144+0000', 'last_peered': '2024-09-15T21:30:55.118144+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:30:55.118144+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:17.271 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:31:17.705 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:17.705 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28727/24 objects degraded (119695.833%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":729,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"peering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1570787328,"bytes_avail":288339505152,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":28727,"degraded_total":24,"degraded_ratio":1196.9583333333333,"recovering_objects_per_sec":215,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2157,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:31:17.718 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:31:17.719 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:18.079 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:18.080 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:18.095 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-15T21:31:17.160525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28727,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20748,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1533972,"kb_used_data":199188,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281581548,"statfs":{"total":289910292480,"available":288339505152,"internally_reserved":0,"allocated":203968512,"data_stored":203044232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1366739971},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2157,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.273081"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21242,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.120153+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:30:55.120153+0000","last_peered":"2024-09-15T21:30:55.120153+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:30:55.120153+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20794,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132782+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:30:55.132782+0000","last_peered":"2024-09-15T21:30:55.132782+0000","last_clean":"2024-09-15T21:30:55.132782+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:30:55.132782+0000","last_undegraded":"2024-09-15T21:30:55.132782+0000","last_fullsized":"2024-09-15T21:30:55.132782+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29795,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157852+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:16.157852+0000","last_peered":"2024-09-15T21:31:16.157852+0000","last_clean":"2024-09-15T21:31:16.157852+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:16.157852+0000","last_undegraded":"2024-09-15T21:31:16.157852+0000","last_fullsized":"2024-09-15T21:31:16.157852+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132287+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:30:55.132287+0000","last_peered":"2024-09-15T21:30:55.132287+0000","last_clean":"2024-09-15T21:30:55.132287+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:30:55.132287+0000","last_undegraded":"2024-09-15T21:30:55.132287+0000","last_fullsized":"2024-09-15T21:30:55.132287+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20653,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118386+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:30:55.118386+0000","last_peered":"2024-09-15T21:30:55.118386+0000","last_clean":"2024-09-15T21:30:55.118386+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:30:55.118386+0000","last_undegraded":"2024-09-15T21:30:55.118386+0000","last_fullsized":"2024-09-15T21:30:55.118386+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20786,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118093+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:30:55.118093+0000","last_peered":"2024-09-15T21:30:55.118093+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:30:55.118093+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20821,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118034+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:30:55.118034+0000","last_peered":"2024-09-15T21:30:55.118034+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:30:55.118034+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21012,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.117918+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:30:55.117918+0000","last_peered":"2024-09-15T21:30:55.117918+0000","last_clean":"2024-09-15T21:30:55.117918+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:30:55.117918+0000","last_undegraded":"2024-09-15T21:30:55.117918+0000","last_fullsized":"2024-09-15T21:30:55.117918+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157660+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:16.157660+0000","last_peered":"2024-09-15T21:31:16.157660+0000","last_clean":"2024-09-15T21:31:16.157660+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:16.157660+0000","last_undegraded":"2024-09-15T21:31:16.157660+0000","last_fullsized":"2024-09-15T21:31:16.157660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20654,"reported_epoch":52,"state":"peering","last_fresh":"2024-09-15T21:31:15.546118+0000","last_change":"2024-09-15T21:31:16.435123+0000","last_active":"2024-09-15T21:31:16.435123+0000","last_peered":"2024-09-15T21:31:15.546118+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:31:15.546118+0000","last_undegraded":"2024-09-15T21:31:15.546118+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132075+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:30:55.132075+0000","last_peered":"2024-09-15T21:30:55.132075+0000","last_clean":"2024-09-15T21:30:55.132075+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:30:55.132075+0000","last_undegraded":"2024-09-15T21:30:55.132075+0000","last_fullsized":"2024-09-15T21:30:55.132075+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20628,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118135+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:30:55.118135+0000","last_peered":"2024-09-15T21:30:55.118135+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:30:55.118135+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.118023+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:30:55.118023+0000","last_peered":"2024-09-15T21:30:55.118023+0000","last_clean":"2024-09-15T21:30:55.118023+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:30:55.118023+0000","last_undegraded":"2024-09-15T21:30:55.118023+0000","last_fullsized":"2024-09-15T21:30:55.118023+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20777,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.132083+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:30:55.132083+0000","last_peered":"2024-09-15T21:30:55.132083+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:30:55.132083+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:16.157778+0000","last_peered":"2024-09-15T21:31:16.157778+0000","last_clean":"2024-09-15T21:31:16.157778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:16.157778+0000","last_undegraded":"2024-09-15T21:31:16.157778+0000","last_fullsized":"2024-09-15T21:31:16.157778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29034,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.131847+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:30:55.131847+0000","last_peered":"2024-09-15T21:30:55.131847+0000","last_clean":"2024-09-15T21:30:55.131847+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:30:55.131847+0000","last_undegraded":"2024-09-15T21:30:55.131847+0000","last_fullsized":"2024-09-15T21:30:55.131847+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20835,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.119500+0000","last_change":"2024-09-15T21:21:14.882392+0000","last_active":"2024-09-15T21:30:55.119500+0000","last_peered":"2024-09-15T21:30:55.119500+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:30:55.119500+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157753+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:16.157753+0000","last_peered":"2024-09-15T21:31:16.157753+0000","last_clean":"2024-09-15T21:31:16.157753+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:16.157753+0000","last_undegraded":"2024-09-15T21:31:16.157753+0000","last_fullsized":"2024-09-15T21:31:16.157753+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21599,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157867+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:16.157867+0000","last_peered":"2024-09-15T21:31:16.157867+0000","last_clean":"2024-09-15T21:31:16.157867+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:16.157867+0000","last_undegraded":"2024-09-15T21:31:16.157867+0000","last_fullsized":"2024-09-15T21:31:16.157867+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-15T21:30:55.132523+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:30:55.132523+0000","last_peered":"2024-09-15T21:30:55.132523+0000","last_clean":"2024-09-15T21:30:55.132523+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:30:55.132523+0000","last_undegraded":"2024-09-15T21:30:55.132523+0000","last_fullsized":"2024-09-15T21:30:55.132523+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28714,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157947+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:16.157947+0000","last_peered":"2024-09-15T21:31:16.157947+0000","last_clean":"2024-09-15T21:31:16.157947+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:16.157947+0000","last_undegraded":"2024-09-15T21:31:16.157947+0000","last_fullsized":"2024-09-15T21:31:16.157947+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157809+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:16.157809+0000","last_peered":"2024-09-15T21:31:16.157809+0000","last_clean":"2024-09-15T21:31:16.157809+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:16.157809+0000","last_undegraded":"2024-09-15T21:31:16.157809+0000","last_fullsized":"2024-09-15T21:31:16.157809+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.158033+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:16.158033+0000","last_peered":"2024-09-15T21:31:16.158033+0000","last_clean":"2024-09-15T21:31:16.158033+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:16.158033+0000","last_undegraded":"2024-09-15T21:31:16.158033+0000","last_fullsized":"2024-09-15T21:31:16.158033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21020,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:30:55.118144+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:30:55.118144+0000","last_peered":"2024-09-15T21:30:55.118144+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:30:55.118144+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28727,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20747,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492572,"kb_used_data":131844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879268,"statfs":{"total":96636764160,"available":96132370432,"internally_reserved":0,"allocated":135008256,"data_stored":134707698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535820,"kb_used_data":66868,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836020,"statfs":{"total":96636764160,"available":96088084480,"internally_reserved":0,"allocated":68472832,"data_stored":68168197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505580,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866260,"statfs":{"total":96636764160,"available":96119050240,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:21.097 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:21.397 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:21.397 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:21.409 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-15T21:31:20.889704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21196,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534000,"kb_used_data":199220,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281581520,"statfs":{"total":289910292480,"available":288339476480,"internally_reserved":0,"allocated":204001280,"data_stored":203057478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1366739971},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1773,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001571"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29795,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157852+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:16.157852+0000","last_peered":"2024-09-15T21:31:16.157852+0000","last_clean":"2024-09-15T21:31:16.157852+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:16.157852+0000","last_undegraded":"2024-09-15T21:31:16.157852+0000","last_fullsized":"2024-09-15T21:31:16.157852+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163016+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163016+0000","last_peered":"2024-09-15T21:31:17.163016+0000","last_clean":"2024-09-15T21:31:17.163016+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163016+0000","last_undegraded":"2024-09-15T21:31:17.163016+0000","last_fullsized":"2024-09-15T21:31:17.163016+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157660+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:16.157660+0000","last_peered":"2024-09-15T21:31:16.157660+0000","last_clean":"2024-09-15T21:31:16.157660+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:16.157660+0000","last_undegraded":"2024-09-15T21:31:16.157660+0000","last_fullsized":"2024-09-15T21:31:16.157660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20654,"reported_epoch":52,"state":"peering","last_fresh":"2024-09-15T21:31:15.546118+0000","last_change":"2024-09-15T21:31:16.435123+0000","last_active":"2024-09-15T21:31:16.435123+0000","last_peered":"2024-09-15T21:31:15.546118+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:31:15.546118+0000","last_undegraded":"2024-09-15T21:31:15.546118+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:16.157778+0000","last_peered":"2024-09-15T21:31:16.157778+0000","last_clean":"2024-09-15T21:31:16.157778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:16.157778+0000","last_undegraded":"2024-09-15T21:31:16.157778+0000","last_fullsized":"2024-09-15T21:31:16.157778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":21736,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:19.552403+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:19.552403+0000","last_peered":"2024-09-15T21:31:19.552403+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:19.552403+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3647}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157753+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:16.157753+0000","last_peered":"2024-09-15T21:31:16.157753+0000","last_clean":"2024-09-15T21:31:16.157753+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:16.157753+0000","last_undegraded":"2024-09-15T21:31:16.157753+0000","last_fullsized":"2024-09-15T21:31:16.157753+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21599,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157867+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:16.157867+0000","last_peered":"2024-09-15T21:31:16.157867+0000","last_clean":"2024-09-15T21:31:16.157867+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:16.157867+0000","last_undegraded":"2024-09-15T21:31:16.157867+0000","last_fullsized":"2024-09-15T21:31:16.157867+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28714,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157947+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:16.157947+0000","last_peered":"2024-09-15T21:31:16.157947+0000","last_clean":"2024-09-15T21:31:16.157947+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:16.157947+0000","last_undegraded":"2024-09-15T21:31:16.157947+0000","last_fullsized":"2024-09-15T21:31:16.157947+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157809+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:16.157809+0000","last_peered":"2024-09-15T21:31:16.157809+0000","last_clean":"2024-09-15T21:31:16.157809+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:16.157809+0000","last_undegraded":"2024-09-15T21:31:16.157809+0000","last_fullsized":"2024-09-15T21:31:16.157809+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.158033+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:16.158033+0000","last_peered":"2024-09-15T21:31:16.158033+0000","last_clean":"2024-09-15T21:31:16.158033+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:16.158033+0000","last_undegraded":"2024-09-15T21:31:16.158033+0000","last_fullsized":"2024-09-15T21:31:16.158033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21195,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505580,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866260,"statfs":{"total":96636764160,"available":96119050240,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:21.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:21.729 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:21.729 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:21.742 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-15T21:31:20.889704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21196,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534000,"kb_used_data":199220,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281581520,"statfs":{"total":289910292480,"available":288339476480,"internally_reserved":0,"allocated":204001280,"data_stored":203057478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1366739971},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1773,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001571"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29795,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157852+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:16.157852+0000","last_peered":"2024-09-15T21:31:16.157852+0000","last_clean":"2024-09-15T21:31:16.157852+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:16.157852+0000","last_undegraded":"2024-09-15T21:31:16.157852+0000","last_fullsized":"2024-09-15T21:31:16.157852+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163016+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163016+0000","last_peered":"2024-09-15T21:31:17.163016+0000","last_clean":"2024-09-15T21:31:17.163016+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163016+0000","last_undegraded":"2024-09-15T21:31:17.163016+0000","last_fullsized":"2024-09-15T21:31:17.163016+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157660+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:16.157660+0000","last_peered":"2024-09-15T21:31:16.157660+0000","last_clean":"2024-09-15T21:31:16.157660+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:16.157660+0000","last_undegraded":"2024-09-15T21:31:16.157660+0000","last_fullsized":"2024-09-15T21:31:16.157660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20654,"reported_epoch":52,"state":"peering","last_fresh":"2024-09-15T21:31:15.546118+0000","last_change":"2024-09-15T21:31:16.435123+0000","last_active":"2024-09-15T21:31:16.435123+0000","last_peered":"2024-09-15T21:31:15.546118+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:31:15.546118+0000","last_undegraded":"2024-09-15T21:31:15.546118+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:16.157778+0000","last_peered":"2024-09-15T21:31:16.157778+0000","last_clean":"2024-09-15T21:31:16.157778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:16.157778+0000","last_undegraded":"2024-09-15T21:31:16.157778+0000","last_fullsized":"2024-09-15T21:31:16.157778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":21736,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:19.552403+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:19.552403+0000","last_peered":"2024-09-15T21:31:19.552403+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:19.552403+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3647}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157753+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:16.157753+0000","last_peered":"2024-09-15T21:31:16.157753+0000","last_clean":"2024-09-15T21:31:16.157753+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:16.157753+0000","last_undegraded":"2024-09-15T21:31:16.157753+0000","last_fullsized":"2024-09-15T21:31:16.157753+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21599,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157867+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:16.157867+0000","last_peered":"2024-09-15T21:31:16.157867+0000","last_clean":"2024-09-15T21:31:16.157867+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:16.157867+0000","last_undegraded":"2024-09-15T21:31:16.157867+0000","last_fullsized":"2024-09-15T21:31:16.157867+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28714,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157947+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:16.157947+0000","last_peered":"2024-09-15T21:31:16.157947+0000","last_clean":"2024-09-15T21:31:16.157947+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:16.157947+0000","last_undegraded":"2024-09-15T21:31:16.157947+0000","last_fullsized":"2024-09-15T21:31:16.157947+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157809+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:16.157809+0000","last_peered":"2024-09-15T21:31:16.157809+0000","last_clean":"2024-09-15T21:31:16.157809+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:16.157809+0000","last_undegraded":"2024-09-15T21:31:16.157809+0000","last_fullsized":"2024-09-15T21:31:16.157809+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.158033+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:16.158033+0000","last_peered":"2024-09-15T21:31:16.158033+0000","last_clean":"2024-09-15T21:31:16.158033+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:16.158033+0000","last_undegraded":"2024-09-15T21:31:16.158033+0000","last_fullsized":"2024-09-15T21:31:16.158033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21195,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505580,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866260,"statfs":{"total":96636764160,"available":96119050240,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:21.743 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:31:21.743 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21247, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.170590+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:31:17.170590+0000', 'last_peered': '2024-09-15T21:31:17.170590+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:31:17.170590+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:21.743 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:31:21.743 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20790, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.164628+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:31:17.164628+0000', 'last_peered': '2024-09-15T21:31:17.164628+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:31:17.164628+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:21.743 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:31:21.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20825, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.164412+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:31:17.164412+0000', 'last_peered': '2024-09-15T21:31:17.164412+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:31:17.164412+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:21.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.3 is not active+clean 2024-09-15T21:31:21.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.3', 'version': "49'20513", 'reported_seq': 20654, 'reported_epoch': 52, 'state': 'peering', 'last_fresh': '2024-09-15T21:31:15.546118+0000', 'last_change': '2024-09-15T21:31:16.435123+0000', 'last_active': '2024-09-15T21:31:16.435123+0000', 'last_peered': '2024-09-15T21:31:15.546118+0000', 'last_clean': '2024-09-15T21:21:12.300354+0000', 'last_became_active': '2024-09-15T21:21:14.853307+0000', 'last_became_peered': '2024-09-15T21:21:14.853307+0000', 'last_unstale': '2024-09-15T21:31:15.546118+0000', 'last_undegraded': '2024-09-15T21:31:15.546118+0000', 'last_fullsized': '2024-09-15T21:21:14.353125+0000', 'mapping_epoch': 52, 'log_start': "45'10500", 'ondisk_log_start': "45'10500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2088", 'last_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:46.454755+0000', 'objects_scrubbed': 2062, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 567, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 32768, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4088, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-15T21:31:21.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:31:21.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20632, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.165428+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:31:17.165428+0000', 'last_peered': '2024-09-15T21:31:17.165428+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:31:17.165428+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:21.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:31:21.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20781, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.162389+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:31:17.162389+0000', 'last_peered': '2024-09-15T21:31:17.162389+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:31:17.162389+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:21.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:31:21.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 21736, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:19.552403+0000', 'last_change': '2024-09-15T21:31:15.546423+0000', 'last_active': '2024-09-15T21:31:19.552403+0000', 'last_peered': '2024-09-15T21:31:19.552403+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:31:19.552403+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3647, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 449, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3647}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:21.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:31:21.745 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21024, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.166895+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:31:17.166895+0000', 'last_peered': '2024-09-15T21:31:17.166895+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:31:17.166895+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:21.745 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:31:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:22.130 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 28279/24 objects degraded (117829.167%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":733,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1570816000,"bytes_avail":288339476480,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":28279,"degraded_total":24,"degraded_ratio":1178.2916666666667,"recovering_objects_per_sec":221,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1773,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:31:22.143 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:31:22.143 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:22.448 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:22.448 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:22.461 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-15T21:31:20.889704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21196,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534000,"kb_used_data":199220,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281581520,"statfs":{"total":289910292480,"available":288339476480,"internally_reserved":0,"allocated":204001280,"data_stored":203057478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1366739971},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1773,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1773,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001571"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29795,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157852+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:16.157852+0000","last_peered":"2024-09-15T21:31:16.157852+0000","last_clean":"2024-09-15T21:31:16.157852+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:16.157852+0000","last_undegraded":"2024-09-15T21:31:16.157852+0000","last_fullsized":"2024-09-15T21:31:16.157852+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163016+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163016+0000","last_peered":"2024-09-15T21:31:17.163016+0000","last_clean":"2024-09-15T21:31:17.163016+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163016+0000","last_undegraded":"2024-09-15T21:31:17.163016+0000","last_fullsized":"2024-09-15T21:31:17.163016+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157660+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:16.157660+0000","last_peered":"2024-09-15T21:31:16.157660+0000","last_clean":"2024-09-15T21:31:16.157660+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:16.157660+0000","last_undegraded":"2024-09-15T21:31:16.157660+0000","last_fullsized":"2024-09-15T21:31:16.157660+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20654,"reported_epoch":52,"state":"peering","last_fresh":"2024-09-15T21:31:15.546118+0000","last_change":"2024-09-15T21:31:16.435123+0000","last_active":"2024-09-15T21:31:16.435123+0000","last_peered":"2024-09-15T21:31:15.546118+0000","last_clean":"2024-09-15T21:21:12.300354+0000","last_became_active":"2024-09-15T21:21:14.853307+0000","last_became_peered":"2024-09-15T21:21:14.853307+0000","last_unstale":"2024-09-15T21:31:15.546118+0000","last_undegraded":"2024-09-15T21:31:15.546118+0000","last_fullsized":"2024-09-15T21:21:14.353125+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":96,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157778+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:16.157778+0000","last_peered":"2024-09-15T21:31:16.157778+0000","last_clean":"2024-09-15T21:31:16.157778+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:16.157778+0000","last_undegraded":"2024-09-15T21:31:16.157778+0000","last_fullsized":"2024-09-15T21:31:16.157778+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":21736,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:19.552403+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:19.552403+0000","last_peered":"2024-09-15T21:31:19.552403+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:19.552403+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":449,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3647}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157753+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:16.157753+0000","last_peered":"2024-09-15T21:31:16.157753+0000","last_clean":"2024-09-15T21:31:16.157753+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:16.157753+0000","last_undegraded":"2024-09-15T21:31:16.157753+0000","last_fullsized":"2024-09-15T21:31:16.157753+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21599,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157867+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:16.157867+0000","last_peered":"2024-09-15T21:31:16.157867+0000","last_clean":"2024-09-15T21:31:16.157867+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:16.157867+0000","last_undegraded":"2024-09-15T21:31:16.157867+0000","last_fullsized":"2024-09-15T21:31:16.157867+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28714,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157947+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:16.157947+0000","last_peered":"2024-09-15T21:31:16.157947+0000","last_clean":"2024-09-15T21:31:16.157947+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:16.157947+0000","last_undegraded":"2024-09-15T21:31:16.157947+0000","last_fullsized":"2024-09-15T21:31:16.157947+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.157809+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:16.157809+0000","last_peered":"2024-09-15T21:31:16.157809+0000","last_clean":"2024-09-15T21:31:16.157809+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:16.157809+0000","last_undegraded":"2024-09-15T21:31:16.157809+0000","last_fullsized":"2024-09-15T21:31:16.157809+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-15T21:31:16.158033+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:16.158033+0000","last_peered":"2024-09-15T21:31:16.158033+0000","last_clean":"2024-09-15T21:31:16.158033+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:16.158033+0000","last_undegraded":"2024-09-15T21:31:16.158033+0000","last_fullsized":"2024-09-15T21:31:16.158033+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21195,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505580,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866260,"statfs":{"total":96636764160,"available":96119050240,"internally_reserved":0,"allocated":487424,"data_stored":168337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:25.463 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:25.828 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:25.828 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:25.841 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-15T21:31:24.890389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21773,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1542332,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281573188,"statfs":{"total":289910292480,"available":288330944512,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1375259651},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.733678"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163016+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163016+0000","last_peered":"2024-09-15T21:31:17.163016+0000","last_clean":"2024-09-15T21:31:17.163016+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163016+0000","last_undegraded":"2024-09-15T21:31:17.163016+0000","last_fullsized":"2024-09-15T21:31:17.163016+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163334+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163334+0000","last_peered":"2024-09-15T21:31:17.163334+0000","last_clean":"2024-09-15T21:31:17.163334+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163334+0000","last_undegraded":"2024-09-15T21:31:17.163334+0000","last_fullsized":"2024-09-15T21:31:17.163334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":22890,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:24.554563+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:24.554563+0000","last_peered":"2024-09-15T21:31:24.554563+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:24.554563+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1026,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163177+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163177+0000","last_peered":"2024-09-15T21:31:17.163177+0000","last_clean":"2024-09-15T21:31:17.163177+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163177+0000","last_undegraded":"2024-09-15T21:31:17.163177+0000","last_fullsized":"2024-09-15T21:31:17.163177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21772,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513912,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":513403,"kb_avail":93857928,"statfs":{"total":96636764160,"available":96110518272,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":525725359},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:25.844 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:26.141 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:26.141 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:26.153 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-15T21:31:24.890389+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21773,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1542332,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281573188,"statfs":{"total":289910292480,"available":288330944512,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1375259651},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1238,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.733678"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163016+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163016+0000","last_peered":"2024-09-15T21:31:17.163016+0000","last_clean":"2024-09-15T21:31:17.163016+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163016+0000","last_undegraded":"2024-09-15T21:31:17.163016+0000","last_fullsized":"2024-09-15T21:31:17.163016+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163334+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163334+0000","last_peered":"2024-09-15T21:31:17.163334+0000","last_clean":"2024-09-15T21:31:17.163334+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163334+0000","last_undegraded":"2024-09-15T21:31:17.163334+0000","last_fullsized":"2024-09-15T21:31:17.163334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":22890,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:24.554563+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:24.554563+0000","last_peered":"2024-09-15T21:31:24.554563+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:24.554563+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1026,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163177+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163177+0000","last_peered":"2024-09-15T21:31:17.163177+0000","last_clean":"2024-09-15T21:31:17.163177+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163177+0000","last_undegraded":"2024-09-15T21:31:17.163177+0000","last_fullsized":"2024-09-15T21:31:17.163177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21772,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513912,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":513403,"kb_avail":93857928,"statfs":{"total":96636764160,"available":96110518272,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":525725359},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:26.155 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:31:26.155 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21247, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.170590+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:31:17.170590+0000', 'last_peered': '2024-09-15T21:31:17.170590+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:31:17.170590+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:26.155 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:31:26.155 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20790, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.164628+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:31:17.164628+0000', 'last_peered': '2024-09-15T21:31:17.164628+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:31:17.164628+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:26.156 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:31:26.156 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20825, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.164412+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:31:17.164412+0000', 'last_peered': '2024-09-15T21:31:17.164412+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:31:17.164412+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:26.156 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:31:26.156 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20632, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.165428+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:31:17.165428+0000', 'last_peered': '2024-09-15T21:31:17.165428+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:31:17.165428+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:26.156 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:31:26.156 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20781, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.162389+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:31:17.162389+0000', 'last_peered': '2024-09-15T21:31:17.162389+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:31:17.162389+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:26.156 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:31:26.156 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 22890, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:24.554563+0000', 'last_change': '2024-09-15T21:31:15.546423+0000', 'last_active': '2024-09-15T21:31:24.554563+0000', 'last_peered': '2024-09-15T21:31:24.554563+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:31:24.554563+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1026, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3070}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:26.157 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:31:26.157 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21024, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.166895+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:31:17.166895+0000', 'last_peered': '2024-09-15T21:31:17.166895+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:31:17.166895+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:26.157 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:31:26.570 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:26.570 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27702/24 objects degraded (115425.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":738,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1579347968,"bytes_avail":288330944512,"bytes_total":289910292480,"degraded_objects":27702,"degraded_total":24,"degraded_ratio":1154.25,"recovering_objects_per_sec":141,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1238,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:31:26.584 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:31:26.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:26.946 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:26.947 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:26.958 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":415,"stamp":"2024-09-15T21:31:26.890828+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21773,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529852,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281585668,"statfs":{"total":289910292480,"available":288343724032,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1362480131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001948"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163016+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163016+0000","last_peered":"2024-09-15T21:31:17.163016+0000","last_clean":"2024-09-15T21:31:17.163016+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163016+0000","last_undegraded":"2024-09-15T21:31:17.163016+0000","last_fullsized":"2024-09-15T21:31:17.163016+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163334+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163334+0000","last_peered":"2024-09-15T21:31:17.163334+0000","last_clean":"2024-09-15T21:31:17.163334+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163334+0000","last_undegraded":"2024-09-15T21:31:17.163334+0000","last_fullsized":"2024-09-15T21:31:17.163334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":22890,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:24.554563+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:24.554563+0000","last_peered":"2024-09-15T21:31:24.554563+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:24.554563+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1026,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163177+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163177+0000","last_peered":"2024-09-15T21:31:17.163177+0000","last_clean":"2024-09-15T21:31:17.163177+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163177+0000","last_undegraded":"2024-09-15T21:31:17.163177+0000","last_fullsized":"2024-09-15T21:31:17.163177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21772,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:29.960 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:30.267 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:30.268 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:30.280 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-15T21:31:28.891201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21773,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529852,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281585668,"statfs":{"total":289910292480,"available":288343724032,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1362480131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.730676"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163016+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163016+0000","last_peered":"2024-09-15T21:31:17.163016+0000","last_clean":"2024-09-15T21:31:17.163016+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163016+0000","last_undegraded":"2024-09-15T21:31:17.163016+0000","last_fullsized":"2024-09-15T21:31:17.163016+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163334+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163334+0000","last_peered":"2024-09-15T21:31:17.163334+0000","last_clean":"2024-09-15T21:31:17.163334+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163334+0000","last_undegraded":"2024-09-15T21:31:17.163334+0000","last_fullsized":"2024-09-15T21:31:17.163334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":22890,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:24.554563+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:24.554563+0000","last_peered":"2024-09-15T21:31:24.554563+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:24.554563+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1026,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163177+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163177+0000","last_peered":"2024-09-15T21:31:17.163177+0000","last_clean":"2024-09-15T21:31:17.163177+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163177+0000","last_undegraded":"2024-09-15T21:31:17.163177+0000","last_fullsized":"2024-09-15T21:31:17.163177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21772,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:30.283 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:30.602 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:30.603 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:30.617 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-15T21:31:28.891201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21773,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529852,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281585668,"statfs":{"total":289910292480,"available":288343724032,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1362480131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.730676"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163016+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163016+0000","last_peered":"2024-09-15T21:31:17.163016+0000","last_clean":"2024-09-15T21:31:17.163016+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163016+0000","last_undegraded":"2024-09-15T21:31:17.163016+0000","last_fullsized":"2024-09-15T21:31:17.163016+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163334+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163334+0000","last_peered":"2024-09-15T21:31:17.163334+0000","last_clean":"2024-09-15T21:31:17.163334+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163334+0000","last_undegraded":"2024-09-15T21:31:17.163334+0000","last_fullsized":"2024-09-15T21:31:17.163334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":22890,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:24.554563+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:24.554563+0000","last_peered":"2024-09-15T21:31:24.554563+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:24.554563+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3070,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1026,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3070}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163177+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163177+0000","last_peered":"2024-09-15T21:31:17.163177+0000","last_clean":"2024-09-15T21:31:17.163177+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163177+0000","last_undegraded":"2024-09-15T21:31:17.163177+0000","last_fullsized":"2024-09-15T21:31:17.163177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27702,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21772,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:30.618 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:31:30.618 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21247, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.170590+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:31:17.170590+0000', 'last_peered': '2024-09-15T21:31:17.170590+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:31:17.170590+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:30.618 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:31:30.618 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20790, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.164628+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:31:17.164628+0000', 'last_peered': '2024-09-15T21:31:17.164628+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:31:17.164628+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:30.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:31:30.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20825, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.164412+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:31:17.164412+0000', 'last_peered': '2024-09-15T21:31:17.164412+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:31:17.164412+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:30.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:31:30.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20632, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.165428+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:31:17.165428+0000', 'last_peered': '2024-09-15T21:31:17.165428+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:31:17.165428+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:30.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:31:30.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20781, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.162389+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:31:17.162389+0000', 'last_peered': '2024-09-15T21:31:17.162389+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:31:17.162389+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:30.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:31:30.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 22890, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:24.554563+0000', 'last_change': '2024-09-15T21:31:15.546423+0000', 'last_active': '2024-09-15T21:31:24.554563+0000', 'last_peered': '2024-09-15T21:31:24.554563+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:31:24.554563+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3070, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1026, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3070}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:30.619 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:31:30.619 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21024, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.166895+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:31:17.166895+0000', 'last_peered': '2024-09-15T21:31:17.166895+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:31:17.166895+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:30.620 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:31:30.999 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:30.999 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27702/24 objects degraded (115425.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":742,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1566568448,"bytes_avail":288343724032,"bytes_total":289910292480,"degraded_objects":27702,"degraded_total":24,"degraded_ratio":1154.25,"recovering_objects_per_sec":87,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1025,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:31:31.013 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:31:31.013 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:31.321 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:31.321 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:31.332 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":417,"stamp":"2024-09-15T21:31:30.891538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27080,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22395,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529852,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281585668,"statfs":{"total":289910292480,"available":288343724032,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1362480131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1647,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002201"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163016+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163016+0000","last_peered":"2024-09-15T21:31:17.163016+0000","last_clean":"2024-09-15T21:31:17.163016+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163016+0000","last_undegraded":"2024-09-15T21:31:17.163016+0000","last_fullsized":"2024-09-15T21:31:17.163016+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163334+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163334+0000","last_peered":"2024-09-15T21:31:17.163334+0000","last_clean":"2024-09-15T21:31:17.163334+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163334+0000","last_undegraded":"2024-09-15T21:31:17.163334+0000","last_fullsized":"2024-09-15T21:31:17.163334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":24134,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:29.557099+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:29.557099+0000","last_peered":"2024-09-15T21:31:29.557099+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:29.557099+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1648,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2448}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163177+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163177+0000","last_peered":"2024-09-15T21:31:17.163177+0000","last_clean":"2024-09-15T21:31:17.163177+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163177+0000","last_undegraded":"2024-09-15T21:31:17.163177+0000","last_fullsized":"2024-09-15T21:31:17.163177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27080,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22394,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:34.334 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:34.665 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:34.666 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:34.679 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-09-15T21:31:32.891940+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27080,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22395,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534012,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281581508,"statfs":{"total":289910292480,"available":288339464192,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1366739971},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1199,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1199,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002235"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163016+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163016+0000","last_peered":"2024-09-15T21:31:17.163016+0000","last_clean":"2024-09-15T21:31:17.163016+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163016+0000","last_undegraded":"2024-09-15T21:31:17.163016+0000","last_fullsized":"2024-09-15T21:31:17.163016+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163334+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163334+0000","last_peered":"2024-09-15T21:31:17.163334+0000","last_clean":"2024-09-15T21:31:17.163334+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163334+0000","last_undegraded":"2024-09-15T21:31:17.163334+0000","last_fullsized":"2024-09-15T21:31:17.163334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":24134,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:29.557099+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:29.557099+0000","last_peered":"2024-09-15T21:31:29.557099+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:29.557099+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1648,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2448}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163177+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163177+0000","last_peered":"2024-09-15T21:31:17.163177+0000","last_clean":"2024-09-15T21:31:17.163177+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163177+0000","last_undegraded":"2024-09-15T21:31:17.163177+0000","last_fullsized":"2024-09-15T21:31:17.163177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27080,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22394,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505592,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866248,"statfs":{"total":96636764160,"available":96119037952,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:34.680 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:34.984 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:34.985 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:34.997 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-15T21:31:34.892305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23134,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534012,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281581508,"statfs":{"total":289910292480,"available":288339464192,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1366739971},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1938,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002261"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20404,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163057+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163057+0000","last_peered":"2024-09-15T21:31:17.163057+0000","last_clean":"2024-09-15T21:31:17.163057+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163057+0000","last_undegraded":"2024-09-15T21:31:17.163057+0000","last_fullsized":"2024-09-15T21:31:17.163057+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163334+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163334+0000","last_peered":"2024-09-15T21:31:17.163334+0000","last_clean":"2024-09-15T21:31:17.163334+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163334+0000","last_undegraded":"2024-09-15T21:31:17.163334+0000","last_fullsized":"2024-09-15T21:31:17.163334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":25612,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:34.555097+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:34.555097+0000","last_peered":"2024-09-15T21:31:34.555097+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:34.555097+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2387,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1709}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163177+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163177+0000","last_peered":"2024-09-15T21:31:17.163177+0000","last_clean":"2024-09-15T21:31:17.163177+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163177+0000","last_undegraded":"2024-09-15T21:31:17.163177+0000","last_fullsized":"2024-09-15T21:31:17.163177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23133,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505592,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866248,"statfs":{"total":96636764160,"available":96119037952,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:34.998 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:31:34.998 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21247, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.170590+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:31:17.170590+0000', 'last_peered': '2024-09-15T21:31:17.170590+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:31:17.170590+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:34.999 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:31:34.999 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20790, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.164628+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:31:17.164628+0000', 'last_peered': '2024-09-15T21:31:17.164628+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:31:17.164628+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:34.999 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:31:34.999 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20825, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.164412+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:31:17.164412+0000', 'last_peered': '2024-09-15T21:31:17.164412+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:31:17.164412+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:34.999 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:31:34.999 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20632, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.165428+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:31:17.165428+0000', 'last_peered': '2024-09-15T21:31:17.165428+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:31:17.165428+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:34.999 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:31:34.999 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20781, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.162389+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:31:17.162389+0000', 'last_peered': '2024-09-15T21:31:17.162389+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:31:17.162389+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:34.999 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:31:35.000 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 25612, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:34.555097+0000', 'last_change': '2024-09-15T21:31:15.546423+0000', 'last_active': '2024-09-15T21:31:34.555097+0000', 'last_peered': '2024-09-15T21:31:34.555097+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:31:34.555097+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1709, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2387, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1709}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:35.000 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:31:35.000 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21024, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.166895+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:31:17.166895+0000', 'last_peered': '2024-09-15T21:31:17.166895+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:31:17.166895+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:35.000 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:31:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:35.395 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26341/24 objects degraded (109754.167%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":747,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1570828288,"bytes_avail":288339464192,"bytes_total":289910292480,"degraded_objects":26341,"degraded_total":24,"degraded_ratio":1097.5416666666667,"recovering_objects_per_sec":161,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1938,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:31:35.409 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:31:35.409 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:35.720 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:35.720 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:35.733 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-15T21:31:34.892305+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23134,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534012,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281581508,"statfs":{"total":289910292480,"available":288339464192,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1366739971},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1938,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002261"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20404,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163057+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163057+0000","last_peered":"2024-09-15T21:31:17.163057+0000","last_clean":"2024-09-15T21:31:17.163057+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163057+0000","last_undegraded":"2024-09-15T21:31:17.163057+0000","last_fullsized":"2024-09-15T21:31:17.163057+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163334+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163334+0000","last_peered":"2024-09-15T21:31:17.163334+0000","last_clean":"2024-09-15T21:31:17.163334+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163334+0000","last_undegraded":"2024-09-15T21:31:17.163334+0000","last_fullsized":"2024-09-15T21:31:17.163334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":25612,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:34.555097+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:34.555097+0000","last_peered":"2024-09-15T21:31:34.555097+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:34.555097+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2387,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1709}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163177+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163177+0000","last_peered":"2024-09-15T21:31:17.163177+0000","last_clean":"2024-09-15T21:31:17.163177+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163177+0000","last_undegraded":"2024-09-15T21:31:17.163177+0000","last_fullsized":"2024-09-15T21:31:17.163177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23133,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505592,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866248,"statfs":{"total":96636764160,"available":96119037952,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:38.736 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:39.064 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:39.077 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-15T21:31:38.893199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23134,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538172,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577348,"statfs":{"total":289910292480,"available":288335204352,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002371"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20404,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163057+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163057+0000","last_peered":"2024-09-15T21:31:17.163057+0000","last_clean":"2024-09-15T21:31:17.163057+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163057+0000","last_undegraded":"2024-09-15T21:31:17.163057+0000","last_fullsized":"2024-09-15T21:31:17.163057+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163334+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163334+0000","last_peered":"2024-09-15T21:31:17.163334+0000","last_clean":"2024-09-15T21:31:17.163334+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163334+0000","last_undegraded":"2024-09-15T21:31:17.163334+0000","last_fullsized":"2024-09-15T21:31:17.163334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":25612,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:34.555097+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:34.555097+0000","last_peered":"2024-09-15T21:31:34.555097+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:34.555097+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2387,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1709}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163177+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163177+0000","last_peered":"2024-09-15T21:31:17.163177+0000","last_clean":"2024-09-15T21:31:17.163177+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163177+0000","last_undegraded":"2024-09-15T21:31:17.163177+0000","last_fullsized":"2024-09-15T21:31:17.163177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23133,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509752,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862088,"statfs":{"total":96636764160,"available":96114778112,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:39.079 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:39.429 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:39.429 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:39.441 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-15T21:31:38.893199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23134,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538172,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577348,"statfs":{"total":289910292480,"available":288335204352,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002371"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20404,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163057+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163057+0000","last_peered":"2024-09-15T21:31:17.163057+0000","last_clean":"2024-09-15T21:31:17.163057+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163057+0000","last_undegraded":"2024-09-15T21:31:17.163057+0000","last_fullsized":"2024-09-15T21:31:17.163057+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163334+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163334+0000","last_peered":"2024-09-15T21:31:17.163334+0000","last_clean":"2024-09-15T21:31:17.163334+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163334+0000","last_undegraded":"2024-09-15T21:31:17.163334+0000","last_fullsized":"2024-09-15T21:31:17.163334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":25612,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:34.555097+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:34.555097+0000","last_peered":"2024-09-15T21:31:34.555097+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:34.555097+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2387,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1709}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163177+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163177+0000","last_peered":"2024-09-15T21:31:17.163177+0000","last_clean":"2024-09-15T21:31:17.163177+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163177+0000","last_undegraded":"2024-09-15T21:31:17.163177+0000","last_fullsized":"2024-09-15T21:31:17.163177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23133,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509752,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862088,"statfs":{"total":96636764160,"available":96114778112,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:39.442 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:31:39.443 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21247, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.170590+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:31:17.170590+0000', 'last_peered': '2024-09-15T21:31:17.170590+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:31:17.170590+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:39.443 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:31:39.443 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20790, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.164628+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:31:17.164628+0000', 'last_peered': '2024-09-15T21:31:17.164628+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:31:17.164628+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:39.443 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:31:39.443 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20825, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.164412+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:31:17.164412+0000', 'last_peered': '2024-09-15T21:31:17.164412+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:31:17.164412+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:39.443 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:31:39.443 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20632, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.165428+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:31:17.165428+0000', 'last_peered': '2024-09-15T21:31:17.165428+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:31:17.165428+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:39.443 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:31:39.443 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20781, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.162389+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:31:17.162389+0000', 'last_peered': '2024-09-15T21:31:17.162389+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:31:17.162389+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:39.443 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:31:39.444 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 25612, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:34.555097+0000', 'last_change': '2024-09-15T21:31:15.546423+0000', 'last_active': '2024-09-15T21:31:34.555097+0000', 'last_peered': '2024-09-15T21:31:34.555097+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:31:34.555097+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1709, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2387, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1709}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:39.444 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:31:39.444 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21024, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.166895+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:31:17.166895+0000', 'last_peered': '2024-09-15T21:31:17.166895+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:31:17.166895+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:39.444 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:31:39.858 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:39.859 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26341/24 objects degraded (109754.167%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":751,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1575088128,"bytes_avail":288335204352,"bytes_total":289910292480,"degraded_objects":26341,"degraded_total":24,"degraded_ratio":1097.5416666666667,"recovering_objects_per_sec":113,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1361,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:31:39.873 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:31:39.874 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:40.190 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:40.190 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:40.202 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-15T21:31:38.893199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23134,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538172,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577348,"statfs":{"total":289910292480,"available":288335204352,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1361,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002371"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20404,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163057+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163057+0000","last_peered":"2024-09-15T21:31:17.163057+0000","last_clean":"2024-09-15T21:31:17.163057+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163057+0000","last_undegraded":"2024-09-15T21:31:17.163057+0000","last_fullsized":"2024-09-15T21:31:17.163057+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163334+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163334+0000","last_peered":"2024-09-15T21:31:17.163334+0000","last_clean":"2024-09-15T21:31:17.163334+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163334+0000","last_undegraded":"2024-09-15T21:31:17.163334+0000","last_fullsized":"2024-09-15T21:31:17.163334+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":25612,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:34.555097+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:34.555097+0000","last_peered":"2024-09-15T21:31:34.555097+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:34.555097+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1709,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2387,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1709}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163177+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163177+0000","last_peered":"2024-09-15T21:31:17.163177+0000","last_clean":"2024-09-15T21:31:17.163177+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163177+0000","last_undegraded":"2024-09-15T21:31:17.163177+0000","last_fullsized":"2024-09-15T21:31:17.163177+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26341,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23133,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509752,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862088,"statfs":{"total":96636764160,"available":96114778112,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:43.204 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:43.551 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:43.551 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:43.564 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-09-15T21:31:43.236042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25413,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24062,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529852,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281585668,"statfs":{"total":289910292480,"available":288343724032,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1362480131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1667,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.344102"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20404,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163057+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163057+0000","last_peered":"2024-09-15T21:31:17.163057+0000","last_clean":"2024-09-15T21:31:17.163057+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163057+0000","last_undegraded":"2024-09-15T21:31:17.163057+0000","last_fullsized":"2024-09-15T21:31:17.163057+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163370+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163370+0000","last_peered":"2024-09-15T21:31:17.163370+0000","last_clean":"2024-09-15T21:31:17.163370+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163370+0000","last_undegraded":"2024-09-15T21:31:17.163370+0000","last_fullsized":"2024-09-15T21:31:17.163370+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":27468,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:39.559673+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:39.559673+0000","last_peered":"2024-09-15T21:31:39.559673+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:39.559673+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":781,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3315,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-781}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163214+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163214+0000","last_peered":"2024-09-15T21:31:17.163214+0000","last_clean":"2024-09-15T21:31:17.163214+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163214+0000","last_undegraded":"2024-09-15T21:31:17.163214+0000","last_fullsized":"2024-09-15T21:31:17.163214+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25413,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24061,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:43.565 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:43.897 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:43.909 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":424,"stamp":"2024-09-15T21:31:43.236042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25413,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24062,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529852,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281585668,"statfs":{"total":289910292480,"available":288343724032,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1362480131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1667,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.344102"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20404,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163057+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163057+0000","last_peered":"2024-09-15T21:31:17.163057+0000","last_clean":"2024-09-15T21:31:17.163057+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163057+0000","last_undegraded":"2024-09-15T21:31:17.163057+0000","last_fullsized":"2024-09-15T21:31:17.163057+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163370+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163370+0000","last_peered":"2024-09-15T21:31:17.163370+0000","last_clean":"2024-09-15T21:31:17.163370+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163370+0000","last_undegraded":"2024-09-15T21:31:17.163370+0000","last_fullsized":"2024-09-15T21:31:17.163370+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":27468,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:39.559673+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:39.559673+0000","last_peered":"2024-09-15T21:31:39.559673+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:39.559673+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":781,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3315,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-781}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163214+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163214+0000","last_peered":"2024-09-15T21:31:17.163214+0000","last_clean":"2024-09-15T21:31:17.163214+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163214+0000","last_undegraded":"2024-09-15T21:31:17.163214+0000","last_fullsized":"2024-09-15T21:31:17.163214+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25413,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24061,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:43.911 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:31:43.911 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21247, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.170590+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:31:17.170590+0000', 'last_peered': '2024-09-15T21:31:17.170590+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:31:17.170590+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:43.912 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:31:43.912 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20790, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.164628+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:31:17.164628+0000', 'last_peered': '2024-09-15T21:31:17.164628+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:31:17.164628+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:43.912 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:31:43.912 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20825, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.164412+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:31:17.164412+0000', 'last_peered': '2024-09-15T21:31:17.164412+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:31:17.164412+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:43.912 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:31:43.912 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20632, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.165428+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:31:17.165428+0000', 'last_peered': '2024-09-15T21:31:17.165428+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:31:17.165428+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:43.912 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:31:43.912 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 20781, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.162389+0000', 'last_change': '2024-09-15T21:21:14.881950+0000', 'last_active': '2024-09-15T21:31:17.162389+0000', 'last_peered': '2024-09-15T21:31:17.162389+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:31:17.162389+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4091, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1}, {'shards': '2', 'objects': -4089}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:43.912 INFO:tasks.lost_unfound.ceph_manager:PG 2.7 is not active+clean 2024-09-15T21:31:43.913 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.7', 'version': "49'20686", 'reported_seq': 27468, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:39.559673+0000', 'last_change': '2024-09-15T21:31:15.546423+0000', 'last_active': '2024-09-15T21:31:39.559673+0000', 'last_peered': '2024-09-15T21:31:39.559673+0000', 'last_clean': '2024-09-15T21:21:12.333121+0000', 'last_became_active': '2024-09-15T21:21:14.874754+0000', 'last_became_peered': '2024-09-15T21:21:14.874754+0000', 'last_unstale': '2024-09-15T21:31:39.559673+0000', 'last_undegraded': '2024-09-15T21:21:14.456521+0000', 'last_fullsized': '2024-09-15T21:21:14.356664+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3101", 'last_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:57.978134+0000', 'objects_scrubbed': 3054, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1164, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 781, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17630, 'num_write_kb': 29152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3315, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -781}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:43.913 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:31:43.913 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21024, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:17.166895+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:31:17.166895+0000', 'last_peered': '2024-09-15T21:31:17.166895+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:31:17.166895+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:43.913 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:31:44.386 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:44.386 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25413/24 objects degraded (105887.500%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":756,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1566568448,"bytes_avail":288343724032,"bytes_total":289910292480,"degraded_objects":25413,"degraded_total":24,"degraded_ratio":1058.875,"recovering_objects_per_sec":138,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1667,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:31:44.400 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:31:44.400 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:44.734 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:44.735 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:44.749 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-15T21:31:44.247108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25413,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24062,"num_bytes_recovered":32629238,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529852,"kb_used_data":199232,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281585668,"statfs":{"total":289910292480,"available":288343724032,"internally_reserved":0,"allocated":204013568,"data_stored":203063618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1362480131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-928,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":928,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.354802"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21247,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.170590+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:17.170590+0000","last_peered":"2024-09-15T21:31:17.170590+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:17.170590+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20798,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162731+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:17.162731+0000","last_peered":"2024-09-15T21:31:17.162731+0000","last_clean":"2024-09-15T21:31:17.162731+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:17.162731+0000","last_undegraded":"2024-09-15T21:31:17.162731+0000","last_fullsized":"2024-09-15T21:31:17.162731+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29797,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163063+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:17.163063+0000","last_peered":"2024-09-15T21:31:17.163063+0000","last_clean":"2024-09-15T21:31:17.163063+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:17.163063+0000","last_undegraded":"2024-09-15T21:31:17.163063+0000","last_fullsized":"2024-09-15T21:31:17.163063+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20404,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163057+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:17.163057+0000","last_peered":"2024-09-15T21:31:17.163057+0000","last_clean":"2024-09-15T21:31:17.163057+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:17.163057+0000","last_undegraded":"2024-09-15T21:31:17.163057+0000","last_fullsized":"2024-09-15T21:31:17.163057+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20657,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.166787+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:17.166787+0000","last_peered":"2024-09-15T21:31:17.166787+0000","last_clean":"2024-09-15T21:31:17.166787+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:17.166787+0000","last_undegraded":"2024-09-15T21:31:17.166787+0000","last_fullsized":"2024-09-15T21:31:17.166787+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20790,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164628+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:17.164628+0000","last_peered":"2024-09-15T21:31:17.164628+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:17.164628+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20825,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.164412+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:17.164412+0000","last_peered":"2024-09-15T21:31:17.164412+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:17.164412+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21016,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163956+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:17.163956+0000","last_peered":"2024-09-15T21:31:17.163956+0000","last_clean":"2024-09-15T21:31:17.163956+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:17.163956+0000","last_undegraded":"2024-09-15T21:31:17.163956+0000","last_fullsized":"2024-09-15T21:31:17.163956+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162429+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:17.162429+0000","last_peered":"2024-09-15T21:31:17.162429+0000","last_clean":"2024-09-15T21:31:17.162429+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:17.162429+0000","last_undegraded":"2024-09-15T21:31:17.162429+0000","last_fullsized":"2024-09-15T21:31:17.162429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20663,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.261191+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:17.261191+0000","last_peered":"2024-09-15T21:31:17.261191+0000","last_clean":"2024-09-15T21:31:17.261191+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:17.261191+0000","last_undegraded":"2024-09-15T21:31:17.261191+0000","last_fullsized":"2024-09-15T21:31:17.261191+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162335+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:17.162335+0000","last_peered":"2024-09-15T21:31:17.162335+0000","last_clean":"2024-09-15T21:31:17.162335+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:17.162335+0000","last_undegraded":"2024-09-15T21:31:17.162335+0000","last_fullsized":"2024-09-15T21:31:17.162335+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20632,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.165428+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:17.165428+0000","last_peered":"2024-09-15T21:31:17.165428+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:17.165428+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.164528+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:17.164528+0000","last_peered":"2024-09-15T21:31:17.164528+0000","last_clean":"2024-09-15T21:31:17.164528+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:17.164528+0000","last_undegraded":"2024-09-15T21:31:17.164528+0000","last_fullsized":"2024-09-15T21:31:17.164528+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":20781,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.162389+0000","last_change":"2024-09-15T21:21:14.881950+0000","last_active":"2024-09-15T21:31:17.162389+0000","last_peered":"2024-09-15T21:31:17.162389+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:17.162389+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4091,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1},{"shards":"2","objects":-4089}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163370+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:17.163370+0000","last_peered":"2024-09-15T21:31:17.163370+0000","last_clean":"2024-09-15T21:31:17.163370+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:17.163370+0000","last_undegraded":"2024-09-15T21:31:17.163370+0000","last_fullsized":"2024-09-15T21:31:17.163370+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29038,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.162206+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:17.162206+0000","last_peered":"2024-09-15T21:31:17.162206+0000","last_clean":"2024-09-15T21:31:17.162206+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:17.162206+0000","last_undegraded":"2024-09-15T21:31:17.162206+0000","last_fullsized":"2024-09-15T21:31:17.162206+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":27468,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:39.559673+0000","last_change":"2024-09-15T21:31:15.546423+0000","last_active":"2024-09-15T21:31:39.559673+0000","last_peered":"2024-09-15T21:31:39.559673+0000","last_clean":"2024-09-15T21:21:12.333121+0000","last_became_active":"2024-09-15T21:21:14.874754+0000","last_became_peered":"2024-09-15T21:21:14.874754+0000","last_unstale":"2024-09-15T21:31:39.559673+0000","last_undegraded":"2024-09-15T21:21:14.456521+0000","last_fullsized":"2024-09-15T21:21:14.356664+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":781,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3315,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-781}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163214+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:17.163214+0000","last_peered":"2024-09-15T21:31:17.163214+0000","last_clean":"2024-09-15T21:31:17.163214+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:17.163214+0000","last_undegraded":"2024-09-15T21:31:17.163214+0000","last_fullsized":"2024-09-15T21:31:17.163214+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21601,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163422+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:17.163422+0000","last_peered":"2024-09-15T21:31:17.163422+0000","last_clean":"2024-09-15T21:31:17.163422+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:17.163422+0000","last_undegraded":"2024-09-15T21:31:17.163422+0000","last_fullsized":"2024-09-15T21:31:17.163422+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163160+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:17.163160+0000","last_peered":"2024-09-15T21:31:17.163160+0000","last_clean":"2024-09-15T21:31:17.163160+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:17.163160+0000","last_undegraded":"2024-09-15T21:31:17.163160+0000","last_fullsized":"2024-09-15T21:31:17.163160+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28717,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163124+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:17.163124+0000","last_peered":"2024-09-15T21:31:17.163124+0000","last_clean":"2024-09-15T21:31:17.163124+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:17.163124+0000","last_undegraded":"2024-09-15T21:31:17.163124+0000","last_fullsized":"2024-09-15T21:31:17.163124+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163179+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:17.163179+0000","last_peered":"2024-09-15T21:31:17.163179+0000","last_clean":"2024-09-15T21:31:17.163179+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:17.163179+0000","last_undegraded":"2024-09-15T21:31:17.163179+0000","last_fullsized":"2024-09-15T21:31:17.163179+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-15T21:31:17.163940+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:17.163940+0000","last_peered":"2024-09-15T21:31:17.163940+0000","last_clean":"2024-09-15T21:31:17.163940+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:17.163940+0000","last_undegraded":"2024-09-15T21:31:17.163940+0000","last_fullsized":"2024-09-15T21:31:17.163940+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21024,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:17.166895+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:17.166895+0000","last_peered":"2024-09-15T21:31:17.166895+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:17.166895+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25413,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24061,"num_bytes_recovered":32629219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202678272,"data_stored":202559043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492588,"kb_used_data":131860,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879252,"statfs":{"total":96636764160,"available":96132354048,"internally_reserved":0,"allocated":135024640,"data_stored":134714321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535832,"kb_used_data":66884,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836008,"statfs":{"total":96636764160,"available":96088072192,"internally_reserved":0,"allocated":68489216,"data_stored":68174820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":499712,"data_stored":174477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:47.752 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:48.101 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:48.102 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:48.114 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-15T21:31:46.894556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24315,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25160,"num_bytes_recovered":32629721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538224,"kb_used_data":199276,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577296,"statfs":{"total":289910292480,"available":288335151104,"internally_reserved":0,"allocated":204058624,"data_stored":203082854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2026,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001356"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21251,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248456+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:44.248456+0000","last_peered":"2024-09-15T21:31:44.248456+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:44.248456+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20802,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249719+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:44.249719+0000","last_peered":"2024-09-15T21:31:44.249719+0000","last_clean":"2024-09-15T21:31:44.249719+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:44.249719+0000","last_undegraded":"2024-09-15T21:31:44.249719+0000","last_fullsized":"2024-09-15T21:31:44.249719+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20408,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:44.247848+0000","last_peered":"2024-09-15T21:31:44.247848+0000","last_clean":"2024-09-15T21:31:44.247848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:44.247848+0000","last_undegraded":"2024-09-15T21:31:44.247848+0000","last_fullsized":"2024-09-15T21:31:44.247848+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20661,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250468+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:44.250468+0000","last_peered":"2024-09-15T21:31:44.250468+0000","last_clean":"2024-09-15T21:31:44.250468+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:44.250468+0000","last_undegraded":"2024-09-15T21:31:44.250468+0000","last_fullsized":"2024-09-15T21:31:44.250468+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20794,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250325+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:44.250325+0000","last_peered":"2024-09-15T21:31:44.250325+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:44.250325+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20829,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248602+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:44.248602+0000","last_peered":"2024-09-15T21:31:44.248602+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:44.248602+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21020,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248406+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:44.248406+0000","last_peered":"2024-09-15T21:31:44.248406+0000","last_clean":"2024-09-15T21:31:44.248406+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:44.248406+0000","last_undegraded":"2024-09-15T21:31:44.248406+0000","last_fullsized":"2024-09-15T21:31:44.248406+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248028+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:44.248028+0000","last_peered":"2024-09-15T21:31:44.248028+0000","last_clean":"2024-09-15T21:31:44.248028+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:44.248028+0000","last_undegraded":"2024-09-15T21:31:44.248028+0000","last_fullsized":"2024-09-15T21:31:44.248028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20636,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250586+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:44.250586+0000","last_peered":"2024-09-15T21:31:44.250586+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:44.250586+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250232+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:44.250232+0000","last_peered":"2024-09-15T21:31:44.250232+0000","last_clean":"2024-09-15T21:31:44.250232+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:44.250232+0000","last_undegraded":"2024-09-15T21:31:44.250232+0000","last_fullsized":"2024-09-15T21:31:44.250232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":21420,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.770877+0000","last_change":"2024-09-15T21:31:42.557303+0000","last_active":"2024-09-15T21:31:44.770877+0000","last_peered":"2024-09-15T21:31:44.770877+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:44.770877+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3774,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3773}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29042,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247910+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:44.247910+0000","last_peered":"2024-09-15T21:31:44.247910+0000","last_clean":"2024-09-15T21:31:44.247910+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:44.247910+0000","last_undegraded":"2024-09-15T21:31:44.247910+0000","last_fullsized":"2024-09-15T21:31:44.247910+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249618+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:44.249618+0000","last_peered":"2024-09-15T21:31:44.249618+0000","last_clean":"2024-09-15T21:31:44.249618+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:44.249618+0000","last_undegraded":"2024-09-15T21:31:44.249618+0000","last_fullsized":"2024-09-15T21:31:44.249618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:44.250878+0000","last_peered":"2024-09-15T21:31:44.250878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:44.250878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24315,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25159,"num_bytes_recovered":32629702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202674176,"data_stored":202558560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492604,"kb_used_data":131876,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879236,"statfs":{"total":96636764160,"available":96132337664,"internally_reserved":0,"allocated":135041024,"data_stored":134720894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509772,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862068,"statfs":{"total":96636764160,"available":96114757632,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:48.115 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:48.474 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:48.475 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:48.487 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-09-15T21:31:46.894556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24315,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25160,"num_bytes_recovered":32629721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538224,"kb_used_data":199276,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577296,"statfs":{"total":289910292480,"available":288335151104,"internally_reserved":0,"allocated":204058624,"data_stored":203082854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2026,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001356"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21251,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248456+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:44.248456+0000","last_peered":"2024-09-15T21:31:44.248456+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:44.248456+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20802,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249719+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:44.249719+0000","last_peered":"2024-09-15T21:31:44.249719+0000","last_clean":"2024-09-15T21:31:44.249719+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:44.249719+0000","last_undegraded":"2024-09-15T21:31:44.249719+0000","last_fullsized":"2024-09-15T21:31:44.249719+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20408,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:44.247848+0000","last_peered":"2024-09-15T21:31:44.247848+0000","last_clean":"2024-09-15T21:31:44.247848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:44.247848+0000","last_undegraded":"2024-09-15T21:31:44.247848+0000","last_fullsized":"2024-09-15T21:31:44.247848+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20661,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250468+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:44.250468+0000","last_peered":"2024-09-15T21:31:44.250468+0000","last_clean":"2024-09-15T21:31:44.250468+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:44.250468+0000","last_undegraded":"2024-09-15T21:31:44.250468+0000","last_fullsized":"2024-09-15T21:31:44.250468+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20794,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250325+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:44.250325+0000","last_peered":"2024-09-15T21:31:44.250325+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:44.250325+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20829,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248602+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:44.248602+0000","last_peered":"2024-09-15T21:31:44.248602+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:44.248602+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21020,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248406+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:44.248406+0000","last_peered":"2024-09-15T21:31:44.248406+0000","last_clean":"2024-09-15T21:31:44.248406+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:44.248406+0000","last_undegraded":"2024-09-15T21:31:44.248406+0000","last_fullsized":"2024-09-15T21:31:44.248406+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248028+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:44.248028+0000","last_peered":"2024-09-15T21:31:44.248028+0000","last_clean":"2024-09-15T21:31:44.248028+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:44.248028+0000","last_undegraded":"2024-09-15T21:31:44.248028+0000","last_fullsized":"2024-09-15T21:31:44.248028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20636,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250586+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:44.250586+0000","last_peered":"2024-09-15T21:31:44.250586+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:44.250586+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250232+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:44.250232+0000","last_peered":"2024-09-15T21:31:44.250232+0000","last_clean":"2024-09-15T21:31:44.250232+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:44.250232+0000","last_undegraded":"2024-09-15T21:31:44.250232+0000","last_fullsized":"2024-09-15T21:31:44.250232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":21420,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.770877+0000","last_change":"2024-09-15T21:31:42.557303+0000","last_active":"2024-09-15T21:31:44.770877+0000","last_peered":"2024-09-15T21:31:44.770877+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:44.770877+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3774,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3773}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29042,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247910+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:44.247910+0000","last_peered":"2024-09-15T21:31:44.247910+0000","last_clean":"2024-09-15T21:31:44.247910+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:44.247910+0000","last_undegraded":"2024-09-15T21:31:44.247910+0000","last_fullsized":"2024-09-15T21:31:44.247910+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249618+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:44.249618+0000","last_peered":"2024-09-15T21:31:44.249618+0000","last_clean":"2024-09-15T21:31:44.249618+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:44.249618+0000","last_undegraded":"2024-09-15T21:31:44.249618+0000","last_fullsized":"2024-09-15T21:31:44.249618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:44.250878+0000","last_peered":"2024-09-15T21:31:44.250878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:44.250878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24315,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25159,"num_bytes_recovered":32629702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202674176,"data_stored":202558560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492604,"kb_used_data":131876,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879236,"statfs":{"total":96636764160,"available":96132337664,"internally_reserved":0,"allocated":135041024,"data_stored":134720894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509772,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862068,"statfs":{"total":96636764160,"available":96114757632,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:48.488 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:31:48.488 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21251, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.248456+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:31:44.248456+0000', 'last_peered': '2024-09-15T21:31:44.248456+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:31:44.248456+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:48.488 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:31:48.488 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20794, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.250325+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:31:44.250325+0000', 'last_peered': '2024-09-15T21:31:44.250325+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:31:44.250325+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:48.489 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:31:48.489 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20829, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.248602+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:31:44.248602+0000', 'last_peered': '2024-09-15T21:31:44.248602+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:31:44.248602+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:48.489 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:31:48.489 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20636, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.250586+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:31:44.250586+0000', 'last_peered': '2024-09-15T21:31:44.250586+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:31:44.250586+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:48.489 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:31:48.489 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 21420, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.770877+0000', 'last_change': '2024-09-15T21:31:42.557303+0000', 'last_active': '2024-09-15T21:31:44.770877+0000', 'last_peered': '2024-09-15T21:31:44.770877+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:31:44.770877+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3774, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 317, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3773}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:48.489 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:31:48.489 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21028, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.250878+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:31:44.250878+0000', 'last_peered': '2024-09-15T21:31:44.250878+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:31:44.250878+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:48.490 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:31:48.917 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:48.917 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24315/24 objects degraded (101312.500%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":760,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1575141376,"bytes_avail":288335151104,"bytes_total":289910292480,"degraded_objects":24315,"degraded_total":24,"degraded_ratio":1013.125,"recovering_objects_per_sec":253,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":2026,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:31:48.931 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:31:48.931 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:49.291 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:49.292 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:49.304 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-09-15T21:31:48.894929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24315,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25160,"num_bytes_recovered":32629721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538224,"kb_used_data":199276,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577296,"statfs":{"total":289910292480,"available":288335151104,"internally_reserved":0,"allocated":204058624,"data_stored":203082854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1098,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001431"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21251,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248456+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:44.248456+0000","last_peered":"2024-09-15T21:31:44.248456+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:44.248456+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20802,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249719+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:44.249719+0000","last_peered":"2024-09-15T21:31:44.249719+0000","last_clean":"2024-09-15T21:31:44.249719+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:44.249719+0000","last_undegraded":"2024-09-15T21:31:44.249719+0000","last_fullsized":"2024-09-15T21:31:44.249719+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20408,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:44.247848+0000","last_peered":"2024-09-15T21:31:44.247848+0000","last_clean":"2024-09-15T21:31:44.247848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:44.247848+0000","last_undegraded":"2024-09-15T21:31:44.247848+0000","last_fullsized":"2024-09-15T21:31:44.247848+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20661,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250468+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:44.250468+0000","last_peered":"2024-09-15T21:31:44.250468+0000","last_clean":"2024-09-15T21:31:44.250468+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:44.250468+0000","last_undegraded":"2024-09-15T21:31:44.250468+0000","last_fullsized":"2024-09-15T21:31:44.250468+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20794,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250325+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:44.250325+0000","last_peered":"2024-09-15T21:31:44.250325+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:44.250325+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20829,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248602+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:44.248602+0000","last_peered":"2024-09-15T21:31:44.248602+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:44.248602+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21020,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248406+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:44.248406+0000","last_peered":"2024-09-15T21:31:44.248406+0000","last_clean":"2024-09-15T21:31:44.248406+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:44.248406+0000","last_undegraded":"2024-09-15T21:31:44.248406+0000","last_fullsized":"2024-09-15T21:31:44.248406+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248028+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:44.248028+0000","last_peered":"2024-09-15T21:31:44.248028+0000","last_clean":"2024-09-15T21:31:44.248028+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:44.248028+0000","last_undegraded":"2024-09-15T21:31:44.248028+0000","last_fullsized":"2024-09-15T21:31:44.248028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20636,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250586+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:44.250586+0000","last_peered":"2024-09-15T21:31:44.250586+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:44.250586+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250232+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:44.250232+0000","last_peered":"2024-09-15T21:31:44.250232+0000","last_clean":"2024-09-15T21:31:44.250232+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:44.250232+0000","last_undegraded":"2024-09-15T21:31:44.250232+0000","last_fullsized":"2024-09-15T21:31:44.250232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":21420,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.770877+0000","last_change":"2024-09-15T21:31:42.557303+0000","last_active":"2024-09-15T21:31:44.770877+0000","last_peered":"2024-09-15T21:31:44.770877+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:44.770877+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3774,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3773}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29042,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247910+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:44.247910+0000","last_peered":"2024-09-15T21:31:44.247910+0000","last_clean":"2024-09-15T21:31:44.247910+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:44.247910+0000","last_undegraded":"2024-09-15T21:31:44.247910+0000","last_fullsized":"2024-09-15T21:31:44.247910+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249618+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:44.249618+0000","last_peered":"2024-09-15T21:31:44.249618+0000","last_clean":"2024-09-15T21:31:44.249618+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:44.249618+0000","last_undegraded":"2024-09-15T21:31:44.249618+0000","last_fullsized":"2024-09-15T21:31:44.249618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:44.250878+0000","last_peered":"2024-09-15T21:31:44.250878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:44.250878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24315,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25159,"num_bytes_recovered":32629702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202674176,"data_stored":202558560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492604,"kb_used_data":131876,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879236,"statfs":{"total":96636764160,"available":96132337664,"internally_reserved":0,"allocated":135041024,"data_stored":134720894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509772,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862068,"statfs":{"total":96636764160,"available":96114757632,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:52.306 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:52.694 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:52.707 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-15T21:31:50.895311+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26191,"num_bytes_recovered":32629721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538224,"kb_used_data":199276,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577296,"statfs":{"total":289910292480,"available":288335151104,"internally_reserved":0,"allocated":204058624,"data_stored":203082854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2129,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001454"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21251,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248456+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:44.248456+0000","last_peered":"2024-09-15T21:31:44.248456+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:44.248456+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20802,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249719+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:44.249719+0000","last_peered":"2024-09-15T21:31:44.249719+0000","last_clean":"2024-09-15T21:31:44.249719+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:44.249719+0000","last_undegraded":"2024-09-15T21:31:44.249719+0000","last_fullsized":"2024-09-15T21:31:44.249719+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20408,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:44.247848+0000","last_peered":"2024-09-15T21:31:44.247848+0000","last_clean":"2024-09-15T21:31:44.247848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:44.247848+0000","last_undegraded":"2024-09-15T21:31:44.247848+0000","last_fullsized":"2024-09-15T21:31:44.247848+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20661,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250468+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:44.250468+0000","last_peered":"2024-09-15T21:31:44.250468+0000","last_clean":"2024-09-15T21:31:44.250468+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:44.250468+0000","last_undegraded":"2024-09-15T21:31:44.250468+0000","last_fullsized":"2024-09-15T21:31:44.250468+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20794,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250325+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:44.250325+0000","last_peered":"2024-09-15T21:31:44.250325+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:44.250325+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20829,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248602+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:44.248602+0000","last_peered":"2024-09-15T21:31:44.248602+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:44.248602+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21020,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248406+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:44.248406+0000","last_peered":"2024-09-15T21:31:44.248406+0000","last_clean":"2024-09-15T21:31:44.248406+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:44.248406+0000","last_undegraded":"2024-09-15T21:31:44.248406+0000","last_fullsized":"2024-09-15T21:31:44.248406+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248028+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:44.248028+0000","last_peered":"2024-09-15T21:31:44.248028+0000","last_clean":"2024-09-15T21:31:44.248028+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:44.248028+0000","last_undegraded":"2024-09-15T21:31:44.248028+0000","last_fullsized":"2024-09-15T21:31:44.248028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20636,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250586+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:44.250586+0000","last_peered":"2024-09-15T21:31:44.250586+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:44.250586+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250232+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:44.250232+0000","last_peered":"2024-09-15T21:31:44.250232+0000","last_clean":"2024-09-15T21:31:44.250232+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:44.250232+0000","last_undegraded":"2024-09-15T21:31:44.250232+0000","last_fullsized":"2024-09-15T21:31:44.250232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":23482,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:49.768463+0000","last_change":"2024-09-15T21:31:42.557303+0000","last_active":"2024-09-15T21:31:49.768463+0000","last_peered":"2024-09-15T21:31:49.768463+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:49.768463+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2743,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1348,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2742}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29042,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247910+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:44.247910+0000","last_peered":"2024-09-15T21:31:44.247910+0000","last_clean":"2024-09-15T21:31:44.247910+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:44.247910+0000","last_undegraded":"2024-09-15T21:31:44.247910+0000","last_fullsized":"2024-09-15T21:31:44.247910+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249618+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:44.249618+0000","last_peered":"2024-09-15T21:31:44.249618+0000","last_clean":"2024-09-15T21:31:44.249618+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:44.249618+0000","last_undegraded":"2024-09-15T21:31:44.249618+0000","last_fullsized":"2024-09-15T21:31:44.249618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:44.250878+0000","last_peered":"2024-09-15T21:31:44.250878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:44.250878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26190,"num_bytes_recovered":32629702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202674176,"data_stored":202558560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492604,"kb_used_data":131876,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879236,"statfs":{"total":96636764160,"available":96132337664,"internally_reserved":0,"allocated":135041024,"data_stored":134720894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509772,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862068,"statfs":{"total":96636764160,"available":96114757632,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:52.708 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:53.029 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:53.029 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:53.042 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-15T21:31:52.895704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26191,"num_bytes_recovered":32629721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534056,"kb_used_data":199276,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281581464,"statfs":{"total":289910292480,"available":288339419136,"internally_reserved":0,"allocated":204058624,"data_stored":203082854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1366739971},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2129,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.659661"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21251,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248456+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:44.248456+0000","last_peered":"2024-09-15T21:31:44.248456+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:44.248456+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20802,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249719+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:44.249719+0000","last_peered":"2024-09-15T21:31:44.249719+0000","last_clean":"2024-09-15T21:31:44.249719+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:44.249719+0000","last_undegraded":"2024-09-15T21:31:44.249719+0000","last_fullsized":"2024-09-15T21:31:44.249719+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20408,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:44.247848+0000","last_peered":"2024-09-15T21:31:44.247848+0000","last_clean":"2024-09-15T21:31:44.247848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:44.247848+0000","last_undegraded":"2024-09-15T21:31:44.247848+0000","last_fullsized":"2024-09-15T21:31:44.247848+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20661,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250468+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:44.250468+0000","last_peered":"2024-09-15T21:31:44.250468+0000","last_clean":"2024-09-15T21:31:44.250468+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:44.250468+0000","last_undegraded":"2024-09-15T21:31:44.250468+0000","last_fullsized":"2024-09-15T21:31:44.250468+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20794,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250325+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:44.250325+0000","last_peered":"2024-09-15T21:31:44.250325+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:44.250325+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20829,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248602+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:44.248602+0000","last_peered":"2024-09-15T21:31:44.248602+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:44.248602+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21020,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248406+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:44.248406+0000","last_peered":"2024-09-15T21:31:44.248406+0000","last_clean":"2024-09-15T21:31:44.248406+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:44.248406+0000","last_undegraded":"2024-09-15T21:31:44.248406+0000","last_fullsized":"2024-09-15T21:31:44.248406+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248028+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:44.248028+0000","last_peered":"2024-09-15T21:31:44.248028+0000","last_clean":"2024-09-15T21:31:44.248028+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:44.248028+0000","last_undegraded":"2024-09-15T21:31:44.248028+0000","last_fullsized":"2024-09-15T21:31:44.248028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20636,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250586+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:44.250586+0000","last_peered":"2024-09-15T21:31:44.250586+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:44.250586+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250232+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:44.250232+0000","last_peered":"2024-09-15T21:31:44.250232+0000","last_clean":"2024-09-15T21:31:44.250232+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:44.250232+0000","last_undegraded":"2024-09-15T21:31:44.250232+0000","last_fullsized":"2024-09-15T21:31:44.250232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":23482,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:49.768463+0000","last_change":"2024-09-15T21:31:42.557303+0000","last_active":"2024-09-15T21:31:49.768463+0000","last_peered":"2024-09-15T21:31:49.768463+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:49.768463+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2743,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1348,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2742}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29042,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247910+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:44.247910+0000","last_peered":"2024-09-15T21:31:44.247910+0000","last_clean":"2024-09-15T21:31:44.247910+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:44.247910+0000","last_undegraded":"2024-09-15T21:31:44.247910+0000","last_fullsized":"2024-09-15T21:31:44.247910+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249618+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:44.249618+0000","last_peered":"2024-09-15T21:31:44.249618+0000","last_clean":"2024-09-15T21:31:44.249618+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:44.249618+0000","last_undegraded":"2024-09-15T21:31:44.249618+0000","last_fullsized":"2024-09-15T21:31:44.249618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:44.250878+0000","last_peered":"2024-09-15T21:31:44.250878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:44.250878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26190,"num_bytes_recovered":32629702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202674176,"data_stored":202558560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492604,"kb_used_data":131876,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879236,"statfs":{"total":96636764160,"available":96132337664,"internally_reserved":0,"allocated":135041024,"data_stored":134720894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505604,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866236,"statfs":{"total":96636764160,"available":96119025664,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:53.043 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:31:53.043 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21251, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.248456+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:31:44.248456+0000', 'last_peered': '2024-09-15T21:31:44.248456+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:31:44.248456+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:53.043 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:31:53.043 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20794, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.250325+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:31:44.250325+0000', 'last_peered': '2024-09-15T21:31:44.250325+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:31:44.250325+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:53.044 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:31:53.044 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20829, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.248602+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:31:44.248602+0000', 'last_peered': '2024-09-15T21:31:44.248602+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:31:44.248602+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:53.044 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:31:53.044 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20636, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.250586+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:31:44.250586+0000', 'last_peered': '2024-09-15T21:31:44.250586+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:31:44.250586+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:53.044 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:31:53.044 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 23482, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:49.768463+0000', 'last_change': '2024-09-15T21:31:42.557303+0000', 'last_active': '2024-09-15T21:31:49.768463+0000', 'last_peered': '2024-09-15T21:31:49.768463+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:31:49.768463+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2743, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1348, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2742}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:53.044 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:31:53.044 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21028, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.250878+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:31:44.250878+0000', 'last_peered': '2024-09-15T21:31:44.250878+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:31:44.250878+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:53.045 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:31:53.442 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:53.442 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23284/24 objects degraded (97016.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":765,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1570873344,"bytes_avail":288339419136,"bytes_total":289910292480,"degraded_objects":23284,"degraded_total":24,"degraded_ratio":970.16666666666652,"recovering_objects_per_sec":220,"recovering_bytes_per_sec":50,"recovering_keys_per_sec":0,"num_objects_recovered":2129,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:31:53.455 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:31:53.456 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:53.759 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:53.759 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:53.771 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-15T21:31:52.895704+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26191,"num_bytes_recovered":32629721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534056,"kb_used_data":199276,"kb_used_omap":12,"kb_used_meta":1334707,"kb_avail":281581464,"statfs":{"total":289910292480,"available":288339419136,"internally_reserved":0,"allocated":204058624,"data_stored":203082854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1366739971},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2129,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.659661"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21251,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248456+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:44.248456+0000","last_peered":"2024-09-15T21:31:44.248456+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:44.248456+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20802,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249719+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:44.249719+0000","last_peered":"2024-09-15T21:31:44.249719+0000","last_clean":"2024-09-15T21:31:44.249719+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:44.249719+0000","last_undegraded":"2024-09-15T21:31:44.249719+0000","last_fullsized":"2024-09-15T21:31:44.249719+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20408,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:44.247848+0000","last_peered":"2024-09-15T21:31:44.247848+0000","last_clean":"2024-09-15T21:31:44.247848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:44.247848+0000","last_undegraded":"2024-09-15T21:31:44.247848+0000","last_fullsized":"2024-09-15T21:31:44.247848+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20661,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250468+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:44.250468+0000","last_peered":"2024-09-15T21:31:44.250468+0000","last_clean":"2024-09-15T21:31:44.250468+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:44.250468+0000","last_undegraded":"2024-09-15T21:31:44.250468+0000","last_fullsized":"2024-09-15T21:31:44.250468+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20794,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250325+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:44.250325+0000","last_peered":"2024-09-15T21:31:44.250325+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:44.250325+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20829,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248602+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:44.248602+0000","last_peered":"2024-09-15T21:31:44.248602+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:44.248602+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21020,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248406+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:44.248406+0000","last_peered":"2024-09-15T21:31:44.248406+0000","last_clean":"2024-09-15T21:31:44.248406+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:44.248406+0000","last_undegraded":"2024-09-15T21:31:44.248406+0000","last_fullsized":"2024-09-15T21:31:44.248406+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248028+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:44.248028+0000","last_peered":"2024-09-15T21:31:44.248028+0000","last_clean":"2024-09-15T21:31:44.248028+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:44.248028+0000","last_undegraded":"2024-09-15T21:31:44.248028+0000","last_fullsized":"2024-09-15T21:31:44.248028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20636,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250586+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:44.250586+0000","last_peered":"2024-09-15T21:31:44.250586+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:44.250586+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250232+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:44.250232+0000","last_peered":"2024-09-15T21:31:44.250232+0000","last_clean":"2024-09-15T21:31:44.250232+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:44.250232+0000","last_undegraded":"2024-09-15T21:31:44.250232+0000","last_fullsized":"2024-09-15T21:31:44.250232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":23482,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:49.768463+0000","last_change":"2024-09-15T21:31:42.557303+0000","last_active":"2024-09-15T21:31:49.768463+0000","last_peered":"2024-09-15T21:31:49.768463+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:49.768463+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2743,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1348,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2742}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29042,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247910+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:44.247910+0000","last_peered":"2024-09-15T21:31:44.247910+0000","last_clean":"2024-09-15T21:31:44.247910+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:44.247910+0000","last_undegraded":"2024-09-15T21:31:44.247910+0000","last_fullsized":"2024-09-15T21:31:44.247910+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249618+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:44.249618+0000","last_peered":"2024-09-15T21:31:44.249618+0000","last_clean":"2024-09-15T21:31:44.249618+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:44.249618+0000","last_undegraded":"2024-09-15T21:31:44.249618+0000","last_fullsized":"2024-09-15T21:31:44.249618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:44.250878+0000","last_peered":"2024-09-15T21:31:44.250878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:44.250878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23284,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26190,"num_bytes_recovered":32629702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202674176,"data_stored":202558560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492604,"kb_used_data":131876,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879236,"statfs":{"total":96636764160,"available":96132337664,"internally_reserved":0,"allocated":135041024,"data_stored":134720894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505604,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866236,"statfs":{"total":96636764160,"available":96119025664,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517205679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:56.774 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:57.075 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:57.076 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:57.088 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-15T21:31:56.896369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27053,"num_bytes_recovered":32629721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538216,"kb_used_data":199276,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577304,"statfs":{"total":289910292480,"available":288335159296,"internally_reserved":0,"allocated":204058624,"data_stored":203082854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2674,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2674,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002187"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21251,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248456+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:44.248456+0000","last_peered":"2024-09-15T21:31:44.248456+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:44.248456+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20802,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249719+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:44.249719+0000","last_peered":"2024-09-15T21:31:44.249719+0000","last_clean":"2024-09-15T21:31:44.249719+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:44.249719+0000","last_undegraded":"2024-09-15T21:31:44.249719+0000","last_fullsized":"2024-09-15T21:31:44.249719+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20408,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:44.247848+0000","last_peered":"2024-09-15T21:31:44.247848+0000","last_clean":"2024-09-15T21:31:44.247848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:44.247848+0000","last_undegraded":"2024-09-15T21:31:44.247848+0000","last_fullsized":"2024-09-15T21:31:44.247848+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20661,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250468+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:44.250468+0000","last_peered":"2024-09-15T21:31:44.250468+0000","last_clean":"2024-09-15T21:31:44.250468+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:44.250468+0000","last_undegraded":"2024-09-15T21:31:44.250468+0000","last_fullsized":"2024-09-15T21:31:44.250468+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.251469+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:44.251469+0000","last_peered":"2024-09-15T21:31:44.251469+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:44.251469+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20829,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248602+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:44.248602+0000","last_peered":"2024-09-15T21:31:44.248602+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:44.248602+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21020,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248406+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:44.248406+0000","last_peered":"2024-09-15T21:31:44.248406+0000","last_clean":"2024-09-15T21:31:44.248406+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:44.248406+0000","last_undegraded":"2024-09-15T21:31:44.248406+0000","last_fullsized":"2024-09-15T21:31:44.248406+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248028+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:44.248028+0000","last_peered":"2024-09-15T21:31:44.248028+0000","last_clean":"2024-09-15T21:31:44.248028+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:44.248028+0000","last_undegraded":"2024-09-15T21:31:44.248028+0000","last_fullsized":"2024-09-15T21:31:44.248028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20636,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250586+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:44.250586+0000","last_peered":"2024-09-15T21:31:44.250586+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:44.250586+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250232+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:44.250232+0000","last_peered":"2024-09-15T21:31:44.250232+0000","last_clean":"2024-09-15T21:31:44.250232+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:44.250232+0000","last_undegraded":"2024-09-15T21:31:44.250232+0000","last_fullsized":"2024-09-15T21:31:44.250232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":25206,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:54.770874+0000","last_change":"2024-09-15T21:31:42.557303+0000","last_active":"2024-09-15T21:31:54.770874+0000","last_peered":"2024-09-15T21:31:54.770874+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:54.770874+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1881,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2210,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1880}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29042,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247910+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:44.247910+0000","last_peered":"2024-09-15T21:31:44.247910+0000","last_clean":"2024-09-15T21:31:44.247910+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:44.247910+0000","last_undegraded":"2024-09-15T21:31:44.247910+0000","last_fullsized":"2024-09-15T21:31:44.247910+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249618+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:44.249618+0000","last_peered":"2024-09-15T21:31:44.249618+0000","last_clean":"2024-09-15T21:31:44.249618+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:44.249618+0000","last_undegraded":"2024-09-15T21:31:44.249618+0000","last_fullsized":"2024-09-15T21:31:44.249618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:44.250878+0000","last_peered":"2024-09-15T21:31:44.250878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:44.250878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27052,"num_bytes_recovered":32629702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202674176,"data_stored":202558560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492604,"kb_used_data":131876,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879236,"statfs":{"total":96636764160,"available":96132337664,"internally_reserved":0,"allocated":135041024,"data_stored":134720894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509764,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862076,"statfs":{"total":96636764160,"available":96114765824,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:57.089 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:57.423 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:57.424 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:57.436 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-15T21:31:56.896369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27053,"num_bytes_recovered":32629721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538216,"kb_used_data":199276,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577304,"statfs":{"total":289910292480,"available":288335159296,"internally_reserved":0,"allocated":204058624,"data_stored":203082854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2674,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2674,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002187"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21251,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248456+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:44.248456+0000","last_peered":"2024-09-15T21:31:44.248456+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:44.248456+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20802,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249719+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:44.249719+0000","last_peered":"2024-09-15T21:31:44.249719+0000","last_clean":"2024-09-15T21:31:44.249719+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:44.249719+0000","last_undegraded":"2024-09-15T21:31:44.249719+0000","last_fullsized":"2024-09-15T21:31:44.249719+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20408,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:44.247848+0000","last_peered":"2024-09-15T21:31:44.247848+0000","last_clean":"2024-09-15T21:31:44.247848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:44.247848+0000","last_undegraded":"2024-09-15T21:31:44.247848+0000","last_fullsized":"2024-09-15T21:31:44.247848+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20661,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250468+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:44.250468+0000","last_peered":"2024-09-15T21:31:44.250468+0000","last_clean":"2024-09-15T21:31:44.250468+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:44.250468+0000","last_undegraded":"2024-09-15T21:31:44.250468+0000","last_fullsized":"2024-09-15T21:31:44.250468+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.251469+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:44.251469+0000","last_peered":"2024-09-15T21:31:44.251469+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:44.251469+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20829,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248602+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:44.248602+0000","last_peered":"2024-09-15T21:31:44.248602+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:44.248602+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21020,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248406+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:44.248406+0000","last_peered":"2024-09-15T21:31:44.248406+0000","last_clean":"2024-09-15T21:31:44.248406+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:44.248406+0000","last_undegraded":"2024-09-15T21:31:44.248406+0000","last_fullsized":"2024-09-15T21:31:44.248406+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248028+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:44.248028+0000","last_peered":"2024-09-15T21:31:44.248028+0000","last_clean":"2024-09-15T21:31:44.248028+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:44.248028+0000","last_undegraded":"2024-09-15T21:31:44.248028+0000","last_fullsized":"2024-09-15T21:31:44.248028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20636,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250586+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:44.250586+0000","last_peered":"2024-09-15T21:31:44.250586+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:44.250586+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250232+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:44.250232+0000","last_peered":"2024-09-15T21:31:44.250232+0000","last_clean":"2024-09-15T21:31:44.250232+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:44.250232+0000","last_undegraded":"2024-09-15T21:31:44.250232+0000","last_fullsized":"2024-09-15T21:31:44.250232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":25206,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:54.770874+0000","last_change":"2024-09-15T21:31:42.557303+0000","last_active":"2024-09-15T21:31:54.770874+0000","last_peered":"2024-09-15T21:31:54.770874+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:54.770874+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1881,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2210,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1880}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29042,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247910+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:44.247910+0000","last_peered":"2024-09-15T21:31:44.247910+0000","last_clean":"2024-09-15T21:31:44.247910+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:44.247910+0000","last_undegraded":"2024-09-15T21:31:44.247910+0000","last_fullsized":"2024-09-15T21:31:44.247910+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249618+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:44.249618+0000","last_peered":"2024-09-15T21:31:44.249618+0000","last_clean":"2024-09-15T21:31:44.249618+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:44.249618+0000","last_undegraded":"2024-09-15T21:31:44.249618+0000","last_fullsized":"2024-09-15T21:31:44.249618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:44.250878+0000","last_peered":"2024-09-15T21:31:44.250878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:44.250878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27052,"num_bytes_recovered":32629702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202674176,"data_stored":202558560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492604,"kb_used_data":131876,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879236,"statfs":{"total":96636764160,"available":96132337664,"internally_reserved":0,"allocated":135041024,"data_stored":134720894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509764,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862076,"statfs":{"total":96636764160,"available":96114765824,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:31:57.437 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:31:57.437 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21251, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.248456+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:31:44.248456+0000', 'last_peered': '2024-09-15T21:31:44.248456+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:31:44.248456+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:57.437 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:31:57.437 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20795, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.251469+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:31:44.251469+0000', 'last_peered': '2024-09-15T21:31:44.251469+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:31:44.251469+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:57.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:31:57.438 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20829, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.248602+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:31:44.248602+0000', 'last_peered': '2024-09-15T21:31:44.248602+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:31:44.248602+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:57.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:31:57.438 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20636, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.250586+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:31:44.250586+0000', 'last_peered': '2024-09-15T21:31:44.250586+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:31:44.250586+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:57.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:31:57.438 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 25206, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:54.770874+0000', 'last_change': '2024-09-15T21:31:42.557303+0000', 'last_active': '2024-09-15T21:31:54.770874+0000', 'last_peered': '2024-09-15T21:31:54.770874+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:31:54.770874+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1881, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2210, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1880}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:31:57.438 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:31:57.438 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21028, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.250878+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:31:44.250878+0000', 'last_peered': '2024-09-15T21:31:44.250878+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:31:44.250878+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:31:57.439 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:31:57.828 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:57.828 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22422/24 objects degraded (93425.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":769,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1575133184,"bytes_avail":288335159296,"bytes_total":289910292480,"degraded_objects":22422,"degraded_total":24,"degraded_ratio":934.25,"recovering_objects_per_sec":222,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2674,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:31:57.841 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:31:57.841 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:31:58.146 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:31:58.146 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:31:58.159 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-15T21:31:56.896369+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27053,"num_bytes_recovered":32629721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538216,"kb_used_data":199276,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577304,"statfs":{"total":289910292480,"available":288335159296,"internally_reserved":0,"allocated":204058624,"data_stored":203082854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2674,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2674,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002187"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21251,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248456+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:44.248456+0000","last_peered":"2024-09-15T21:31:44.248456+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:44.248456+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20802,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249719+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:44.249719+0000","last_peered":"2024-09-15T21:31:44.249719+0000","last_clean":"2024-09-15T21:31:44.249719+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:44.249719+0000","last_undegraded":"2024-09-15T21:31:44.249719+0000","last_fullsized":"2024-09-15T21:31:44.249719+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20408,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:44.247848+0000","last_peered":"2024-09-15T21:31:44.247848+0000","last_clean":"2024-09-15T21:31:44.247848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:44.247848+0000","last_undegraded":"2024-09-15T21:31:44.247848+0000","last_fullsized":"2024-09-15T21:31:44.247848+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20661,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250468+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:44.250468+0000","last_peered":"2024-09-15T21:31:44.250468+0000","last_clean":"2024-09-15T21:31:44.250468+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:44.250468+0000","last_undegraded":"2024-09-15T21:31:44.250468+0000","last_fullsized":"2024-09-15T21:31:44.250468+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.251469+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:44.251469+0000","last_peered":"2024-09-15T21:31:44.251469+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:44.251469+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20829,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248602+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:44.248602+0000","last_peered":"2024-09-15T21:31:44.248602+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:44.248602+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21020,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248406+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:44.248406+0000","last_peered":"2024-09-15T21:31:44.248406+0000","last_clean":"2024-09-15T21:31:44.248406+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:44.248406+0000","last_undegraded":"2024-09-15T21:31:44.248406+0000","last_fullsized":"2024-09-15T21:31:44.248406+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248028+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:44.248028+0000","last_peered":"2024-09-15T21:31:44.248028+0000","last_clean":"2024-09-15T21:31:44.248028+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:44.248028+0000","last_undegraded":"2024-09-15T21:31:44.248028+0000","last_fullsized":"2024-09-15T21:31:44.248028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20636,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250586+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:44.250586+0000","last_peered":"2024-09-15T21:31:44.250586+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:44.250586+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250232+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:44.250232+0000","last_peered":"2024-09-15T21:31:44.250232+0000","last_clean":"2024-09-15T21:31:44.250232+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:44.250232+0000","last_undegraded":"2024-09-15T21:31:44.250232+0000","last_fullsized":"2024-09-15T21:31:44.250232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":25206,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:54.770874+0000","last_change":"2024-09-15T21:31:42.557303+0000","last_active":"2024-09-15T21:31:54.770874+0000","last_peered":"2024-09-15T21:31:54.770874+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:54.770874+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1881,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2210,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1880}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29042,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247910+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:44.247910+0000","last_peered":"2024-09-15T21:31:44.247910+0000","last_clean":"2024-09-15T21:31:44.247910+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:44.247910+0000","last_undegraded":"2024-09-15T21:31:44.247910+0000","last_fullsized":"2024-09-15T21:31:44.247910+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249618+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:44.249618+0000","last_peered":"2024-09-15T21:31:44.249618+0000","last_clean":"2024-09-15T21:31:44.249618+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:44.249618+0000","last_undegraded":"2024-09-15T21:31:44.249618+0000","last_fullsized":"2024-09-15T21:31:44.249618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:44.250878+0000","last_peered":"2024-09-15T21:31:44.250878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:44.250878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27052,"num_bytes_recovered":32629702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202674176,"data_stored":202558560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492604,"kb_used_data":131876,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879236,"statfs":{"total":96636764160,"available":96132337664,"internally_reserved":0,"allocated":135041024,"data_stored":134720894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509764,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862076,"statfs":{"total":96636764160,"available":96114765824,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:01.161 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:01.483 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:01.483 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:01.497 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-15T21:32:00.897128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27986,"num_bytes_recovered":32629721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538216,"kb_used_data":199276,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577304,"statfs":{"total":289910292480,"available":288335159296,"internally_reserved":0,"allocated":204058624,"data_stored":203082854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2826,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2826,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002198"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21251,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248456+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:44.248456+0000","last_peered":"2024-09-15T21:31:44.248456+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:44.248456+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20803,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249762+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:44.249762+0000","last_peered":"2024-09-15T21:31:44.249762+0000","last_clean":"2024-09-15T21:31:44.249762+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:44.249762+0000","last_undegraded":"2024-09-15T21:31:44.249762+0000","last_fullsized":"2024-09-15T21:31:44.249762+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20408,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:44.247848+0000","last_peered":"2024-09-15T21:31:44.247848+0000","last_clean":"2024-09-15T21:31:44.247848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:44.247848+0000","last_undegraded":"2024-09-15T21:31:44.247848+0000","last_fullsized":"2024-09-15T21:31:44.247848+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20661,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250468+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:44.250468+0000","last_peered":"2024-09-15T21:31:44.250468+0000","last_clean":"2024-09-15T21:31:44.250468+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:44.250468+0000","last_undegraded":"2024-09-15T21:31:44.250468+0000","last_fullsized":"2024-09-15T21:31:44.250468+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.251469+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:44.251469+0000","last_peered":"2024-09-15T21:31:44.251469+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:44.251469+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.249739+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:44.249739+0000","last_peered":"2024-09-15T21:31:44.249739+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:44.249739+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21020,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248406+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:44.248406+0000","last_peered":"2024-09-15T21:31:44.248406+0000","last_clean":"2024-09-15T21:31:44.248406+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:44.248406+0000","last_undegraded":"2024-09-15T21:31:44.248406+0000","last_fullsized":"2024-09-15T21:31:44.248406+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248028+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:44.248028+0000","last_peered":"2024-09-15T21:31:44.248028+0000","last_clean":"2024-09-15T21:31:44.248028+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:44.248028+0000","last_undegraded":"2024-09-15T21:31:44.248028+0000","last_fullsized":"2024-09-15T21:31:44.248028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20636,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250586+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:44.250586+0000","last_peered":"2024-09-15T21:31:44.250586+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:44.250586+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250232+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:44.250232+0000","last_peered":"2024-09-15T21:31:44.250232+0000","last_clean":"2024-09-15T21:31:44.250232+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:44.250232+0000","last_undegraded":"2024-09-15T21:31:44.250232+0000","last_fullsized":"2024-09-15T21:31:44.250232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":27072,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:59.772162+0000","last_change":"2024-09-15T21:31:42.557303+0000","last_active":"2024-09-15T21:31:59.772162+0000","last_peered":"2024-09-15T21:31:59.772162+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:59.772162+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":948,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3143,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-947}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29042,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247910+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:44.247910+0000","last_peered":"2024-09-15T21:31:44.247910+0000","last_clean":"2024-09-15T21:31:44.247910+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:44.247910+0000","last_undegraded":"2024-09-15T21:31:44.247910+0000","last_fullsized":"2024-09-15T21:31:44.247910+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249618+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:44.249618+0000","last_peered":"2024-09-15T21:31:44.249618+0000","last_clean":"2024-09-15T21:31:44.249618+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:44.249618+0000","last_undegraded":"2024-09-15T21:31:44.249618+0000","last_fullsized":"2024-09-15T21:31:44.249618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:44.250878+0000","last_peered":"2024-09-15T21:31:44.250878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:44.250878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27985,"num_bytes_recovered":32629702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202674176,"data_stored":202558560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492604,"kb_used_data":131876,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879236,"statfs":{"total":96636764160,"available":96132337664,"internally_reserved":0,"allocated":135041024,"data_stored":134720894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509764,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862076,"statfs":{"total":96636764160,"available":96114765824,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:01.498 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:01.818 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:01.819 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:01.831 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-15T21:32:00.897128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27986,"num_bytes_recovered":32629721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538216,"kb_used_data":199276,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577304,"statfs":{"total":289910292480,"available":288335159296,"internally_reserved":0,"allocated":204058624,"data_stored":203082854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2826,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2826,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002198"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21251,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248456+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:44.248456+0000","last_peered":"2024-09-15T21:31:44.248456+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:44.248456+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20803,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249762+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:44.249762+0000","last_peered":"2024-09-15T21:31:44.249762+0000","last_clean":"2024-09-15T21:31:44.249762+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:44.249762+0000","last_undegraded":"2024-09-15T21:31:44.249762+0000","last_fullsized":"2024-09-15T21:31:44.249762+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20408,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:44.247848+0000","last_peered":"2024-09-15T21:31:44.247848+0000","last_clean":"2024-09-15T21:31:44.247848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:44.247848+0000","last_undegraded":"2024-09-15T21:31:44.247848+0000","last_fullsized":"2024-09-15T21:31:44.247848+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20661,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250468+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:44.250468+0000","last_peered":"2024-09-15T21:31:44.250468+0000","last_clean":"2024-09-15T21:31:44.250468+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:44.250468+0000","last_undegraded":"2024-09-15T21:31:44.250468+0000","last_fullsized":"2024-09-15T21:31:44.250468+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.251469+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:44.251469+0000","last_peered":"2024-09-15T21:31:44.251469+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:44.251469+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.249739+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:44.249739+0000","last_peered":"2024-09-15T21:31:44.249739+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:44.249739+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21020,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248406+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:44.248406+0000","last_peered":"2024-09-15T21:31:44.248406+0000","last_clean":"2024-09-15T21:31:44.248406+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:44.248406+0000","last_undegraded":"2024-09-15T21:31:44.248406+0000","last_fullsized":"2024-09-15T21:31:44.248406+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248028+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:44.248028+0000","last_peered":"2024-09-15T21:31:44.248028+0000","last_clean":"2024-09-15T21:31:44.248028+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:44.248028+0000","last_undegraded":"2024-09-15T21:31:44.248028+0000","last_fullsized":"2024-09-15T21:31:44.248028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20636,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250586+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:44.250586+0000","last_peered":"2024-09-15T21:31:44.250586+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:44.250586+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250232+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:44.250232+0000","last_peered":"2024-09-15T21:31:44.250232+0000","last_clean":"2024-09-15T21:31:44.250232+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:44.250232+0000","last_undegraded":"2024-09-15T21:31:44.250232+0000","last_fullsized":"2024-09-15T21:31:44.250232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":27072,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:59.772162+0000","last_change":"2024-09-15T21:31:42.557303+0000","last_active":"2024-09-15T21:31:59.772162+0000","last_peered":"2024-09-15T21:31:59.772162+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:59.772162+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":948,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3143,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-947}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29042,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247910+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:44.247910+0000","last_peered":"2024-09-15T21:31:44.247910+0000","last_clean":"2024-09-15T21:31:44.247910+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:44.247910+0000","last_undegraded":"2024-09-15T21:31:44.247910+0000","last_fullsized":"2024-09-15T21:31:44.247910+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249618+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:44.249618+0000","last_peered":"2024-09-15T21:31:44.249618+0000","last_clean":"2024-09-15T21:31:44.249618+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:44.249618+0000","last_undegraded":"2024-09-15T21:31:44.249618+0000","last_fullsized":"2024-09-15T21:31:44.249618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:44.250878+0000","last_peered":"2024-09-15T21:31:44.250878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:44.250878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27985,"num_bytes_recovered":32629702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202674176,"data_stored":202558560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492604,"kb_used_data":131876,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879236,"statfs":{"total":96636764160,"available":96132337664,"internally_reserved":0,"allocated":135041024,"data_stored":134720894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509764,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862076,"statfs":{"total":96636764160,"available":96114765824,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:01.832 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:01.832 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21251, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.248456+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:31:44.248456+0000', 'last_peered': '2024-09-15T21:31:44.248456+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:31:44.248456+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:01.833 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:01.833 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20795, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.251469+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:31:44.251469+0000', 'last_peered': '2024-09-15T21:31:44.251469+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:31:44.251469+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:01.833 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:32:01.833 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20830, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.249739+0000', 'last_change': '2024-09-15T21:21:14.884697+0000', 'last_active': '2024-09-15T21:31:44.249739+0000', 'last_peered': '2024-09-15T21:31:44.249739+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:31:44.249739+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4161, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4159}, {'shards': '0,2', 'objects': 2}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:01.833 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:32:01.833 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20636, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.250586+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:31:44.250586+0000', 'last_peered': '2024-09-15T21:31:44.250586+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:31:44.250586+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:01.833 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:32:01.833 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 27072, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:59.772162+0000', 'last_change': '2024-09-15T21:31:42.557303+0000', 'last_active': '2024-09-15T21:31:59.772162+0000', 'last_peered': '2024-09-15T21:31:59.772162+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:31:59.772162+0000', 'last_undegraded': '2024-09-15T21:21:14.399536+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 948, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3143, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -947}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:32:01.833 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:01.834 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21028, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:31:44.250878+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:31:44.250878+0000', 'last_peered': '2024-09-15T21:31:44.250878+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:31:44.250878+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:01.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:32:02.226 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:02.226 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21489/24 objects degraded (89537.500%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":773,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1575133184,"bytes_avail":288335159296,"bytes_total":289910292480,"degraded_objects":21489,"degraded_total":24,"degraded_ratio":895.375,"recovering_objects_per_sec":235,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2826,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:32:02.239 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:32:02.239 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:02.543 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:02.544 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:02.555 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-15T21:32:00.897128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27986,"num_bytes_recovered":32629721,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538216,"kb_used_data":199276,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577304,"statfs":{"total":289910292480,"available":288335159296,"internally_reserved":0,"allocated":204058624,"data_stored":203082854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2826,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2826,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002198"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21251,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.248456+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:31:44.248456+0000","last_peered":"2024-09-15T21:31:44.248456+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:31:44.248456+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20803,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249762+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:31:44.249762+0000","last_peered":"2024-09-15T21:31:44.249762+0000","last_clean":"2024-09-15T21:31:44.249762+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:31:44.249762+0000","last_undegraded":"2024-09-15T21:31:44.249762+0000","last_fullsized":"2024-09-15T21:31:44.249762+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20408,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247848+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:31:44.247848+0000","last_peered":"2024-09-15T21:31:44.247848+0000","last_clean":"2024-09-15T21:31:44.247848+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:31:44.247848+0000","last_undegraded":"2024-09-15T21:31:44.247848+0000","last_fullsized":"2024-09-15T21:31:44.247848+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20661,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250468+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:31:44.250468+0000","last_peered":"2024-09-15T21:31:44.250468+0000","last_clean":"2024-09-15T21:31:44.250468+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:31:44.250468+0000","last_undegraded":"2024-09-15T21:31:44.250468+0000","last_fullsized":"2024-09-15T21:31:44.250468+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.251469+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:31:44.251469+0000","last_peered":"2024-09-15T21:31:44.251469+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:31:44.251469+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.249739+0000","last_change":"2024-09-15T21:21:14.884697+0000","last_active":"2024-09-15T21:31:44.249739+0000","last_peered":"2024-09-15T21:31:44.249739+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:31:44.249739+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4161,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4159},{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21020,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248406+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:31:44.248406+0000","last_peered":"2024-09-15T21:31:44.248406+0000","last_clean":"2024-09-15T21:31:44.248406+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:31:44.248406+0000","last_undegraded":"2024-09-15T21:31:44.248406+0000","last_fullsized":"2024-09-15T21:31:44.248406+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.248028+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:31:44.248028+0000","last_peered":"2024-09-15T21:31:44.248028+0000","last_clean":"2024-09-15T21:31:44.248028+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:31:44.248028+0000","last_undegraded":"2024-09-15T21:31:44.248028+0000","last_fullsized":"2024-09-15T21:31:44.248028+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20636,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250586+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:31:44.250586+0000","last_peered":"2024-09-15T21:31:44.250586+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:31:44.250586+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.250232+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:31:44.250232+0000","last_peered":"2024-09-15T21:31:44.250232+0000","last_clean":"2024-09-15T21:31:44.250232+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:31:44.250232+0000","last_undegraded":"2024-09-15T21:31:44.250232+0000","last_fullsized":"2024-09-15T21:31:44.250232+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":27072,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:59.772162+0000","last_change":"2024-09-15T21:31:42.557303+0000","last_active":"2024-09-15T21:31:59.772162+0000","last_peered":"2024-09-15T21:31:59.772162+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:31:59.772162+0000","last_undegraded":"2024-09-15T21:21:14.399536+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":948,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3143,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-947}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29042,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.247910+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:31:44.247910+0000","last_peered":"2024-09-15T21:31:44.247910+0000","last_clean":"2024-09-15T21:31:44.247910+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:31:44.247910+0000","last_undegraded":"2024-09-15T21:31:44.247910+0000","last_fullsized":"2024-09-15T21:31:44.247910+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.249618+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:31:44.249618+0000","last_peered":"2024-09-15T21:31:44.249618+0000","last_clean":"2024-09-15T21:31:44.249618+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:31:44.249618+0000","last_undegraded":"2024-09-15T21:31:44.249618+0000","last_fullsized":"2024-09-15T21:31:44.249618+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21028,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:31:44.250878+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:31:44.250878+0000","last_peered":"2024-09-15T21:31:44.250878+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:31:44.250878+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21489,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27985,"num_bytes_recovered":32629702,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202674176,"data_stored":202558560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492604,"kb_used_data":131876,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879236,"statfs":{"total":96636764160,"available":96132337664,"internally_reserved":0,"allocated":135041024,"data_stored":134720894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509764,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862076,"statfs":{"total":96636764160,"available":96114765824,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134582272,"data_stored":134546142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:05.557 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:05.877 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:05.890 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-09-15T21:32:05.301786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29017,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529552,"kb_used_data":198936,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281585968,"statfs":{"total":289910292480,"available":288344031232,"internally_reserved":0,"allocated":203710464,"data_stored":202729774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1362480131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1964,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.405417"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21253,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.295565+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:04.295565+0000","last_peered":"2024-09-15T21:32:04.295565+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:04.295565+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20805,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.296245+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:04.296245+0000","last_peered":"2024-09-15T21:32:04.296245+0000","last_clean":"2024-09-15T21:32:04.296245+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:04.296245+0000","last_undegraded":"2024-09-15T21:32:04.296245+0000","last_fullsized":"2024-09-15T21:32:04.296245+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20410,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295981+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:04.295981+0000","last_peered":"2024-09-15T21:32:04.295981+0000","last_clean":"2024-09-15T21:32:04.295981+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:04.295981+0000","last_undegraded":"2024-09-15T21:32:04.295981+0000","last_fullsized":"2024-09-15T21:32:04.295981+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20663,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295112+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:04.295112+0000","last_peered":"2024-09-15T21:32:04.295112+0000","last_clean":"2024-09-15T21:32:04.295112+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:04.295112+0000","last_undegraded":"2024-09-15T21:32:04.295112+0000","last_fullsized":"2024-09-15T21:32:04.295112+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20797,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.294614+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:04.294614+0000","last_peered":"2024-09-15T21:32:04.294614+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:04.294614+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20999,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.563696+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:04.563696+0000","last_peered":"2024-09-15T21:32:04.563696+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:04.563696+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21022,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.294658+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:04.294658+0000","last_peered":"2024-09-15T21:32:04.294658+0000","last_clean":"2024-09-15T21:32:04.294658+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:04.294658+0000","last_undegraded":"2024-09-15T21:32:04.294658+0000","last_fullsized":"2024-09-15T21:32:04.294658+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.302384+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:04.302384+0000","last_peered":"2024-09-15T21:32:04.302384+0000","last_clean":"2024-09-15T21:32:04.302384+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:04.302384+0000","last_undegraded":"2024-09-15T21:32:04.302384+0000","last_fullsized":"2024-09-15T21:32:04.302384+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20638,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.294770+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:04.294770+0000","last_peered":"2024-09-15T21:32:04.294770+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:04.294770+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.294508+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:04.294508+0000","last_peered":"2024-09-15T21:32:04.294508+0000","last_clean":"2024-09-15T21:32:04.294508+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:04.294508+0000","last_undegraded":"2024-09-15T21:32:04.294508+0000","last_fullsized":"2024-09-15T21:32:04.294508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28970,"reported_epoch":55,"state":"peering","last_fresh":"2024-09-15T21:32:03.896943+0000","last_change":"2024-09-15T21:32:04.301856+0000","last_active":"2024-09-15T21:32:04.301856+0000","last_peered":"2024-09-15T21:32:03.896943+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:32:03.896943+0000","last_undegraded":"2024-09-15T21:32:03.896943+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29044,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295516+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:04.295516+0000","last_peered":"2024-09-15T21:32:04.295516+0000","last_clean":"2024-09-15T21:32:04.295516+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:04.295516+0000","last_undegraded":"2024-09-15T21:32:04.295516+0000","last_fullsized":"2024-09-15T21:32:04.295516+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.296106+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:04.296106+0000","last_peered":"2024-09-15T21:32:04.296106+0000","last_clean":"2024-09-15T21:32:04.296106+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:04.296106+0000","last_undegraded":"2024-09-15T21:32:04.296106+0000","last_fullsized":"2024-09-15T21:32:04.296106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21030,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.295263+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:04.295263+0000","last_peered":"2024-09-15T21:32:04.295263+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:04.295263+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29016,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202317824,"data_stored":202202208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492260,"kb_used_data":131536,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879580,"statfs":{"total":96636764160,"available":96132689920,"internally_reserved":0,"allocated":134692864,"data_stored":134367814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501444,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870396,"statfs":{"total":96636764160,"available":96123285504,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134225920,"data_stored":134189790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:05.891 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:06.249 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:06.250 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:06.262 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-09-15T21:32:05.301786+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29017,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529552,"kb_used_data":198936,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281585968,"statfs":{"total":289910292480,"available":288344031232,"internally_reserved":0,"allocated":203710464,"data_stored":202729774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1362480131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1964,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.405417"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21253,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.295565+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:04.295565+0000","last_peered":"2024-09-15T21:32:04.295565+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:04.295565+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20805,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.296245+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:04.296245+0000","last_peered":"2024-09-15T21:32:04.296245+0000","last_clean":"2024-09-15T21:32:04.296245+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:04.296245+0000","last_undegraded":"2024-09-15T21:32:04.296245+0000","last_fullsized":"2024-09-15T21:32:04.296245+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20410,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295981+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:04.295981+0000","last_peered":"2024-09-15T21:32:04.295981+0000","last_clean":"2024-09-15T21:32:04.295981+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:04.295981+0000","last_undegraded":"2024-09-15T21:32:04.295981+0000","last_fullsized":"2024-09-15T21:32:04.295981+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20663,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295112+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:04.295112+0000","last_peered":"2024-09-15T21:32:04.295112+0000","last_clean":"2024-09-15T21:32:04.295112+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:04.295112+0000","last_undegraded":"2024-09-15T21:32:04.295112+0000","last_fullsized":"2024-09-15T21:32:04.295112+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20797,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.294614+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:04.294614+0000","last_peered":"2024-09-15T21:32:04.294614+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:04.294614+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20999,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.563696+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:04.563696+0000","last_peered":"2024-09-15T21:32:04.563696+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:04.563696+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21022,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.294658+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:04.294658+0000","last_peered":"2024-09-15T21:32:04.294658+0000","last_clean":"2024-09-15T21:32:04.294658+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:04.294658+0000","last_undegraded":"2024-09-15T21:32:04.294658+0000","last_fullsized":"2024-09-15T21:32:04.294658+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252611+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:31:44.252611+0000","last_peered":"2024-09-15T21:31:44.252611+0000","last_clean":"2024-09-15T21:31:44.252611+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:31:44.252611+0000","last_undegraded":"2024-09-15T21:31:44.252611+0000","last_fullsized":"2024-09-15T21:31:44.252611+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20667,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252779+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:31:44.252779+0000","last_peered":"2024-09-15T21:31:44.252779+0000","last_clean":"2024-09-15T21:31:44.252779+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:31:44.252779+0000","last_undegraded":"2024-09-15T21:31:44.252779+0000","last_fullsized":"2024-09-15T21:31:44.252779+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.302384+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:04.302384+0000","last_peered":"2024-09-15T21:32:04.302384+0000","last_clean":"2024-09-15T21:32:04.302384+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:04.302384+0000","last_undegraded":"2024-09-15T21:32:04.302384+0000","last_fullsized":"2024-09-15T21:32:04.302384+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20638,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.294770+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:04.294770+0000","last_peered":"2024-09-15T21:32:04.294770+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:04.294770+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.294508+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:04.294508+0000","last_peered":"2024-09-15T21:32:04.294508+0000","last_clean":"2024-09-15T21:32:04.294508+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:04.294508+0000","last_undegraded":"2024-09-15T21:32:04.294508+0000","last_fullsized":"2024-09-15T21:32:04.294508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28970,"reported_epoch":55,"state":"peering","last_fresh":"2024-09-15T21:32:03.896943+0000","last_change":"2024-09-15T21:32:04.301856+0000","last_active":"2024-09-15T21:32:04.301856+0000","last_peered":"2024-09-15T21:32:03.896943+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:32:03.896943+0000","last_undegraded":"2024-09-15T21:32:03.896943+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252675+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:31:44.252675+0000","last_peered":"2024-09-15T21:31:44.252675+0000","last_clean":"2024-09-15T21:31:44.252675+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:31:44.252675+0000","last_undegraded":"2024-09-15T21:31:44.252675+0000","last_fullsized":"2024-09-15T21:31:44.252675+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29044,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295516+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:04.295516+0000","last_peered":"2024-09-15T21:32:04.295516+0000","last_clean":"2024-09-15T21:32:04.295516+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:04.295516+0000","last_undegraded":"2024-09-15T21:32:04.295516+0000","last_fullsized":"2024-09-15T21:32:04.295516+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20840,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.349778+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:31:44.349778+0000","last_peered":"2024-09-15T21:31:44.349778+0000","last_clean":"2024-09-15T21:31:44.349778+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:31:44.349778+0000","last_undegraded":"2024-09-15T21:31:44.349778+0000","last_fullsized":"2024-09-15T21:31:44.349778+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252941+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:31:44.252941+0000","last_peered":"2024-09-15T21:31:44.252941+0000","last_clean":"2024-09-15T21:31:44.252941+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:31:44.252941+0000","last_undegraded":"2024-09-15T21:31:44.252941+0000","last_fullsized":"2024-09-15T21:31:44.252941+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21605,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252783+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:31:44.252783+0000","last_peered":"2024-09-15T21:31:44.252783+0000","last_clean":"2024-09-15T21:31:44.252783+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:31:44.252783+0000","last_undegraded":"2024-09-15T21:31:44.252783+0000","last_fullsized":"2024-09-15T21:31:44.252783+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.296106+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:04.296106+0000","last_peered":"2024-09-15T21:32:04.296106+0000","last_clean":"2024-09-15T21:32:04.296106+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:04.296106+0000","last_undegraded":"2024-09-15T21:32:04.296106+0000","last_fullsized":"2024-09-15T21:32:04.296106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28721,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252880+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:31:44.252880+0000","last_peered":"2024-09-15T21:31:44.252880+0000","last_clean":"2024-09-15T21:31:44.252880+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:31:44.252880+0000","last_undegraded":"2024-09-15T21:31:44.252880+0000","last_fullsized":"2024-09-15T21:31:44.252880+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.252730+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:31:44.252730+0000","last_peered":"2024-09-15T21:31:44.252730+0000","last_clean":"2024-09-15T21:31:44.252730+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:31:44.252730+0000","last_undegraded":"2024-09-15T21:31:44.252730+0000","last_fullsized":"2024-09-15T21:31:44.252730+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-15T21:31:44.253090+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:31:44.253090+0000","last_peered":"2024-09-15T21:31:44.253090+0000","last_clean":"2024-09-15T21:31:44.253090+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:31:44.253090+0000","last_undegraded":"2024-09-15T21:31:44.253090+0000","last_fullsized":"2024-09-15T21:31:44.253090+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21030,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.295263+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:04.295263+0000","last_peered":"2024-09-15T21:32:04.295263+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:04.295263+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29016,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202317824,"data_stored":202202208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492260,"kb_used_data":131536,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879580,"statfs":{"total":96636764160,"available":96132689920,"internally_reserved":0,"allocated":134692864,"data_stored":134367814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501444,"kb_used_data":500,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870396,"statfs":{"total":96636764160,"available":96123285504,"internally_reserved":0,"allocated":512000,"data_stored":180567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134225920,"data_stored":134189790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:06.264 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:06.264 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21253, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:04.295565+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:32:04.295565+0000', 'last_peered': '2024-09-15T21:32:04.295565+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:04.295565+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:06.264 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:06.264 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20797, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:04.294614+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:04.294614+0000', 'last_peered': '2024-09-15T21:32:04.294614+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:04.294614+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:06.264 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:32:06.265 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20999, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:04.563696+0000', 'last_change': '2024-09-15T21:32:03.897219+0000', 'last_active': '2024-09-15T21:32:04.563696+0000', 'last_peered': '2024-09-15T21:32:04.563696+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:32:04.563696+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:06.265 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:32:06.265 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20638, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:04.294770+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:32:04.294770+0000', 'last_peered': '2024-09-15T21:32:04.294770+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:32:04.294770+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:06.265 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:32:06.265 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 28970, 'reported_epoch': 55, 'state': 'peering', 'last_fresh': '2024-09-15T21:32:03.896943+0000', 'last_change': '2024-09-15T21:32:04.301856+0000', 'last_active': '2024-09-15T21:32:04.301856+0000', 'last_peered': '2024-09-15T21:32:03.896943+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:32:03.896943+0000', 'last_undegraded': '2024-09-15T21:32:03.896943+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 56, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4091, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:32:06.265 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:06.265 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21030, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:04.295263+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:04.295263+0000', 'last_peered': '2024-09-15T21:32:04.295263+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:04.295263+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:06.266 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:32:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:06.714 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20458/24 objects degraded (85241.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":778,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1566261248,"bytes_avail":288344031232,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":20458,"degraded_total":24,"degraded_ratio":852.41666666666652,"recovering_objects_per_sec":196,"recovering_bytes_per_sec":96,"recovering_keys_per_sec":0,"num_objects_recovered":1964,"num_bytes_recovered":966,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:32:06.727 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:32:06.728 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:07.054 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:07.067 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-15T21:32:06.898121+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29017,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537896,"kb_used_data":198952,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577624,"statfs":{"total":289910292480,"available":288335486976,"internally_reserved":0,"allocated":203726848,"data_stored":202736297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1964,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1964,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001292"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21253,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.295565+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:04.295565+0000","last_peered":"2024-09-15T21:32:04.295565+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:04.295565+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20805,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.296245+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:04.296245+0000","last_peered":"2024-09-15T21:32:04.296245+0000","last_clean":"2024-09-15T21:32:04.296245+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:04.296245+0000","last_undegraded":"2024-09-15T21:32:04.296245+0000","last_fullsized":"2024-09-15T21:32:04.296245+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20410,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295981+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:04.295981+0000","last_peered":"2024-09-15T21:32:04.295981+0000","last_clean":"2024-09-15T21:32:04.295981+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:04.295981+0000","last_undegraded":"2024-09-15T21:32:04.295981+0000","last_fullsized":"2024-09-15T21:32:04.295981+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20663,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295112+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:04.295112+0000","last_peered":"2024-09-15T21:32:04.295112+0000","last_clean":"2024-09-15T21:32:04.295112+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:04.295112+0000","last_undegraded":"2024-09-15T21:32:04.295112+0000","last_fullsized":"2024-09-15T21:32:04.295112+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20797,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.294614+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:04.294614+0000","last_peered":"2024-09-15T21:32:04.294614+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:04.294614+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20999,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.563696+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:04.563696+0000","last_peered":"2024-09-15T21:32:04.563696+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:04.563696+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21022,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.294658+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:04.294658+0000","last_peered":"2024-09-15T21:32:04.294658+0000","last_clean":"2024-09-15T21:32:04.294658+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:04.294658+0000","last_undegraded":"2024-09-15T21:32:04.294658+0000","last_fullsized":"2024-09-15T21:32:04.294658+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.302384+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:04.302384+0000","last_peered":"2024-09-15T21:32:04.302384+0000","last_clean":"2024-09-15T21:32:04.302384+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:04.302384+0000","last_undegraded":"2024-09-15T21:32:04.302384+0000","last_fullsized":"2024-09-15T21:32:04.302384+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20638,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.294770+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:04.294770+0000","last_peered":"2024-09-15T21:32:04.294770+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:04.294770+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.294508+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:04.294508+0000","last_peered":"2024-09-15T21:32:04.294508+0000","last_clean":"2024-09-15T21:32:04.294508+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:04.294508+0000","last_undegraded":"2024-09-15T21:32:04.294508+0000","last_fullsized":"2024-09-15T21:32:04.294508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28970,"reported_epoch":55,"state":"peering","last_fresh":"2024-09-15T21:32:03.896943+0000","last_change":"2024-09-15T21:32:04.301856+0000","last_active":"2024-09-15T21:32:04.301856+0000","last_peered":"2024-09-15T21:32:03.896943+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:32:03.896943+0000","last_undegraded":"2024-09-15T21:32:03.896943+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29044,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295516+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:04.295516+0000","last_peered":"2024-09-15T21:32:04.295516+0000","last_clean":"2024-09-15T21:32:04.295516+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:04.295516+0000","last_undegraded":"2024-09-15T21:32:04.295516+0000","last_fullsized":"2024-09-15T21:32:04.295516+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.296106+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:04.296106+0000","last_peered":"2024-09-15T21:32:04.296106+0000","last_clean":"2024-09-15T21:32:04.296106+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:04.296106+0000","last_undegraded":"2024-09-15T21:32:04.296106+0000","last_fullsized":"2024-09-15T21:32:04.296106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21030,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.295263+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:04.295263+0000","last_peered":"2024-09-15T21:32:04.295263+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:04.295263+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29016,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202317824,"data_stored":202202208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492260,"kb_used_data":131536,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879580,"statfs":{"total":96636764160,"available":96132689920,"internally_reserved":0,"allocated":134692864,"data_stored":134367814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509788,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862052,"statfs":{"total":96636764160,"available":96114741248,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134225920,"data_stored":134189790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:10.068 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:10.419 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:10.419 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:10.433 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-15T21:32:08.898575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29017,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537896,"kb_used_data":198952,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577624,"statfs":{"total":289910292480,"available":288335486976,"internally_reserved":0,"allocated":203726848,"data_stored":202736297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1031,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001446"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21253,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.295565+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:04.295565+0000","last_peered":"2024-09-15T21:32:04.295565+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:04.295565+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20805,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.296245+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:04.296245+0000","last_peered":"2024-09-15T21:32:04.296245+0000","last_clean":"2024-09-15T21:32:04.296245+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:04.296245+0000","last_undegraded":"2024-09-15T21:32:04.296245+0000","last_fullsized":"2024-09-15T21:32:04.296245+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20410,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295981+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:04.295981+0000","last_peered":"2024-09-15T21:32:04.295981+0000","last_clean":"2024-09-15T21:32:04.295981+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:04.295981+0000","last_undegraded":"2024-09-15T21:32:04.295981+0000","last_fullsized":"2024-09-15T21:32:04.295981+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20663,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295112+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:04.295112+0000","last_peered":"2024-09-15T21:32:04.295112+0000","last_clean":"2024-09-15T21:32:04.295112+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:04.295112+0000","last_undegraded":"2024-09-15T21:32:04.295112+0000","last_fullsized":"2024-09-15T21:32:04.295112+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20797,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.294614+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:04.294614+0000","last_peered":"2024-09-15T21:32:04.294614+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:04.294614+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20999,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.563696+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:04.563696+0000","last_peered":"2024-09-15T21:32:04.563696+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:04.563696+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21022,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.294658+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:04.294658+0000","last_peered":"2024-09-15T21:32:04.294658+0000","last_clean":"2024-09-15T21:32:04.294658+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:04.294658+0000","last_undegraded":"2024-09-15T21:32:04.294658+0000","last_fullsized":"2024-09-15T21:32:04.294658+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.302384+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:04.302384+0000","last_peered":"2024-09-15T21:32:04.302384+0000","last_clean":"2024-09-15T21:32:04.302384+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:04.302384+0000","last_undegraded":"2024-09-15T21:32:04.302384+0000","last_fullsized":"2024-09-15T21:32:04.302384+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20638,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.294770+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:04.294770+0000","last_peered":"2024-09-15T21:32:04.294770+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:04.294770+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.294508+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:04.294508+0000","last_peered":"2024-09-15T21:32:04.294508+0000","last_clean":"2024-09-15T21:32:04.294508+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:04.294508+0000","last_undegraded":"2024-09-15T21:32:04.294508+0000","last_fullsized":"2024-09-15T21:32:04.294508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28970,"reported_epoch":55,"state":"peering","last_fresh":"2024-09-15T21:32:03.896943+0000","last_change":"2024-09-15T21:32:04.301856+0000","last_active":"2024-09-15T21:32:04.301856+0000","last_peered":"2024-09-15T21:32:03.896943+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:32:03.896943+0000","last_undegraded":"2024-09-15T21:32:03.896943+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29044,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295516+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:04.295516+0000","last_peered":"2024-09-15T21:32:04.295516+0000","last_clean":"2024-09-15T21:32:04.295516+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:04.295516+0000","last_undegraded":"2024-09-15T21:32:04.295516+0000","last_fullsized":"2024-09-15T21:32:04.295516+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.296106+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:04.296106+0000","last_peered":"2024-09-15T21:32:04.296106+0000","last_clean":"2024-09-15T21:32:04.296106+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:04.296106+0000","last_undegraded":"2024-09-15T21:32:04.296106+0000","last_fullsized":"2024-09-15T21:32:04.296106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21030,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.295263+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:04.295263+0000","last_peered":"2024-09-15T21:32:04.295263+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:04.295263+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29016,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202317824,"data_stored":202202208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492260,"kb_used_data":131536,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879580,"statfs":{"total":96636764160,"available":96132689920,"internally_reserved":0,"allocated":134692864,"data_stored":134367814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509788,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862052,"statfs":{"total":96636764160,"available":96114741248,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134225920,"data_stored":134189790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:10.435 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:10.785 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:10.801 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-15T21:32:08.898575+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29017,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537896,"kb_used_data":198952,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281577624,"statfs":{"total":289910292480,"available":288335486976,"internally_reserved":0,"allocated":203726848,"data_stored":202736297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1031,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1031,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001446"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21253,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.295565+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:04.295565+0000","last_peered":"2024-09-15T21:32:04.295565+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:04.295565+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20805,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.296245+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:04.296245+0000","last_peered":"2024-09-15T21:32:04.296245+0000","last_clean":"2024-09-15T21:32:04.296245+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:04.296245+0000","last_undegraded":"2024-09-15T21:32:04.296245+0000","last_fullsized":"2024-09-15T21:32:04.296245+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20410,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295981+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:04.295981+0000","last_peered":"2024-09-15T21:32:04.295981+0000","last_clean":"2024-09-15T21:32:04.295981+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:04.295981+0000","last_undegraded":"2024-09-15T21:32:04.295981+0000","last_fullsized":"2024-09-15T21:32:04.295981+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20663,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295112+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:04.295112+0000","last_peered":"2024-09-15T21:32:04.295112+0000","last_clean":"2024-09-15T21:32:04.295112+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:04.295112+0000","last_undegraded":"2024-09-15T21:32:04.295112+0000","last_fullsized":"2024-09-15T21:32:04.295112+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20797,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.294614+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:04.294614+0000","last_peered":"2024-09-15T21:32:04.294614+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:04.294614+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20999,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.563696+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:04.563696+0000","last_peered":"2024-09-15T21:32:04.563696+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:04.563696+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4078,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":83,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4076}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21022,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.294658+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:04.294658+0000","last_peered":"2024-09-15T21:32:04.294658+0000","last_clean":"2024-09-15T21:32:04.294658+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:04.294658+0000","last_undegraded":"2024-09-15T21:32:04.294658+0000","last_fullsized":"2024-09-15T21:32:04.294658+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.302384+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:04.302384+0000","last_peered":"2024-09-15T21:32:04.302384+0000","last_clean":"2024-09-15T21:32:04.302384+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:04.302384+0000","last_undegraded":"2024-09-15T21:32:04.302384+0000","last_fullsized":"2024-09-15T21:32:04.302384+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20638,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.294770+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:04.294770+0000","last_peered":"2024-09-15T21:32:04.294770+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:04.294770+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":126,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.294508+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:04.294508+0000","last_peered":"2024-09-15T21:32:04.294508+0000","last_clean":"2024-09-15T21:32:04.294508+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:04.294508+0000","last_undegraded":"2024-09-15T21:32:04.294508+0000","last_fullsized":"2024-09-15T21:32:04.294508+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28970,"reported_epoch":55,"state":"peering","last_fresh":"2024-09-15T21:32:03.896943+0000","last_change":"2024-09-15T21:32:04.301856+0000","last_active":"2024-09-15T21:32:04.301856+0000","last_peered":"2024-09-15T21:32:03.896943+0000","last_clean":"2024-09-15T21:21:12.325580+0000","last_became_active":"2024-09-15T21:21:14.874439+0000","last_became_peered":"2024-09-15T21:21:14.874439+0000","last_unstale":"2024-09-15T21:32:03.896943+0000","last_undegraded":"2024-09-15T21:32:03.896943+0000","last_fullsized":"2024-09-15T21:21:14.352559+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29044,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.295516+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:04.295516+0000","last_peered":"2024-09-15T21:32:04.295516+0000","last_clean":"2024-09-15T21:32:04.295516+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:04.295516+0000","last_undegraded":"2024-09-15T21:32:04.295516+0000","last_fullsized":"2024-09-15T21:32:04.295516+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-15T21:32:04.296106+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:04.296106+0000","last_peered":"2024-09-15T21:32:04.296106+0000","last_clean":"2024-09-15T21:32:04.296106+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:04.296106+0000","last_undegraded":"2024-09-15T21:32:04.296106+0000","last_fullsized":"2024-09-15T21:32:04.296106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21030,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:04.295263+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:04.295263+0000","last_peered":"2024-09-15T21:32:04.295263+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:04.295263+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20458,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29016,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":202317824,"data_stored":202202208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":492260,"kb_used_data":131536,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93879580,"statfs":{"total":96636764160,"available":96132689920,"internally_reserved":0,"allocated":134692864,"data_stored":134367814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535848,"kb_used_data":66900,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835992,"statfs":{"total":96636764160,"available":96088055808,"internally_reserved":0,"allocated":68505600,"data_stored":68181393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509788,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862052,"statfs":{"total":96636764160,"available":96114741248,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":134225920,"data_stored":134189790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:10.802 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:10.802 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21253, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:04.295565+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:32:04.295565+0000', 'last_peered': '2024-09-15T21:32:04.295565+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:04.295565+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:10.802 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:10.802 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20797, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:04.294614+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:04.294614+0000', 'last_peered': '2024-09-15T21:32:04.294614+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:04.294614+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:10.802 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:32:10.802 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 20999, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:04.563696+0000', 'last_change': '2024-09-15T21:32:03.897219+0000', 'last_active': '2024-09-15T21:32:04.563696+0000', 'last_peered': '2024-09-15T21:32:04.563696+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:32:04.563696+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4078, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 83, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4076}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:10.803 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:32:10.803 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20638, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:04.294770+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:32:04.294770+0000', 'last_peered': '2024-09-15T21:32:04.294770+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:32:04.294770+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:10.803 INFO:tasks.lost_unfound.ceph_manager:PG 2.1 is not active+clean 2024-09-15T21:32:10.803 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.1', 'version': "49'20643", 'reported_seq': 28970, 'reported_epoch': 55, 'state': 'peering', 'last_fresh': '2024-09-15T21:32:03.896943+0000', 'last_change': '2024-09-15T21:32:04.301856+0000', 'last_active': '2024-09-15T21:32:04.301856+0000', 'last_peered': '2024-09-15T21:32:03.896943+0000', 'last_clean': '2024-09-15T21:21:12.325580+0000', 'last_became_active': '2024-09-15T21:21:14.874439+0000', 'last_became_peered': '2024-09-15T21:21:14.874439+0000', 'last_unstale': '2024-09-15T21:32:03.896943+0000', 'last_undegraded': '2024-09-15T21:32:03.896943+0000', 'last_fullsized': '2024-09-15T21:21:14.352559+0000', 'mapping_epoch': 56, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2164", 'last_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:47.548692+0000', 'objects_scrubbed': 2137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 641, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18502, 'num_write_kb': 32732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4091, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-15T21:32:10.803 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:10.803 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21030, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:04.295263+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:04.295263+0000', 'last_peered': '2024-09-15T21:32:04.295263+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:04.295263+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:10.803 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:32:11.276 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:11.276 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20458/24 objects degraded (85241.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":783,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1574805504,"bytes_avail":288335486976,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":20458,"degraded_total":24,"degraded_ratio":852.41666666666652,"recovering_objects_per_sec":128,"recovering_bytes_per_sec":120,"recovering_keys_per_sec":0,"num_objects_recovered":1031,"num_bytes_recovered":966,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:32:11.290 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:32:11.290 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:11.647 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:11.648 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:11.663 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-09-15T21:32:10.898874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19516,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29959,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1536116,"kb_used_data":197176,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281579404,"statfs":{"total":289910292480,"available":288337309696,"internally_reserved":0,"allocated":201908224,"data_stored":200902871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1973,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1973,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001418"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21255,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305872+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:05.305872+0000","last_peered":"2024-09-15T21:32:05.305872+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:05.305872+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:05.304615+0000","last_peered":"2024-09-15T21:32:05.304615+0000","last_clean":"2024-09-15T21:32:05.304615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:05.304615+0000","last_undegraded":"2024-09-15T21:32:05.304615+0000","last_fullsized":"2024-09-15T21:32:05.304615+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304310+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:05.304310+0000","last_peered":"2024-09-15T21:32:05.304310+0000","last_clean":"2024-09-15T21:32:05.304310+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:05.304310+0000","last_undegraded":"2024-09-15T21:32:05.304310+0000","last_fullsized":"2024-09-15T21:32:05.304310+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304982+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:05.304982+0000","last_peered":"2024-09-15T21:32:05.304982+0000","last_clean":"2024-09-15T21:32:05.304982+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:05.304982+0000","last_undegraded":"2024-09-15T21:32:05.304982+0000","last_fullsized":"2024-09-15T21:32:05.304982+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20799,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304140+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:05.304140+0000","last_peered":"2024-09-15T21:32:05.304140+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:05.304140+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":22885,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:09.569903+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:09.569903+0000","last_peered":"2024-09-15T21:32:09.569903+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:09.569903+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3134}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21024,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303944+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:05.303944+0000","last_peered":"2024-09-15T21:32:05.303944+0000","last_clean":"2024-09-15T21:32:05.303944+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:05.303944+0000","last_undegraded":"2024-09-15T21:32:05.303944+0000","last_fullsized":"2024-09-15T21:32:05.303944+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304006+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:05.304006+0000","last_peered":"2024-09-15T21:32:05.304006+0000","last_clean":"2024-09-15T21:32:05.304006+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:05.304006+0000","last_undegraded":"2024-09-15T21:32:05.304006+0000","last_fullsized":"2024-09-15T21:32:05.304006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20640,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:05.304397+0000","last_peered":"2024-09-15T21:32:05.304397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:05.304397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304050+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:05.304050+0000","last_peered":"2024-09-15T21:32:05.304050+0000","last_clean":"2024-09-15T21:32:05.304050+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:05.304050+0000","last_undegraded":"2024-09-15T21:32:05.304050+0000","last_fullsized":"2024-09-15T21:32:05.304050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28979,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.358402+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:05.358402+0000","last_peered":"2024-09-15T21:32:05.358402+0000","last_clean":"2024-09-15T21:32:05.358402+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:05.358402+0000","last_undegraded":"2024-09-15T21:32:05.358402+0000","last_fullsized":"2024-09-15T21:32:05.358402+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29046,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303826+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:05.303826+0000","last_peered":"2024-09-15T21:32:05.303826+0000","last_clean":"2024-09-15T21:32:05.303826+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:05.303826+0000","last_undegraded":"2024-09-15T21:32:05.303826+0000","last_fullsized":"2024-09-15T21:32:05.303826+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304457+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:05.304457+0000","last_peered":"2024-09-15T21:32:05.304457+0000","last_clean":"2024-09-15T21:32:05.304457+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:05.304457+0000","last_undegraded":"2024-09-15T21:32:05.304457+0000","last_fullsized":"2024-09-15T21:32:05.304457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305161+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:05.305161+0000","last_peered":"2024-09-15T21:32:05.305161+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:05.305161+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19516,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29958,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200474624,"data_stored":200359008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":490464,"kb_used_data":129744,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93881376,"statfs":{"total":96636764160,"available":96134529024,"internally_reserved":0,"allocated":132857856,"data_stored":132527865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535864,"kb_used_data":66916,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835976,"statfs":{"total":96636764160,"available":96088039424,"internally_reserved":0,"allocated":68521984,"data_stored":68187916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509788,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862052,"statfs":{"total":96636764160,"available":96114741248,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":132382720,"data_stored":132346590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:14.665 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:15.051 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:15.052 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:15.064 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-15T21:32:14.899586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30825,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525988,"kb_used_data":195376,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281589532,"statfs":{"total":289910292480,"available":288347680768,"internally_reserved":0,"allocated":200065024,"data_stored":199059671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1362480131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1808,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001810"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21255,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305872+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:05.305872+0000","last_peered":"2024-09-15T21:32:05.305872+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:05.305872+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:05.304615+0000","last_peered":"2024-09-15T21:32:05.304615+0000","last_clean":"2024-09-15T21:32:05.304615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:05.304615+0000","last_undegraded":"2024-09-15T21:32:05.304615+0000","last_fullsized":"2024-09-15T21:32:05.304615+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304310+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:05.304310+0000","last_peered":"2024-09-15T21:32:05.304310+0000","last_clean":"2024-09-15T21:32:05.304310+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:05.304310+0000","last_undegraded":"2024-09-15T21:32:05.304310+0000","last_fullsized":"2024-09-15T21:32:05.304310+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304982+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:05.304982+0000","last_peered":"2024-09-15T21:32:05.304982+0000","last_clean":"2024-09-15T21:32:05.304982+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:05.304982+0000","last_undegraded":"2024-09-15T21:32:05.304982+0000","last_fullsized":"2024-09-15T21:32:05.304982+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20799,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304140+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:05.304140+0000","last_peered":"2024-09-15T21:32:05.304140+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:05.304140+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":24617,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:14.569677+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:14.569677+0000","last_peered":"2024-09-15T21:32:14.569677+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:14.569677+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1891,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21024,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303944+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:05.303944+0000","last_peered":"2024-09-15T21:32:05.303944+0000","last_clean":"2024-09-15T21:32:05.303944+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:05.303944+0000","last_undegraded":"2024-09-15T21:32:05.303944+0000","last_fullsized":"2024-09-15T21:32:05.303944+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304006+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:05.304006+0000","last_peered":"2024-09-15T21:32:05.304006+0000","last_clean":"2024-09-15T21:32:05.304006+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:05.304006+0000","last_undegraded":"2024-09-15T21:32:05.304006+0000","last_fullsized":"2024-09-15T21:32:05.304006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20640,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:05.304397+0000","last_peered":"2024-09-15T21:32:05.304397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:05.304397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304050+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:05.304050+0000","last_peered":"2024-09-15T21:32:05.304050+0000","last_clean":"2024-09-15T21:32:05.304050+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:05.304050+0000","last_undegraded":"2024-09-15T21:32:05.304050+0000","last_fullsized":"2024-09-15T21:32:05.304050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28979,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.358402+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:05.358402+0000","last_peered":"2024-09-15T21:32:05.358402+0000","last_clean":"2024-09-15T21:32:05.358402+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:05.358402+0000","last_undegraded":"2024-09-15T21:32:05.358402+0000","last_fullsized":"2024-09-15T21:32:05.358402+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29046,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303826+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:05.303826+0000","last_peered":"2024-09-15T21:32:05.303826+0000","last_clean":"2024-09-15T21:32:05.303826+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:05.303826+0000","last_undegraded":"2024-09-15T21:32:05.303826+0000","last_fullsized":"2024-09-15T21:32:05.303826+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304457+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:05.304457+0000","last_peered":"2024-09-15T21:32:05.304457+0000","last_clean":"2024-09-15T21:32:05.304457+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:05.304457+0000","last_undegraded":"2024-09-15T21:32:05.304457+0000","last_fullsized":"2024-09-15T21:32:05.304457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305161+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:05.305161+0000","last_peered":"2024-09-15T21:32:05.305161+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:05.305161+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30824,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":198631424,"data_stored":198515808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488664,"kb_used_data":127944,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93883176,"statfs":{"total":96636764160,"available":96136372224,"internally_reserved":0,"allocated":131014656,"data_stored":130684665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535864,"kb_used_data":66916,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835976,"statfs":{"total":96636764160,"available":96088039424,"internally_reserved":0,"allocated":68521984,"data_stored":68187916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501460,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870380,"statfs":{"total":96636764160,"available":96123269120,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130539520,"data_stored":130503390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:15.065 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:15.369 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:15.370 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:15.382 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-15T21:32:14.899586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30825,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525988,"kb_used_data":195376,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281589532,"statfs":{"total":289910292480,"available":288347680768,"internally_reserved":0,"allocated":200065024,"data_stored":199059671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1362480131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1808,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001810"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21255,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305872+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:05.305872+0000","last_peered":"2024-09-15T21:32:05.305872+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:05.305872+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:05.304615+0000","last_peered":"2024-09-15T21:32:05.304615+0000","last_clean":"2024-09-15T21:32:05.304615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:05.304615+0000","last_undegraded":"2024-09-15T21:32:05.304615+0000","last_fullsized":"2024-09-15T21:32:05.304615+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304310+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:05.304310+0000","last_peered":"2024-09-15T21:32:05.304310+0000","last_clean":"2024-09-15T21:32:05.304310+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:05.304310+0000","last_undegraded":"2024-09-15T21:32:05.304310+0000","last_fullsized":"2024-09-15T21:32:05.304310+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304982+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:05.304982+0000","last_peered":"2024-09-15T21:32:05.304982+0000","last_clean":"2024-09-15T21:32:05.304982+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:05.304982+0000","last_undegraded":"2024-09-15T21:32:05.304982+0000","last_fullsized":"2024-09-15T21:32:05.304982+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20799,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304140+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:05.304140+0000","last_peered":"2024-09-15T21:32:05.304140+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:05.304140+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":24617,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:14.569677+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:14.569677+0000","last_peered":"2024-09-15T21:32:14.569677+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:14.569677+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1891,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21024,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303944+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:05.303944+0000","last_peered":"2024-09-15T21:32:05.303944+0000","last_clean":"2024-09-15T21:32:05.303944+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:05.303944+0000","last_undegraded":"2024-09-15T21:32:05.303944+0000","last_fullsized":"2024-09-15T21:32:05.303944+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304006+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:05.304006+0000","last_peered":"2024-09-15T21:32:05.304006+0000","last_clean":"2024-09-15T21:32:05.304006+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:05.304006+0000","last_undegraded":"2024-09-15T21:32:05.304006+0000","last_fullsized":"2024-09-15T21:32:05.304006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20640,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:05.304397+0000","last_peered":"2024-09-15T21:32:05.304397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:05.304397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304050+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:05.304050+0000","last_peered":"2024-09-15T21:32:05.304050+0000","last_clean":"2024-09-15T21:32:05.304050+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:05.304050+0000","last_undegraded":"2024-09-15T21:32:05.304050+0000","last_fullsized":"2024-09-15T21:32:05.304050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28979,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.358402+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:05.358402+0000","last_peered":"2024-09-15T21:32:05.358402+0000","last_clean":"2024-09-15T21:32:05.358402+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:05.358402+0000","last_undegraded":"2024-09-15T21:32:05.358402+0000","last_fullsized":"2024-09-15T21:32:05.358402+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29046,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303826+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:05.303826+0000","last_peered":"2024-09-15T21:32:05.303826+0000","last_clean":"2024-09-15T21:32:05.303826+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:05.303826+0000","last_undegraded":"2024-09-15T21:32:05.303826+0000","last_fullsized":"2024-09-15T21:32:05.303826+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304457+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:05.304457+0000","last_peered":"2024-09-15T21:32:05.304457+0000","last_clean":"2024-09-15T21:32:05.304457+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:05.304457+0000","last_undegraded":"2024-09-15T21:32:05.304457+0000","last_fullsized":"2024-09-15T21:32:05.304457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305161+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:05.305161+0000","last_peered":"2024-09-15T21:32:05.305161+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:05.305161+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30824,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":198631424,"data_stored":198515808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488664,"kb_used_data":127944,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93883176,"statfs":{"total":96636764160,"available":96136372224,"internally_reserved":0,"allocated":131014656,"data_stored":130684665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535864,"kb_used_data":66916,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835976,"statfs":{"total":96636764160,"available":96088039424,"internally_reserved":0,"allocated":68521984,"data_stored":68187916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501460,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870380,"statfs":{"total":96636764160,"available":96123269120,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130539520,"data_stored":130503390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:15.383 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:15.383 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21255, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.305872+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:32:05.305872+0000', 'last_peered': '2024-09-15T21:32:05.305872+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:05.305872+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:15.384 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:15.384 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20799, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.304140+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:05.304140+0000', 'last_peered': '2024-09-15T21:32:05.304140+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:05.304140+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:15.384 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:32:15.384 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 24617, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:14.569677+0000', 'last_change': '2024-09-15T21:32:03.897219+0000', 'last_active': '2024-09-15T21:32:14.569677+0000', 'last_peered': '2024-09-15T21:32:14.569677+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:32:14.569677+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2270, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1891, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:15.384 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:32:15.384 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20640, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.304397+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:32:05.304397+0000', 'last_peered': '2024-09-15T21:32:05.304397+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:32:05.304397+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:15.384 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:15.384 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21032, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.305161+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:05.305161+0000', 'last_peered': '2024-09-15T21:32:05.305161+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:05.305161+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:15.385 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:32:15.755 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:15.755 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18650/24 objects degraded (77708.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":787,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1562611712,"bytes_avail":288347680768,"bytes_total":289910292480,"degraded_objects":18650,"degraded_total":24,"degraded_ratio":777.08333333333348,"recovering_objects_per_sec":180,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1808,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:32:15.771 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:32:15.771 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:16.083 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:16.084 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:16.098 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-15T21:32:14.899586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30825,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525988,"kb_used_data":195376,"kb_used_omap":12,"kb_used_meta":1330547,"kb_avail":281589532,"statfs":{"total":289910292480,"available":288347680768,"internally_reserved":0,"allocated":200065024,"data_stored":199059671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1362480131},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1808,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001810"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21255,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305872+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:05.305872+0000","last_peered":"2024-09-15T21:32:05.305872+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:05.305872+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:05.304615+0000","last_peered":"2024-09-15T21:32:05.304615+0000","last_clean":"2024-09-15T21:32:05.304615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:05.304615+0000","last_undegraded":"2024-09-15T21:32:05.304615+0000","last_fullsized":"2024-09-15T21:32:05.304615+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304310+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:05.304310+0000","last_peered":"2024-09-15T21:32:05.304310+0000","last_clean":"2024-09-15T21:32:05.304310+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:05.304310+0000","last_undegraded":"2024-09-15T21:32:05.304310+0000","last_fullsized":"2024-09-15T21:32:05.304310+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304982+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:05.304982+0000","last_peered":"2024-09-15T21:32:05.304982+0000","last_clean":"2024-09-15T21:32:05.304982+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:05.304982+0000","last_undegraded":"2024-09-15T21:32:05.304982+0000","last_fullsized":"2024-09-15T21:32:05.304982+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20799,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304140+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:05.304140+0000","last_peered":"2024-09-15T21:32:05.304140+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:05.304140+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":24617,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:14.569677+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:14.569677+0000","last_peered":"2024-09-15T21:32:14.569677+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:14.569677+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1891,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21024,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303944+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:05.303944+0000","last_peered":"2024-09-15T21:32:05.303944+0000","last_clean":"2024-09-15T21:32:05.303944+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:05.303944+0000","last_undegraded":"2024-09-15T21:32:05.303944+0000","last_fullsized":"2024-09-15T21:32:05.303944+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304006+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:05.304006+0000","last_peered":"2024-09-15T21:32:05.304006+0000","last_clean":"2024-09-15T21:32:05.304006+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:05.304006+0000","last_undegraded":"2024-09-15T21:32:05.304006+0000","last_fullsized":"2024-09-15T21:32:05.304006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20640,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:05.304397+0000","last_peered":"2024-09-15T21:32:05.304397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:05.304397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304050+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:05.304050+0000","last_peered":"2024-09-15T21:32:05.304050+0000","last_clean":"2024-09-15T21:32:05.304050+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:05.304050+0000","last_undegraded":"2024-09-15T21:32:05.304050+0000","last_fullsized":"2024-09-15T21:32:05.304050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28979,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.358402+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:05.358402+0000","last_peered":"2024-09-15T21:32:05.358402+0000","last_clean":"2024-09-15T21:32:05.358402+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:05.358402+0000","last_undegraded":"2024-09-15T21:32:05.358402+0000","last_fullsized":"2024-09-15T21:32:05.358402+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29046,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303826+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:05.303826+0000","last_peered":"2024-09-15T21:32:05.303826+0000","last_clean":"2024-09-15T21:32:05.303826+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:05.303826+0000","last_undegraded":"2024-09-15T21:32:05.303826+0000","last_fullsized":"2024-09-15T21:32:05.303826+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304457+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:05.304457+0000","last_peered":"2024-09-15T21:32:05.304457+0000","last_clean":"2024-09-15T21:32:05.304457+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:05.304457+0000","last_undegraded":"2024-09-15T21:32:05.304457+0000","last_fullsized":"2024-09-15T21:32:05.304457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305161+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:05.305161+0000","last_peered":"2024-09-15T21:32:05.305161+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:05.305161+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30824,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":198631424,"data_stored":198515808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488664,"kb_used_data":127944,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93883176,"statfs":{"total":96636764160,"available":96136372224,"internally_reserved":0,"allocated":131014656,"data_stored":130684665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535864,"kb_used_data":66916,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835976,"statfs":{"total":96636764160,"available":96088039424,"internally_reserved":0,"allocated":68521984,"data_stored":68187916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501460,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870380,"statfs":{"total":96636764160,"available":96123269120,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512945839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130539520,"data_stored":130503390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:19.100 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:19.406 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:19.407 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:19.419 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-15T21:32:18.900229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30825,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534308,"kb_used_data":195376,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281581212,"statfs":{"total":289910292480,"available":288339161088,"internally_reserved":0,"allocated":200065024,"data_stored":199059671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1808,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002108"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21255,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305872+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:05.305872+0000","last_peered":"2024-09-15T21:32:05.305872+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:05.305872+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:05.304615+0000","last_peered":"2024-09-15T21:32:05.304615+0000","last_clean":"2024-09-15T21:32:05.304615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:05.304615+0000","last_undegraded":"2024-09-15T21:32:05.304615+0000","last_fullsized":"2024-09-15T21:32:05.304615+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304310+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:05.304310+0000","last_peered":"2024-09-15T21:32:05.304310+0000","last_clean":"2024-09-15T21:32:05.304310+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:05.304310+0000","last_undegraded":"2024-09-15T21:32:05.304310+0000","last_fullsized":"2024-09-15T21:32:05.304310+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304982+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:05.304982+0000","last_peered":"2024-09-15T21:32:05.304982+0000","last_clean":"2024-09-15T21:32:05.304982+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:05.304982+0000","last_undegraded":"2024-09-15T21:32:05.304982+0000","last_fullsized":"2024-09-15T21:32:05.304982+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20799,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304140+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:05.304140+0000","last_peered":"2024-09-15T21:32:05.304140+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:05.304140+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":24617,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:14.569677+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:14.569677+0000","last_peered":"2024-09-15T21:32:14.569677+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:14.569677+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1891,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21024,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303944+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:05.303944+0000","last_peered":"2024-09-15T21:32:05.303944+0000","last_clean":"2024-09-15T21:32:05.303944+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:05.303944+0000","last_undegraded":"2024-09-15T21:32:05.303944+0000","last_fullsized":"2024-09-15T21:32:05.303944+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304006+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:05.304006+0000","last_peered":"2024-09-15T21:32:05.304006+0000","last_clean":"2024-09-15T21:32:05.304006+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:05.304006+0000","last_undegraded":"2024-09-15T21:32:05.304006+0000","last_fullsized":"2024-09-15T21:32:05.304006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20640,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:05.304397+0000","last_peered":"2024-09-15T21:32:05.304397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:05.304397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304050+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:05.304050+0000","last_peered":"2024-09-15T21:32:05.304050+0000","last_clean":"2024-09-15T21:32:05.304050+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:05.304050+0000","last_undegraded":"2024-09-15T21:32:05.304050+0000","last_fullsized":"2024-09-15T21:32:05.304050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28979,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.358402+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:05.358402+0000","last_peered":"2024-09-15T21:32:05.358402+0000","last_clean":"2024-09-15T21:32:05.358402+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:05.358402+0000","last_undegraded":"2024-09-15T21:32:05.358402+0000","last_fullsized":"2024-09-15T21:32:05.358402+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29046,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303826+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:05.303826+0000","last_peered":"2024-09-15T21:32:05.303826+0000","last_clean":"2024-09-15T21:32:05.303826+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:05.303826+0000","last_undegraded":"2024-09-15T21:32:05.303826+0000","last_fullsized":"2024-09-15T21:32:05.303826+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304457+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:05.304457+0000","last_peered":"2024-09-15T21:32:05.304457+0000","last_clean":"2024-09-15T21:32:05.304457+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:05.304457+0000","last_undegraded":"2024-09-15T21:32:05.304457+0000","last_fullsized":"2024-09-15T21:32:05.304457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305161+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:05.305161+0000","last_peered":"2024-09-15T21:32:05.305161+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:05.305161+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30824,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":198631424,"data_stored":198515808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488664,"kb_used_data":127944,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93883176,"statfs":{"total":96636764160,"available":96136372224,"internally_reserved":0,"allocated":131014656,"data_stored":130684665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535864,"kb_used_data":66916,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835976,"statfs":{"total":96636764160,"available":96088039424,"internally_reserved":0,"allocated":68521984,"data_stored":68187916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509780,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862060,"statfs":{"total":96636764160,"available":96114749440,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130539520,"data_stored":130503390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:19.420 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:19.730 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:19.730 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:19.742 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-15T21:32:18.900229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30825,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534308,"kb_used_data":195376,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281581212,"statfs":{"total":289910292480,"available":288339161088,"internally_reserved":0,"allocated":200065024,"data_stored":199059671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1808,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002108"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21255,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305872+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:05.305872+0000","last_peered":"2024-09-15T21:32:05.305872+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:05.305872+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:05.304615+0000","last_peered":"2024-09-15T21:32:05.304615+0000","last_clean":"2024-09-15T21:32:05.304615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:05.304615+0000","last_undegraded":"2024-09-15T21:32:05.304615+0000","last_fullsized":"2024-09-15T21:32:05.304615+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304310+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:05.304310+0000","last_peered":"2024-09-15T21:32:05.304310+0000","last_clean":"2024-09-15T21:32:05.304310+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:05.304310+0000","last_undegraded":"2024-09-15T21:32:05.304310+0000","last_fullsized":"2024-09-15T21:32:05.304310+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304982+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:05.304982+0000","last_peered":"2024-09-15T21:32:05.304982+0000","last_clean":"2024-09-15T21:32:05.304982+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:05.304982+0000","last_undegraded":"2024-09-15T21:32:05.304982+0000","last_fullsized":"2024-09-15T21:32:05.304982+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20799,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304140+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:05.304140+0000","last_peered":"2024-09-15T21:32:05.304140+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:05.304140+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":24617,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:14.569677+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:14.569677+0000","last_peered":"2024-09-15T21:32:14.569677+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:14.569677+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1891,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21024,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303944+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:05.303944+0000","last_peered":"2024-09-15T21:32:05.303944+0000","last_clean":"2024-09-15T21:32:05.303944+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:05.303944+0000","last_undegraded":"2024-09-15T21:32:05.303944+0000","last_fullsized":"2024-09-15T21:32:05.303944+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304006+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:05.304006+0000","last_peered":"2024-09-15T21:32:05.304006+0000","last_clean":"2024-09-15T21:32:05.304006+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:05.304006+0000","last_undegraded":"2024-09-15T21:32:05.304006+0000","last_fullsized":"2024-09-15T21:32:05.304006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20640,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:05.304397+0000","last_peered":"2024-09-15T21:32:05.304397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:05.304397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304050+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:05.304050+0000","last_peered":"2024-09-15T21:32:05.304050+0000","last_clean":"2024-09-15T21:32:05.304050+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:05.304050+0000","last_undegraded":"2024-09-15T21:32:05.304050+0000","last_fullsized":"2024-09-15T21:32:05.304050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28979,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.358402+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:05.358402+0000","last_peered":"2024-09-15T21:32:05.358402+0000","last_clean":"2024-09-15T21:32:05.358402+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:05.358402+0000","last_undegraded":"2024-09-15T21:32:05.358402+0000","last_fullsized":"2024-09-15T21:32:05.358402+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29046,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303826+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:05.303826+0000","last_peered":"2024-09-15T21:32:05.303826+0000","last_clean":"2024-09-15T21:32:05.303826+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:05.303826+0000","last_undegraded":"2024-09-15T21:32:05.303826+0000","last_fullsized":"2024-09-15T21:32:05.303826+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304457+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:05.304457+0000","last_peered":"2024-09-15T21:32:05.304457+0000","last_clean":"2024-09-15T21:32:05.304457+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:05.304457+0000","last_undegraded":"2024-09-15T21:32:05.304457+0000","last_fullsized":"2024-09-15T21:32:05.304457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305161+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:05.305161+0000","last_peered":"2024-09-15T21:32:05.305161+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:05.305161+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30824,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":198631424,"data_stored":198515808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488664,"kb_used_data":127944,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93883176,"statfs":{"total":96636764160,"available":96136372224,"internally_reserved":0,"allocated":131014656,"data_stored":130684665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535864,"kb_used_data":66916,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835976,"statfs":{"total":96636764160,"available":96088039424,"internally_reserved":0,"allocated":68521984,"data_stored":68187916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509780,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862060,"statfs":{"total":96636764160,"available":96114749440,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130539520,"data_stored":130503390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:19.743 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:19.743 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21255, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.305872+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:32:05.305872+0000', 'last_peered': '2024-09-15T21:32:05.305872+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:05.305872+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:19.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:19.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20799, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.304140+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:05.304140+0000', 'last_peered': '2024-09-15T21:32:05.304140+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:05.304140+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:19.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:32:19.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 24617, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:14.569677+0000', 'last_change': '2024-09-15T21:32:03.897219+0000', 'last_active': '2024-09-15T21:32:14.569677+0000', 'last_peered': '2024-09-15T21:32:14.569677+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:32:14.569677+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2270, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1891, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:19.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:32:19.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20640, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.304397+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:32:05.304397+0000', 'last_peered': '2024-09-15T21:32:05.304397+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:32:05.304397+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:19.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:19.744 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21032, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.305161+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:05.305161+0000', 'last_peered': '2024-09-15T21:32:05.305161+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:05.305161+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:19.745 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:32:20.131 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:20.132 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18650/24 objects degraded (77708.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":791,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1571131392,"bytes_avail":288339161088,"bytes_total":289910292480,"degraded_objects":18650,"degraded_total":24,"degraded_ratio":777.08333333333348,"recovering_objects_per_sec":150,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1808,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:32:20.145 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:32:20.145 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:20.490 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:20.490 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:20.503 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-15T21:32:18.900229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30825,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534308,"kb_used_data":195376,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281581212,"statfs":{"total":289910292480,"available":288339161088,"internally_reserved":0,"allocated":200065024,"data_stored":199059671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1370999811},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1808,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1808,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002108"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21255,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305872+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:05.305872+0000","last_peered":"2024-09-15T21:32:05.305872+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:05.305872+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:05.304615+0000","last_peered":"2024-09-15T21:32:05.304615+0000","last_clean":"2024-09-15T21:32:05.304615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:05.304615+0000","last_undegraded":"2024-09-15T21:32:05.304615+0000","last_fullsized":"2024-09-15T21:32:05.304615+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304310+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:05.304310+0000","last_peered":"2024-09-15T21:32:05.304310+0000","last_clean":"2024-09-15T21:32:05.304310+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:05.304310+0000","last_undegraded":"2024-09-15T21:32:05.304310+0000","last_fullsized":"2024-09-15T21:32:05.304310+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304982+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:05.304982+0000","last_peered":"2024-09-15T21:32:05.304982+0000","last_clean":"2024-09-15T21:32:05.304982+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:05.304982+0000","last_undegraded":"2024-09-15T21:32:05.304982+0000","last_fullsized":"2024-09-15T21:32:05.304982+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20799,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304140+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:05.304140+0000","last_peered":"2024-09-15T21:32:05.304140+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:05.304140+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":24617,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:14.569677+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:14.569677+0000","last_peered":"2024-09-15T21:32:14.569677+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:14.569677+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1891,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21024,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303944+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:05.303944+0000","last_peered":"2024-09-15T21:32:05.303944+0000","last_clean":"2024-09-15T21:32:05.303944+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:05.303944+0000","last_undegraded":"2024-09-15T21:32:05.303944+0000","last_fullsized":"2024-09-15T21:32:05.303944+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304006+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:05.304006+0000","last_peered":"2024-09-15T21:32:05.304006+0000","last_clean":"2024-09-15T21:32:05.304006+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:05.304006+0000","last_undegraded":"2024-09-15T21:32:05.304006+0000","last_fullsized":"2024-09-15T21:32:05.304006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20640,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:05.304397+0000","last_peered":"2024-09-15T21:32:05.304397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:05.304397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304050+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:05.304050+0000","last_peered":"2024-09-15T21:32:05.304050+0000","last_clean":"2024-09-15T21:32:05.304050+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:05.304050+0000","last_undegraded":"2024-09-15T21:32:05.304050+0000","last_fullsized":"2024-09-15T21:32:05.304050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28979,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.358402+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:05.358402+0000","last_peered":"2024-09-15T21:32:05.358402+0000","last_clean":"2024-09-15T21:32:05.358402+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:05.358402+0000","last_undegraded":"2024-09-15T21:32:05.358402+0000","last_fullsized":"2024-09-15T21:32:05.358402+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29046,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303826+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:05.303826+0000","last_peered":"2024-09-15T21:32:05.303826+0000","last_clean":"2024-09-15T21:32:05.303826+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:05.303826+0000","last_undegraded":"2024-09-15T21:32:05.303826+0000","last_fullsized":"2024-09-15T21:32:05.303826+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304457+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:05.304457+0000","last_peered":"2024-09-15T21:32:05.304457+0000","last_clean":"2024-09-15T21:32:05.304457+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:05.304457+0000","last_undegraded":"2024-09-15T21:32:05.304457+0000","last_fullsized":"2024-09-15T21:32:05.304457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305161+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:05.305161+0000","last_peered":"2024-09-15T21:32:05.305161+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:05.305161+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30824,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":198631424,"data_stored":198515808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488664,"kb_used_data":127944,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93883176,"statfs":{"total":96636764160,"available":96136372224,"internally_reserved":0,"allocated":131014656,"data_stored":130684665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535864,"kb_used_data":66916,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835976,"statfs":{"total":96636764160,"available":96088039424,"internally_reserved":0,"allocated":68521984,"data_stored":68187916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509780,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862060,"statfs":{"total":96636764160,"available":96114749440,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521465519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130539520,"data_stored":130503390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:23.505 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:23.802 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:23.814 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-15T21:32:22.900871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31591,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520028,"kb_used_data":193576,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281595492,"statfs":{"total":289910292480,"available":288353783808,"internally_reserved":0,"allocated":198221824,"data_stored":197216471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1358220291},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1632,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001996"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21255,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305872+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:05.305872+0000","last_peered":"2024-09-15T21:32:05.305872+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:05.305872+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:05.304615+0000","last_peered":"2024-09-15T21:32:05.304615+0000","last_clean":"2024-09-15T21:32:05.304615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:05.304615+0000","last_undegraded":"2024-09-15T21:32:05.304615+0000","last_fullsized":"2024-09-15T21:32:05.304615+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304310+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:05.304310+0000","last_peered":"2024-09-15T21:32:05.304310+0000","last_clean":"2024-09-15T21:32:05.304310+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:05.304310+0000","last_undegraded":"2024-09-15T21:32:05.304310+0000","last_fullsized":"2024-09-15T21:32:05.304310+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304982+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:05.304982+0000","last_peered":"2024-09-15T21:32:05.304982+0000","last_clean":"2024-09-15T21:32:05.304982+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:05.304982+0000","last_undegraded":"2024-09-15T21:32:05.304982+0000","last_fullsized":"2024-09-15T21:32:05.304982+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20799,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304140+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:05.304140+0000","last_peered":"2024-09-15T21:32:05.304140+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:05.304140+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":26149,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:19.573109+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:19.573109+0000","last_peered":"2024-09-15T21:32:19.573109+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:19.573109+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1504,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2657,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21024,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303944+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:05.303944+0000","last_peered":"2024-09-15T21:32:05.303944+0000","last_clean":"2024-09-15T21:32:05.303944+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:05.303944+0000","last_undegraded":"2024-09-15T21:32:05.303944+0000","last_fullsized":"2024-09-15T21:32:05.303944+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304006+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:05.304006+0000","last_peered":"2024-09-15T21:32:05.304006+0000","last_clean":"2024-09-15T21:32:05.304006+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:05.304006+0000","last_undegraded":"2024-09-15T21:32:05.304006+0000","last_fullsized":"2024-09-15T21:32:05.304006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20640,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:05.304397+0000","last_peered":"2024-09-15T21:32:05.304397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:05.304397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304050+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:05.304050+0000","last_peered":"2024-09-15T21:32:05.304050+0000","last_clean":"2024-09-15T21:32:05.304050+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:05.304050+0000","last_undegraded":"2024-09-15T21:32:05.304050+0000","last_fullsized":"2024-09-15T21:32:05.304050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28979,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.358402+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:05.358402+0000","last_peered":"2024-09-15T21:32:05.358402+0000","last_clean":"2024-09-15T21:32:05.358402+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:05.358402+0000","last_undegraded":"2024-09-15T21:32:05.358402+0000","last_fullsized":"2024-09-15T21:32:05.358402+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29046,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303826+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:05.303826+0000","last_peered":"2024-09-15T21:32:05.303826+0000","last_clean":"2024-09-15T21:32:05.303826+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:05.303826+0000","last_undegraded":"2024-09-15T21:32:05.303826+0000","last_fullsized":"2024-09-15T21:32:05.303826+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304457+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:05.304457+0000","last_peered":"2024-09-15T21:32:05.304457+0000","last_clean":"2024-09-15T21:32:05.304457+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:05.304457+0000","last_undegraded":"2024-09-15T21:32:05.304457+0000","last_fullsized":"2024-09-15T21:32:05.304457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305161+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:05.305161+0000","last_peered":"2024-09-15T21:32:05.305161+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:05.305161+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31590,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196788224,"data_stored":196672608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486864,"kb_used_data":126144,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93884976,"statfs":{"total":96636764160,"available":96138215424,"internally_reserved":0,"allocated":129171456,"data_stored":128841465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535864,"kb_used_data":66916,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835976,"statfs":{"total":96636764160,"available":96088039424,"internally_reserved":0,"allocated":68521984,"data_stored":68187916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128696320,"data_stored":128660190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:23.815 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:24.126 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:24.127 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:24.139 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-15T21:32:22.900871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31591,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520028,"kb_used_data":193576,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281595492,"statfs":{"total":289910292480,"available":288353783808,"internally_reserved":0,"allocated":198221824,"data_stored":197216471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1358220291},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1632,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001996"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21255,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305872+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:05.305872+0000","last_peered":"2024-09-15T21:32:05.305872+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:05.305872+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:05.304615+0000","last_peered":"2024-09-15T21:32:05.304615+0000","last_clean":"2024-09-15T21:32:05.304615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:05.304615+0000","last_undegraded":"2024-09-15T21:32:05.304615+0000","last_fullsized":"2024-09-15T21:32:05.304615+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304310+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:05.304310+0000","last_peered":"2024-09-15T21:32:05.304310+0000","last_clean":"2024-09-15T21:32:05.304310+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:05.304310+0000","last_undegraded":"2024-09-15T21:32:05.304310+0000","last_fullsized":"2024-09-15T21:32:05.304310+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304982+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:05.304982+0000","last_peered":"2024-09-15T21:32:05.304982+0000","last_clean":"2024-09-15T21:32:05.304982+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:05.304982+0000","last_undegraded":"2024-09-15T21:32:05.304982+0000","last_fullsized":"2024-09-15T21:32:05.304982+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20799,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304140+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:05.304140+0000","last_peered":"2024-09-15T21:32:05.304140+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:05.304140+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":26149,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:19.573109+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:19.573109+0000","last_peered":"2024-09-15T21:32:19.573109+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:19.573109+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1504,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2657,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21024,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303944+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:05.303944+0000","last_peered":"2024-09-15T21:32:05.303944+0000","last_clean":"2024-09-15T21:32:05.303944+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:05.303944+0000","last_undegraded":"2024-09-15T21:32:05.303944+0000","last_fullsized":"2024-09-15T21:32:05.303944+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304006+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:05.304006+0000","last_peered":"2024-09-15T21:32:05.304006+0000","last_clean":"2024-09-15T21:32:05.304006+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:05.304006+0000","last_undegraded":"2024-09-15T21:32:05.304006+0000","last_fullsized":"2024-09-15T21:32:05.304006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20640,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:05.304397+0000","last_peered":"2024-09-15T21:32:05.304397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:05.304397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304050+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:05.304050+0000","last_peered":"2024-09-15T21:32:05.304050+0000","last_clean":"2024-09-15T21:32:05.304050+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:05.304050+0000","last_undegraded":"2024-09-15T21:32:05.304050+0000","last_fullsized":"2024-09-15T21:32:05.304050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28979,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.358402+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:05.358402+0000","last_peered":"2024-09-15T21:32:05.358402+0000","last_clean":"2024-09-15T21:32:05.358402+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:05.358402+0000","last_undegraded":"2024-09-15T21:32:05.358402+0000","last_fullsized":"2024-09-15T21:32:05.358402+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29046,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303826+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:05.303826+0000","last_peered":"2024-09-15T21:32:05.303826+0000","last_clean":"2024-09-15T21:32:05.303826+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:05.303826+0000","last_undegraded":"2024-09-15T21:32:05.303826+0000","last_fullsized":"2024-09-15T21:32:05.303826+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304457+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:05.304457+0000","last_peered":"2024-09-15T21:32:05.304457+0000","last_clean":"2024-09-15T21:32:05.304457+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:05.304457+0000","last_undegraded":"2024-09-15T21:32:05.304457+0000","last_fullsized":"2024-09-15T21:32:05.304457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305161+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:05.305161+0000","last_peered":"2024-09-15T21:32:05.305161+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:05.305161+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31590,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196788224,"data_stored":196672608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486864,"kb_used_data":126144,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93884976,"statfs":{"total":96636764160,"available":96138215424,"internally_reserved":0,"allocated":129171456,"data_stored":128841465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535864,"kb_used_data":66916,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835976,"statfs":{"total":96636764160,"available":96088039424,"internally_reserved":0,"allocated":68521984,"data_stored":68187916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128696320,"data_stored":128660190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:24.140 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:24.140 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21255, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.305872+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:32:05.305872+0000', 'last_peered': '2024-09-15T21:32:05.305872+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:05.305872+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:24.140 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:24.140 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20799, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.304140+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:05.304140+0000', 'last_peered': '2024-09-15T21:32:05.304140+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:05.304140+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:24.141 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:32:24.141 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 26149, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:19.573109+0000', 'last_change': '2024-09-15T21:32:03.897219+0000', 'last_active': '2024-09-15T21:32:19.573109+0000', 'last_peered': '2024-09-15T21:32:19.573109+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:32:19.573109+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1504, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2657, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1502}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:24.141 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:32:24.141 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20640, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.304397+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:32:05.304397+0000', 'last_peered': '2024-09-15T21:32:05.304397+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:32:05.304397+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:24.141 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:24.141 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21032, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.305161+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:05.305161+0000', 'last_peered': '2024-09-15T21:32:05.305161+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:05.305161+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:24.141 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:32:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:24.516 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17884/24 objects degraded (74516.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":796,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1556508672,"bytes_avail":288353783808,"bytes_total":289910292480,"degraded_objects":17884,"degraded_total":24,"degraded_ratio":745.16666666666652,"recovering_objects_per_sec":135,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1632,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:32:24.530 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:32:24.530 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:24.831 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:24.831 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:24.843 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-15T21:32:22.900871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31591,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520028,"kb_used_data":193576,"kb_used_omap":12,"kb_used_meta":1326387,"kb_avail":281595492,"statfs":{"total":289910292480,"available":288353783808,"internally_reserved":0,"allocated":198221824,"data_stored":197216471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1358220291},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1632,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1632,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001996"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21255,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305872+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:05.305872+0000","last_peered":"2024-09-15T21:32:05.305872+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:05.305872+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:05.304615+0000","last_peered":"2024-09-15T21:32:05.304615+0000","last_clean":"2024-09-15T21:32:05.304615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:05.304615+0000","last_undegraded":"2024-09-15T21:32:05.304615+0000","last_fullsized":"2024-09-15T21:32:05.304615+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304310+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:05.304310+0000","last_peered":"2024-09-15T21:32:05.304310+0000","last_clean":"2024-09-15T21:32:05.304310+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:05.304310+0000","last_undegraded":"2024-09-15T21:32:05.304310+0000","last_fullsized":"2024-09-15T21:32:05.304310+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304982+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:05.304982+0000","last_peered":"2024-09-15T21:32:05.304982+0000","last_clean":"2024-09-15T21:32:05.304982+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:05.304982+0000","last_undegraded":"2024-09-15T21:32:05.304982+0000","last_fullsized":"2024-09-15T21:32:05.304982+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20799,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304140+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:05.304140+0000","last_peered":"2024-09-15T21:32:05.304140+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:05.304140+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":26149,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:19.573109+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:19.573109+0000","last_peered":"2024-09-15T21:32:19.573109+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:19.573109+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1504,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2657,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1502}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21024,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303944+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:05.303944+0000","last_peered":"2024-09-15T21:32:05.303944+0000","last_clean":"2024-09-15T21:32:05.303944+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:05.303944+0000","last_undegraded":"2024-09-15T21:32:05.303944+0000","last_fullsized":"2024-09-15T21:32:05.303944+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304006+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:05.304006+0000","last_peered":"2024-09-15T21:32:05.304006+0000","last_clean":"2024-09-15T21:32:05.304006+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:05.304006+0000","last_undegraded":"2024-09-15T21:32:05.304006+0000","last_fullsized":"2024-09-15T21:32:05.304006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20640,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:05.304397+0000","last_peered":"2024-09-15T21:32:05.304397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:05.304397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304050+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:05.304050+0000","last_peered":"2024-09-15T21:32:05.304050+0000","last_clean":"2024-09-15T21:32:05.304050+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:05.304050+0000","last_undegraded":"2024-09-15T21:32:05.304050+0000","last_fullsized":"2024-09-15T21:32:05.304050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28979,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.358402+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:05.358402+0000","last_peered":"2024-09-15T21:32:05.358402+0000","last_clean":"2024-09-15T21:32:05.358402+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:05.358402+0000","last_undegraded":"2024-09-15T21:32:05.358402+0000","last_fullsized":"2024-09-15T21:32:05.358402+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29046,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303826+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:05.303826+0000","last_peered":"2024-09-15T21:32:05.303826+0000","last_clean":"2024-09-15T21:32:05.303826+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:05.303826+0000","last_undegraded":"2024-09-15T21:32:05.303826+0000","last_fullsized":"2024-09-15T21:32:05.303826+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304457+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:05.304457+0000","last_peered":"2024-09-15T21:32:05.304457+0000","last_clean":"2024-09-15T21:32:05.304457+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:05.304457+0000","last_undegraded":"2024-09-15T21:32:05.304457+0000","last_fullsized":"2024-09-15T21:32:05.304457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305161+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:05.305161+0000","last_peered":"2024-09-15T21:32:05.305161+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:05.305161+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31590,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":196788224,"data_stored":196672608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":486864,"kb_used_data":126144,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93884976,"statfs":{"total":96636764160,"available":96138215424,"internally_reserved":0,"allocated":129171456,"data_stored":128841465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535864,"kb_used_data":66916,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835976,"statfs":{"total":96636764160,"available":96088039424,"internally_reserved":0,"allocated":68521984,"data_stored":68187916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497300,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874540,"statfs":{"total":96636764160,"available":96127528960,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508685999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":5796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":128696320,"data_stored":128660190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:27.846 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:28.261 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:28.262 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:28.275 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-09-15T21:32:27.348242+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32611,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544596,"kb_used_data":191776,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281570924,"statfs":{"total":289910292480,"available":288328626176,"internally_reserved":0,"allocated":196378624,"data_stored":195373271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1385221123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1786,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1786,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.448332"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21255,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305872+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:05.305872+0000","last_peered":"2024-09-15T21:32:05.305872+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:05.305872+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:05.304615+0000","last_peered":"2024-09-15T21:32:05.304615+0000","last_clean":"2024-09-15T21:32:05.304615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:05.304615+0000","last_undegraded":"2024-09-15T21:32:05.304615+0000","last_fullsized":"2024-09-15T21:32:05.304615+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304310+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:05.304310+0000","last_peered":"2024-09-15T21:32:05.304310+0000","last_clean":"2024-09-15T21:32:05.304310+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:05.304310+0000","last_undegraded":"2024-09-15T21:32:05.304310+0000","last_fullsized":"2024-09-15T21:32:05.304310+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304982+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:05.304982+0000","last_peered":"2024-09-15T21:32:05.304982+0000","last_clean":"2024-09-15T21:32:05.304982+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:05.304982+0000","last_undegraded":"2024-09-15T21:32:05.304982+0000","last_fullsized":"2024-09-15T21:32:05.304982+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20799,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304140+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:05.304140+0000","last_peered":"2024-09-15T21:32:05.304140+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:05.304140+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":28189,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:24.572609+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:24.572609+0000","last_peered":"2024-09-15T21:32:24.572609+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:24.572609+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3677,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21024,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303944+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:05.303944+0000","last_peered":"2024-09-15T21:32:05.303944+0000","last_clean":"2024-09-15T21:32:05.303944+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:05.303944+0000","last_undegraded":"2024-09-15T21:32:05.303944+0000","last_fullsized":"2024-09-15T21:32:05.303944+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304006+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:05.304006+0000","last_peered":"2024-09-15T21:32:05.304006+0000","last_clean":"2024-09-15T21:32:05.304006+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:05.304006+0000","last_undegraded":"2024-09-15T21:32:05.304006+0000","last_fullsized":"2024-09-15T21:32:05.304006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20640,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:05.304397+0000","last_peered":"2024-09-15T21:32:05.304397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:05.304397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304050+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:05.304050+0000","last_peered":"2024-09-15T21:32:05.304050+0000","last_clean":"2024-09-15T21:32:05.304050+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:05.304050+0000","last_undegraded":"2024-09-15T21:32:05.304050+0000","last_fullsized":"2024-09-15T21:32:05.304050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28979,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.358402+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:05.358402+0000","last_peered":"2024-09-15T21:32:05.358402+0000","last_clean":"2024-09-15T21:32:05.358402+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:05.358402+0000","last_undegraded":"2024-09-15T21:32:05.358402+0000","last_fullsized":"2024-09-15T21:32:05.358402+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29046,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303826+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:05.303826+0000","last_peered":"2024-09-15T21:32:05.303826+0000","last_clean":"2024-09-15T21:32:05.303826+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:05.303826+0000","last_undegraded":"2024-09-15T21:32:05.303826+0000","last_fullsized":"2024-09-15T21:32:05.303826+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304457+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:05.304457+0000","last_peered":"2024-09-15T21:32:05.304457+0000","last_clean":"2024-09-15T21:32:05.304457+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:05.304457+0000","last_undegraded":"2024-09-15T21:32:05.304457+0000","last_fullsized":"2024-09-15T21:32:05.304457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305161+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:05.305161+0000","last_peered":"2024-09-15T21:32:05.305161+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:05.305161+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32610,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":194936832,"data_stored":194828442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485064,"kb_used_data":124344,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93886776,"statfs":{"total":96636764160,"available":96140058624,"internally_reserved":0,"allocated":127328256,"data_stored":126998265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535864,"kb_used_data":66916,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835976,"statfs":{"total":96636764160,"available":96088039424,"internally_reserved":0,"allocated":68521984,"data_stored":68187916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523668,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848172,"statfs":{"total":96636764160,"available":96100528128,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126853120,"data_stored":126816990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:28.277 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:28.651 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:28.652 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:28.666 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-15T21:32:28.358230+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32611,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544596,"kb_used_data":191776,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281570924,"statfs":{"total":289910292480,"available":288328626176,"internally_reserved":0,"allocated":196378624,"data_stored":195373271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1385221123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1786,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1786,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.458001"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21255,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305872+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:05.305872+0000","last_peered":"2024-09-15T21:32:05.305872+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:05.305872+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:05.304615+0000","last_peered":"2024-09-15T21:32:05.304615+0000","last_clean":"2024-09-15T21:32:05.304615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:05.304615+0000","last_undegraded":"2024-09-15T21:32:05.304615+0000","last_fullsized":"2024-09-15T21:32:05.304615+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304310+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:05.304310+0000","last_peered":"2024-09-15T21:32:05.304310+0000","last_clean":"2024-09-15T21:32:05.304310+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:05.304310+0000","last_undegraded":"2024-09-15T21:32:05.304310+0000","last_fullsized":"2024-09-15T21:32:05.304310+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304982+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:05.304982+0000","last_peered":"2024-09-15T21:32:05.304982+0000","last_clean":"2024-09-15T21:32:05.304982+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:05.304982+0000","last_undegraded":"2024-09-15T21:32:05.304982+0000","last_fullsized":"2024-09-15T21:32:05.304982+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20799,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304140+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:05.304140+0000","last_peered":"2024-09-15T21:32:05.304140+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:05.304140+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":28189,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:24.572609+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:24.572609+0000","last_peered":"2024-09-15T21:32:24.572609+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:24.572609+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3677,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21024,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303944+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:05.303944+0000","last_peered":"2024-09-15T21:32:05.303944+0000","last_clean":"2024-09-15T21:32:05.303944+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:05.303944+0000","last_undegraded":"2024-09-15T21:32:05.303944+0000","last_fullsized":"2024-09-15T21:32:05.303944+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304006+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:05.304006+0000","last_peered":"2024-09-15T21:32:05.304006+0000","last_clean":"2024-09-15T21:32:05.304006+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:05.304006+0000","last_undegraded":"2024-09-15T21:32:05.304006+0000","last_fullsized":"2024-09-15T21:32:05.304006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20640,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:05.304397+0000","last_peered":"2024-09-15T21:32:05.304397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:05.304397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304050+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:05.304050+0000","last_peered":"2024-09-15T21:32:05.304050+0000","last_clean":"2024-09-15T21:32:05.304050+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:05.304050+0000","last_undegraded":"2024-09-15T21:32:05.304050+0000","last_fullsized":"2024-09-15T21:32:05.304050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28979,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.358402+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:05.358402+0000","last_peered":"2024-09-15T21:32:05.358402+0000","last_clean":"2024-09-15T21:32:05.358402+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:05.358402+0000","last_undegraded":"2024-09-15T21:32:05.358402+0000","last_fullsized":"2024-09-15T21:32:05.358402+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29046,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303826+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:05.303826+0000","last_peered":"2024-09-15T21:32:05.303826+0000","last_clean":"2024-09-15T21:32:05.303826+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:05.303826+0000","last_undegraded":"2024-09-15T21:32:05.303826+0000","last_fullsized":"2024-09-15T21:32:05.303826+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304457+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:05.304457+0000","last_peered":"2024-09-15T21:32:05.304457+0000","last_clean":"2024-09-15T21:32:05.304457+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:05.304457+0000","last_undegraded":"2024-09-15T21:32:05.304457+0000","last_fullsized":"2024-09-15T21:32:05.304457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305161+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:05.305161+0000","last_peered":"2024-09-15T21:32:05.305161+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:05.305161+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32610,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":194936832,"data_stored":194828442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485064,"kb_used_data":124344,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93886776,"statfs":{"total":96636764160,"available":96140058624,"internally_reserved":0,"allocated":127328256,"data_stored":126998265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535864,"kb_used_data":66916,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835976,"statfs":{"total":96636764160,"available":96088039424,"internally_reserved":0,"allocated":68521984,"data_stored":68187916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523668,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848172,"statfs":{"total":96636764160,"available":96100528128,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126853120,"data_stored":126816990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:28.667 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:28.667 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21255, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.305872+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:32:05.305872+0000', 'last_peered': '2024-09-15T21:32:05.305872+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:05.305872+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:28.667 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:28.667 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20799, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.304140+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:05.304140+0000', 'last_peered': '2024-09-15T21:32:05.304140+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:05.304140+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:28.668 INFO:tasks.lost_unfound.ceph_manager:PG 2.9 is not active+clean 2024-09-15T21:32:28.668 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.9', 'version': "49'20670", 'reported_seq': 28189, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:24.572609+0000', 'last_change': '2024-09-15T21:32:03.897219+0000', 'last_active': '2024-09-15T21:32:24.572609+0000', 'last_peered': '2024-09-15T21:32:24.572609+0000', 'last_clean': '2024-09-15T21:21:12.331632+0000', 'last_became_active': '2024-09-15T21:21:14.876230+0000', 'last_became_peered': '2024-09-15T21:21:14.876230+0000', 'last_unstale': '2024-09-15T21:32:24.572609+0000', 'last_undegraded': '2024-09-15T21:21:14.437153+0000', 'last_fullsized': '2024-09-15T21:21:14.352604+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3735", 'last_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:05.072580+0000', 'objects_scrubbed': 3685, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1290, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 484, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16981, 'num_write_kb': 26596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3677, 'num_bytes_recovered': 966, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -482}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:28.668 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:32:28.668 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 20640, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.304397+0000', 'last_change': '2024-09-15T21:21:14.886332+0000', 'last_active': '2024-09-15T21:32:05.304397+0000', 'last_peered': '2024-09-15T21:32:05.304397+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:32:05.304397+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4020}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:28.668 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:28.668 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21032, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:05.305161+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:05.305161+0000', 'last_peered': '2024-09-15T21:32:05.305161+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:05.305161+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:28.668 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:32:29.147 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:29.147 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16864/24 objects degraded (70266.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":800,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1581666304,"bytes_avail":288328626176,"bytes_total":289910292480,"degraded_objects":16864,"degraded_total":24,"degraded_ratio":702.66666666666652,"recovering_objects_per_sec":148,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1786,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:32:29.162 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:32:29.162 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:29.532 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:29.532 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:29.545 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-09-15T21:32:28.901868+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32611,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1544596,"kb_used_data":191776,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281570924,"statfs":{"total":289910292480,"available":288328626176,"internally_reserved":0,"allocated":196378624,"data_stored":195373271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1385221123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1020,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001337"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21255,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305872+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:05.305872+0000","last_peered":"2024-09-15T21:32:05.305872+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:05.305872+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20807,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304615+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:05.304615+0000","last_peered":"2024-09-15T21:32:05.304615+0000","last_clean":"2024-09-15T21:32:05.304615+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:05.304615+0000","last_undegraded":"2024-09-15T21:32:05.304615+0000","last_fullsized":"2024-09-15T21:32:05.304615+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304310+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:05.304310+0000","last_peered":"2024-09-15T21:32:05.304310+0000","last_clean":"2024-09-15T21:32:05.304310+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:05.304310+0000","last_undegraded":"2024-09-15T21:32:05.304310+0000","last_fullsized":"2024-09-15T21:32:05.304310+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20665,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304982+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:05.304982+0000","last_peered":"2024-09-15T21:32:05.304982+0000","last_clean":"2024-09-15T21:32:05.304982+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:05.304982+0000","last_undegraded":"2024-09-15T21:32:05.304982+0000","last_fullsized":"2024-09-15T21:32:05.304982+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20799,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304140+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:05.304140+0000","last_peered":"2024-09-15T21:32:05.304140+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:05.304140+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":28189,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:24.572609+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:24.572609+0000","last_peered":"2024-09-15T21:32:24.572609+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:24.572609+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3677,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21024,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303944+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:05.303944+0000","last_peered":"2024-09-15T21:32:05.303944+0000","last_clean":"2024-09-15T21:32:05.303944+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:05.303944+0000","last_undegraded":"2024-09-15T21:32:05.303944+0000","last_fullsized":"2024-09-15T21:32:05.303944+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304006+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:05.304006+0000","last_peered":"2024-09-15T21:32:05.304006+0000","last_clean":"2024-09-15T21:32:05.304006+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:05.304006+0000","last_undegraded":"2024-09-15T21:32:05.304006+0000","last_fullsized":"2024-09-15T21:32:05.304006+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":20640,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.304397+0000","last_change":"2024-09-15T21:21:14.886332+0000","last_active":"2024-09-15T21:32:05.304397+0000","last_peered":"2024-09-15T21:32:05.304397+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:05.304397+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4020}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":128,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304050+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:05.304050+0000","last_peered":"2024-09-15T21:32:05.304050+0000","last_clean":"2024-09-15T21:32:05.304050+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:05.304050+0000","last_undegraded":"2024-09-15T21:32:05.304050+0000","last_fullsized":"2024-09-15T21:32:05.304050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28979,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.358402+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:05.358402+0000","last_peered":"2024-09-15T21:32:05.358402+0000","last_clean":"2024-09-15T21:32:05.358402+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:05.358402+0000","last_undegraded":"2024-09-15T21:32:05.358402+0000","last_fullsized":"2024-09-15T21:32:05.358402+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29046,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303826+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:05.303826+0000","last_peered":"2024-09-15T21:32:05.303826+0000","last_clean":"2024-09-15T21:32:05.303826+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:05.303826+0000","last_undegraded":"2024-09-15T21:32:05.303826+0000","last_fullsized":"2024-09-15T21:32:05.303826+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.304457+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:05.304457+0000","last_peered":"2024-09-15T21:32:05.304457+0000","last_clean":"2024-09-15T21:32:05.304457+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:05.304457+0000","last_undegraded":"2024-09-15T21:32:05.304457+0000","last_fullsized":"2024-09-15T21:32:05.304457+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21032,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:05.305161+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:05.305161+0000","last_peered":"2024-09-15T21:32:05.305161+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:05.305161+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32610,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":194936832,"data_stored":194828442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485064,"kb_used_data":124344,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93886776,"statfs":{"total":96636764160,"available":96140058624,"internally_reserved":0,"allocated":127328256,"data_stored":126998265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535864,"kb_used_data":66916,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835976,"statfs":{"total":96636764160,"available":96088039424,"internally_reserved":0,"allocated":68521984,"data_stored":68187916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523668,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848172,"statfs":{"total":96636764160,"available":96100528128,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126853120,"data_stored":126816990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:32.548 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:32.903 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:32.904 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:32.918 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-09-15T21:32:30.902166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16387,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33088,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1542604,"kb_used_data":189772,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281572916,"statfs":{"total":289910292480,"available":288330665984,"internally_reserved":0,"allocated":194326528,"data_stored":193301353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1385221123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1497,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1497,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001294"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21259,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.360788+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:28.360788+0000","last_peered":"2024-09-15T21:32:28.360788+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:28.360788+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20811,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357508+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:28.357508+0000","last_peered":"2024-09-15T21:32:28.357508+0000","last_clean":"2024-09-15T21:32:28.357508+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:28.357508+0000","last_undegraded":"2024-09-15T21:32:28.357508+0000","last_fullsized":"2024-09-15T21:32:28.357508+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302320+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:05.302320+0000","last_peered":"2024-09-15T21:32:05.302320+0000","last_clean":"2024-09-15T21:32:05.302320+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:05.302320+0000","last_undegraded":"2024-09-15T21:32:05.302320+0000","last_fullsized":"2024-09-15T21:32:05.302320+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357242+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:28.357242+0000","last_peered":"2024-09-15T21:32:28.357242+0000","last_clean":"2024-09-15T21:32:28.357242+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:28.357242+0000","last_undegraded":"2024-09-15T21:32:28.357242+0000","last_fullsized":"2024-09-15T21:32:28.357242+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20669,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359725+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:28.359725+0000","last_peered":"2024-09-15T21:32:28.359725+0000","last_clean":"2024-09-15T21:32:28.359725+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:28.359725+0000","last_undegraded":"2024-09-15T21:32:28.359725+0000","last_fullsized":"2024-09-15T21:32:28.359725+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20803,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359810+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:28.359810+0000","last_peered":"2024-09-15T21:32:28.359810+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:28.359810+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":28189,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:24.572609+0000","last_change":"2024-09-15T21:32:03.897219+0000","last_active":"2024-09-15T21:32:24.572609+0000","last_peered":"2024-09-15T21:32:24.572609+0000","last_clean":"2024-09-15T21:21:12.331632+0000","last_became_active":"2024-09-15T21:21:14.876230+0000","last_became_peered":"2024-09-15T21:21:14.876230+0000","last_unstale":"2024-09-15T21:32:24.572609+0000","last_undegraded":"2024-09-15T21:21:14.437153+0000","last_fullsized":"2024-09-15T21:21:14.352604+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3677,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21028,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359338+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:28.359338+0000","last_peered":"2024-09-15T21:32:28.359338+0000","last_clean":"2024-09-15T21:32:28.359338+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:28.359338+0000","last_undegraded":"2024-09-15T21:32:28.359338+0000","last_fullsized":"2024-09-15T21:32:28.359338+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302012+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:05.302012+0000","last_peered":"2024-09-15T21:32:05.302012+0000","last_clean":"2024-09-15T21:32:05.302012+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:05.302012+0000","last_undegraded":"2024-09-15T21:32:05.302012+0000","last_fullsized":"2024-09-15T21:32:05.302012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20671,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302148+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:05.302148+0000","last_peered":"2024-09-15T21:32:05.302148+0000","last_clean":"2024-09-15T21:32:05.302148+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:05.302148+0000","last_undegraded":"2024-09-15T21:32:05.302148+0000","last_fullsized":"2024-09-15T21:32:05.302148+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356806+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:28.356806+0000","last_peered":"2024-09-15T21:32:28.356806+0000","last_clean":"2024-09-15T21:32:28.356806+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:28.356806+0000","last_undegraded":"2024-09-15T21:32:28.356806+0000","last_fullsized":"2024-09-15T21:32:28.356806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":21599,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:29.575650+0000","last_change":"2024-09-15T21:32:26.401928+0000","last_active":"2024-09-15T21:32:29.575650+0000","last_peered":"2024-09-15T21:32:29.575650+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:29.575650+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3545,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":477,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3545}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359430+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:28.359430+0000","last_peered":"2024-09-15T21:32:28.359430+0000","last_clean":"2024-09-15T21:32:28.359430+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:28.359430+0000","last_undegraded":"2024-09-15T21:32:28.359430+0000","last_fullsized":"2024-09-15T21:32:28.359430+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28983,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356845+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:28.356845+0000","last_peered":"2024-09-15T21:32:28.356845+0000","last_clean":"2024-09-15T21:32:28.356845+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:28.356845+0000","last_undegraded":"2024-09-15T21:32:28.356845+0000","last_fullsized":"2024-09-15T21:32:28.356845+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302067+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:05.302067+0000","last_peered":"2024-09-15T21:32:05.302067+0000","last_clean":"2024-09-15T21:32:05.302067+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:05.302067+0000","last_undegraded":"2024-09-15T21:32:05.302067+0000","last_fullsized":"2024-09-15T21:32:05.302067+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29050,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356478+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:28.356478+0000","last_peered":"2024-09-15T21:32:28.356478+0000","last_clean":"2024-09-15T21:32:28.356478+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:28.356478+0000","last_undegraded":"2024-09-15T21:32:28.356478+0000","last_fullsized":"2024-09-15T21:32:28.356478+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20844,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.303224+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:05.303224+0000","last_peered":"2024-09-15T21:32:05.303224+0000","last_clean":"2024-09-15T21:32:05.303224+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:05.303224+0000","last_undegraded":"2024-09-15T21:32:05.303224+0000","last_fullsized":"2024-09-15T21:32:05.303224+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302230+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:05.302230+0000","last_peered":"2024-09-15T21:32:05.302230+0000","last_clean":"2024-09-15T21:32:05.302230+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:05.302230+0000","last_undegraded":"2024-09-15T21:32:05.302230+0000","last_fullsized":"2024-09-15T21:32:05.302230+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21609,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302588+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:05.302588+0000","last_peered":"2024-09-15T21:32:05.302588+0000","last_clean":"2024-09-15T21:32:05.302588+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:05.302588+0000","last_undegraded":"2024-09-15T21:32:05.302588+0000","last_fullsized":"2024-09-15T21:32:05.302588+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357418+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:28.357418+0000","last_peered":"2024-09-15T21:32:28.357418+0000","last_clean":"2024-09-15T21:32:28.357418+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:28.357418+0000","last_undegraded":"2024-09-15T21:32:28.357418+0000","last_fullsized":"2024-09-15T21:32:28.357418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28725,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302406+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:05.302406+0000","last_peered":"2024-09-15T21:32:05.302406+0000","last_clean":"2024-09-15T21:32:05.302406+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:05.302406+0000","last_undegraded":"2024-09-15T21:32:05.302406+0000","last_fullsized":"2024-09-15T21:32:05.302406+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302513+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:05.302513+0000","last_peered":"2024-09-15T21:32:05.302513+0000","last_clean":"2024-09-15T21:32:05.302513+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:05.302513+0000","last_undegraded":"2024-09-15T21:32:05.302513+0000","last_fullsized":"2024-09-15T21:32:05.302513+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-15T21:32:05.302079+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:05.302079+0000","last_peered":"2024-09-15T21:32:05.302079+0000","last_clean":"2024-09-15T21:32:05.302079+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:05.302079+0000","last_undegraded":"2024-09-15T21:32:05.302079+0000","last_fullsized":"2024-09-15T21:32:05.302079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359632+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:28.359632+0000","last_peered":"2024-09-15T21:32:28.359632+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:28.359632+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16387,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33087,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192851968,"data_stored":192743578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483052,"kb_used_data":122324,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93888788,"statfs":{"total":96636764160,"available":96142118912,"internally_reserved":0,"allocated":125259776,"data_stored":124919874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523668,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848172,"statfs":{"total":96636764160,"available":96100528128,"internally_reserved":0,"allocated":528384,"data_stored":187090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":124768256,"data_stored":124732126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:32.919 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:33.253 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:33.253 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:33.268 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-15T21:32:32.902534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15903,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33572,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534296,"kb_used_data":189780,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281581224,"statfs":{"total":289910292480,"available":288339173376,"internally_reserved":0,"allocated":194334720,"data_stored":193306860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1376701443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":961,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001346"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21259,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.360788+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:28.360788+0000","last_peered":"2024-09-15T21:32:28.360788+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:28.360788+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20811,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357508+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:28.357508+0000","last_peered":"2024-09-15T21:32:28.357508+0000","last_clean":"2024-09-15T21:32:28.357508+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:28.357508+0000","last_undegraded":"2024-09-15T21:32:28.357508+0000","last_fullsized":"2024-09-15T21:32:28.357508+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357242+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:28.357242+0000","last_peered":"2024-09-15T21:32:28.357242+0000","last_clean":"2024-09-15T21:32:28.357242+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:28.357242+0000","last_undegraded":"2024-09-15T21:32:28.357242+0000","last_fullsized":"2024-09-15T21:32:28.357242+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20669,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359725+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:28.359725+0000","last_peered":"2024-09-15T21:32:28.359725+0000","last_clean":"2024-09-15T21:32:28.359725+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:28.359725+0000","last_undegraded":"2024-09-15T21:32:28.359725+0000","last_fullsized":"2024-09-15T21:32:28.359725+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20803,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359810+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:28.359810+0000","last_peered":"2024-09-15T21:32:28.359810+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:28.359810+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21028,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359338+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:28.359338+0000","last_peered":"2024-09-15T21:32:28.359338+0000","last_clean":"2024-09-15T21:32:28.359338+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:28.359338+0000","last_undegraded":"2024-09-15T21:32:28.359338+0000","last_fullsized":"2024-09-15T21:32:28.359338+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356806+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:28.356806+0000","last_peered":"2024-09-15T21:32:28.356806+0000","last_clean":"2024-09-15T21:32:28.356806+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:28.356806+0000","last_undegraded":"2024-09-15T21:32:28.356806+0000","last_fullsized":"2024-09-15T21:32:28.356806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":21599,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:29.575650+0000","last_change":"2024-09-15T21:32:26.401928+0000","last_active":"2024-09-15T21:32:29.575650+0000","last_peered":"2024-09-15T21:32:29.575650+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:29.575650+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3545,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":477,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3545}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359430+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:28.359430+0000","last_peered":"2024-09-15T21:32:28.359430+0000","last_clean":"2024-09-15T21:32:28.359430+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:28.359430+0000","last_undegraded":"2024-09-15T21:32:28.359430+0000","last_fullsized":"2024-09-15T21:32:28.359430+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28983,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356845+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:28.356845+0000","last_peered":"2024-09-15T21:32:28.356845+0000","last_clean":"2024-09-15T21:32:28.356845+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:28.356845+0000","last_undegraded":"2024-09-15T21:32:28.356845+0000","last_fullsized":"2024-09-15T21:32:28.356845+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29050,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356478+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:28.356478+0000","last_peered":"2024-09-15T21:32:28.356478+0000","last_clean":"2024-09-15T21:32:28.356478+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:28.356478+0000","last_undegraded":"2024-09-15T21:32:28.356478+0000","last_fullsized":"2024-09-15T21:32:28.356478+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357418+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:28.357418+0000","last_peered":"2024-09-15T21:32:28.357418+0000","last_clean":"2024-09-15T21:32:28.357418+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:28.357418+0000","last_undegraded":"2024-09-15T21:32:28.357418+0000","last_fullsized":"2024-09-15T21:32:28.357418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359632+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:28.359632+0000","last_peered":"2024-09-15T21:32:28.359632+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:28.359632+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15903,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33571,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192851968,"data_stored":192743578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483052,"kb_used_data":122324,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93888788,"statfs":{"total":96636764160,"available":96142118912,"internally_reserved":0,"allocated":125259776,"data_stored":124919874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515360,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856480,"statfs":{"total":96636764160,"available":96109035520,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":124768256,"data_stored":124732126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:33.270 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:33.270 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21259, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:28.360788+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:32:28.360788+0000', 'last_peered': '2024-09-15T21:32:28.360788+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:28.360788+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:33.270 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:33.270 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20803, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:28.359810+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:28.359810+0000', 'last_peered': '2024-09-15T21:32:28.359810+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:28.359810+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:33.270 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:32:33.270 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 21599, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:29.575650+0000', 'last_change': '2024-09-15T21:32:26.401928+0000', 'last_active': '2024-09-15T21:32:29.575650+0000', 'last_peered': '2024-09-15T21:32:29.575650+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:32:29.575650+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3545, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 477, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3545}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:33.270 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:33.270 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21036, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:28.359632+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:28.359632+0000', 'last_peered': '2024-09-15T21:32:28.359632+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:28.359632+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:33.271 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:32:33.713 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:33.713 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15903/24 objects degraded (66262.500%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":805,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1571119104,"bytes_avail":288339173376,"bytes_total":289910292480,"degraded_objects":15903,"degraded_total":24,"degraded_ratio":662.625,"recovering_objects_per_sec":120,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":961,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:32:33.727 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:32:33.727 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:34.055 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:34.055 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:34.068 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-15T21:32:32.902534+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15903,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33572,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534296,"kb_used_data":189780,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281581224,"statfs":{"total":289910292480,"available":288339173376,"internally_reserved":0,"allocated":194334720,"data_stored":193306860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1376701443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-961,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":961,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001346"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21259,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.360788+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:28.360788+0000","last_peered":"2024-09-15T21:32:28.360788+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:28.360788+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20811,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357508+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:28.357508+0000","last_peered":"2024-09-15T21:32:28.357508+0000","last_clean":"2024-09-15T21:32:28.357508+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:28.357508+0000","last_undegraded":"2024-09-15T21:32:28.357508+0000","last_fullsized":"2024-09-15T21:32:28.357508+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357242+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:28.357242+0000","last_peered":"2024-09-15T21:32:28.357242+0000","last_clean":"2024-09-15T21:32:28.357242+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:28.357242+0000","last_undegraded":"2024-09-15T21:32:28.357242+0000","last_fullsized":"2024-09-15T21:32:28.357242+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20669,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359725+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:28.359725+0000","last_peered":"2024-09-15T21:32:28.359725+0000","last_clean":"2024-09-15T21:32:28.359725+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:28.359725+0000","last_undegraded":"2024-09-15T21:32:28.359725+0000","last_fullsized":"2024-09-15T21:32:28.359725+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20803,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359810+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:28.359810+0000","last_peered":"2024-09-15T21:32:28.359810+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:28.359810+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21028,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359338+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:28.359338+0000","last_peered":"2024-09-15T21:32:28.359338+0000","last_clean":"2024-09-15T21:32:28.359338+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:28.359338+0000","last_undegraded":"2024-09-15T21:32:28.359338+0000","last_fullsized":"2024-09-15T21:32:28.359338+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356806+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:28.356806+0000","last_peered":"2024-09-15T21:32:28.356806+0000","last_clean":"2024-09-15T21:32:28.356806+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:28.356806+0000","last_undegraded":"2024-09-15T21:32:28.356806+0000","last_fullsized":"2024-09-15T21:32:28.356806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":21599,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:29.575650+0000","last_change":"2024-09-15T21:32:26.401928+0000","last_active":"2024-09-15T21:32:29.575650+0000","last_peered":"2024-09-15T21:32:29.575650+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:29.575650+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3545,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":477,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3545}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":132,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359430+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:28.359430+0000","last_peered":"2024-09-15T21:32:28.359430+0000","last_clean":"2024-09-15T21:32:28.359430+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:28.359430+0000","last_undegraded":"2024-09-15T21:32:28.359430+0000","last_fullsized":"2024-09-15T21:32:28.359430+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28983,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356845+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:28.356845+0000","last_peered":"2024-09-15T21:32:28.356845+0000","last_clean":"2024-09-15T21:32:28.356845+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:28.356845+0000","last_undegraded":"2024-09-15T21:32:28.356845+0000","last_fullsized":"2024-09-15T21:32:28.356845+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29050,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356478+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:28.356478+0000","last_peered":"2024-09-15T21:32:28.356478+0000","last_clean":"2024-09-15T21:32:28.356478+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:28.356478+0000","last_undegraded":"2024-09-15T21:32:28.356478+0000","last_fullsized":"2024-09-15T21:32:28.356478+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357418+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:28.357418+0000","last_peered":"2024-09-15T21:32:28.357418+0000","last_clean":"2024-09-15T21:32:28.357418+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:28.357418+0000","last_undegraded":"2024-09-15T21:32:28.357418+0000","last_fullsized":"2024-09-15T21:32:28.357418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359632+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:28.359632+0000","last_peered":"2024-09-15T21:32:28.359632+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:28.359632+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15903,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33571,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192851968,"data_stored":192743578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483052,"kb_used_data":122324,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93888788,"statfs":{"total":96636764160,"available":96142118912,"internally_reserved":0,"allocated":125259776,"data_stored":124919874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515360,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856480,"statfs":{"total":96636764160,"available":96109035520,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":124768256,"data_stored":124732126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:37.071 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:37.398 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:37.413 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-15T21:32:36.903166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34521,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540204,"kb_used_data":187380,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281575316,"statfs":{"total":289910292480,"available":288333123584,"internally_reserved":0,"allocated":191877120,"data_stored":190849260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1385221123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1910,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.554924"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21259,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.360788+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:28.360788+0000","last_peered":"2024-09-15T21:32:28.360788+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:28.360788+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20811,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357508+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:28.357508+0000","last_peered":"2024-09-15T21:32:28.357508+0000","last_clean":"2024-09-15T21:32:28.357508+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:28.357508+0000","last_undegraded":"2024-09-15T21:32:28.357508+0000","last_fullsized":"2024-09-15T21:32:28.357508+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357242+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:28.357242+0000","last_peered":"2024-09-15T21:32:28.357242+0000","last_clean":"2024-09-15T21:32:28.357242+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:28.357242+0000","last_undegraded":"2024-09-15T21:32:28.357242+0000","last_fullsized":"2024-09-15T21:32:28.357242+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20669,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359725+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:28.359725+0000","last_peered":"2024-09-15T21:32:28.359725+0000","last_clean":"2024-09-15T21:32:28.359725+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:28.359725+0000","last_undegraded":"2024-09-15T21:32:28.359725+0000","last_fullsized":"2024-09-15T21:32:28.359725+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20803,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359810+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:28.359810+0000","last_peered":"2024-09-15T21:32:28.359810+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:28.359810+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21028,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359338+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:28.359338+0000","last_peered":"2024-09-15T21:32:28.359338+0000","last_clean":"2024-09-15T21:32:28.359338+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:28.359338+0000","last_undegraded":"2024-09-15T21:32:28.359338+0000","last_fullsized":"2024-09-15T21:32:28.359338+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356806+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:28.356806+0000","last_peered":"2024-09-15T21:32:28.356806+0000","last_clean":"2024-09-15T21:32:28.356806+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:28.356806+0000","last_undegraded":"2024-09-15T21:32:28.356806+0000","last_fullsized":"2024-09-15T21:32:28.356806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":23498,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:34.576437+0000","last_change":"2024-09-15T21:32:26.401928+0000","last_active":"2024-09-15T21:32:34.576437+0000","last_peered":"2024-09-15T21:32:34.576437+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:34.576437+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1426,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2596}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359473+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:28.359473+0000","last_peered":"2024-09-15T21:32:28.359473+0000","last_clean":"2024-09-15T21:32:28.359473+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:28.359473+0000","last_undegraded":"2024-09-15T21:32:28.359473+0000","last_fullsized":"2024-09-15T21:32:28.359473+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28983,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356845+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:28.356845+0000","last_peered":"2024-09-15T21:32:28.356845+0000","last_clean":"2024-09-15T21:32:28.356845+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:28.356845+0000","last_undegraded":"2024-09-15T21:32:28.356845+0000","last_fullsized":"2024-09-15T21:32:28.356845+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29050,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356478+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:28.356478+0000","last_peered":"2024-09-15T21:32:28.356478+0000","last_clean":"2024-09-15T21:32:28.356478+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:28.356478+0000","last_undegraded":"2024-09-15T21:32:28.356478+0000","last_fullsized":"2024-09-15T21:32:28.356478+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357418+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:28.357418+0000","last_peered":"2024-09-15T21:32:28.357418+0000","last_clean":"2024-09-15T21:32:28.357418+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:28.357418+0000","last_undegraded":"2024-09-15T21:32:28.357418+0000","last_fullsized":"2024-09-15T21:32:28.357418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359632+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:28.359632+0000","last_peered":"2024-09-15T21:32:28.359632+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:28.359632+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34520,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190394368,"data_stored":190285978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480644,"kb_used_data":119924,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93891196,"statfs":{"total":96636764160,"available":96144584704,"internally_reserved":0,"allocated":122802176,"data_stored":122462274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523676,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848164,"statfs":{"total":96636764160,"available":96100519936,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122310656,"data_stored":122274526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:37.414 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:37.730 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:37.731 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:37.743 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-15T21:32:36.903166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34521,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540204,"kb_used_data":187380,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281575316,"statfs":{"total":289910292480,"available":288333123584,"internally_reserved":0,"allocated":191877120,"data_stored":190849260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1385221123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1910,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.554924"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21259,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.360788+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:28.360788+0000","last_peered":"2024-09-15T21:32:28.360788+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:28.360788+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20811,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357508+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:28.357508+0000","last_peered":"2024-09-15T21:32:28.357508+0000","last_clean":"2024-09-15T21:32:28.357508+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:28.357508+0000","last_undegraded":"2024-09-15T21:32:28.357508+0000","last_fullsized":"2024-09-15T21:32:28.357508+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357242+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:28.357242+0000","last_peered":"2024-09-15T21:32:28.357242+0000","last_clean":"2024-09-15T21:32:28.357242+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:28.357242+0000","last_undegraded":"2024-09-15T21:32:28.357242+0000","last_fullsized":"2024-09-15T21:32:28.357242+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20669,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359725+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:28.359725+0000","last_peered":"2024-09-15T21:32:28.359725+0000","last_clean":"2024-09-15T21:32:28.359725+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:28.359725+0000","last_undegraded":"2024-09-15T21:32:28.359725+0000","last_fullsized":"2024-09-15T21:32:28.359725+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20803,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359810+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:28.359810+0000","last_peered":"2024-09-15T21:32:28.359810+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:28.359810+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21028,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359338+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:28.359338+0000","last_peered":"2024-09-15T21:32:28.359338+0000","last_clean":"2024-09-15T21:32:28.359338+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:28.359338+0000","last_undegraded":"2024-09-15T21:32:28.359338+0000","last_fullsized":"2024-09-15T21:32:28.359338+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356806+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:28.356806+0000","last_peered":"2024-09-15T21:32:28.356806+0000","last_clean":"2024-09-15T21:32:28.356806+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:28.356806+0000","last_undegraded":"2024-09-15T21:32:28.356806+0000","last_fullsized":"2024-09-15T21:32:28.356806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":23498,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:34.576437+0000","last_change":"2024-09-15T21:32:26.401928+0000","last_active":"2024-09-15T21:32:34.576437+0000","last_peered":"2024-09-15T21:32:34.576437+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:34.576437+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1426,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2596}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359473+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:28.359473+0000","last_peered":"2024-09-15T21:32:28.359473+0000","last_clean":"2024-09-15T21:32:28.359473+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:28.359473+0000","last_undegraded":"2024-09-15T21:32:28.359473+0000","last_fullsized":"2024-09-15T21:32:28.359473+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28983,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356845+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:28.356845+0000","last_peered":"2024-09-15T21:32:28.356845+0000","last_clean":"2024-09-15T21:32:28.356845+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:28.356845+0000","last_undegraded":"2024-09-15T21:32:28.356845+0000","last_fullsized":"2024-09-15T21:32:28.356845+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29050,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356478+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:28.356478+0000","last_peered":"2024-09-15T21:32:28.356478+0000","last_clean":"2024-09-15T21:32:28.356478+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:28.356478+0000","last_undegraded":"2024-09-15T21:32:28.356478+0000","last_fullsized":"2024-09-15T21:32:28.356478+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357418+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:28.357418+0000","last_peered":"2024-09-15T21:32:28.357418+0000","last_clean":"2024-09-15T21:32:28.357418+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:28.357418+0000","last_undegraded":"2024-09-15T21:32:28.357418+0000","last_fullsized":"2024-09-15T21:32:28.357418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359632+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:28.359632+0000","last_peered":"2024-09-15T21:32:28.359632+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:28.359632+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34520,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190394368,"data_stored":190285978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480644,"kb_used_data":119924,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93891196,"statfs":{"total":96636764160,"available":96144584704,"internally_reserved":0,"allocated":122802176,"data_stored":122462274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523676,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848164,"statfs":{"total":96636764160,"available":96100519936,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122310656,"data_stored":122274526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:37.744 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:37.745 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21259, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:28.360788+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:32:28.360788+0000', 'last_peered': '2024-09-15T21:32:28.360788+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:28.360788+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:37.745 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:37.745 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20803, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:28.359810+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:28.359810+0000', 'last_peered': '2024-09-15T21:32:28.359810+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:28.359810+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:37.745 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:32:37.745 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 23498, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:34.576437+0000', 'last_change': '2024-09-15T21:32:26.401928+0000', 'last_active': '2024-09-15T21:32:34.576437+0000', 'last_peered': '2024-09-15T21:32:34.576437+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:32:34.576437+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2596, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1426, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2596}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:37.745 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:37.745 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21036, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:28.359632+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:28.359632+0000', 'last_peered': '2024-09-15T21:32:28.359632+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:28.359632+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:37.745 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:32:38.151 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:38.151 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14954/24 objects degraded (62308.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":809,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1577168896,"bytes_avail":288333123584,"bytes_total":289910292480,"degraded_objects":14954,"degraded_total":24,"degraded_ratio":623.08333333333337,"recovering_objects_per_sec":199,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1910,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:32:38.164 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:32:38.164 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:38.465 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:38.466 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:38.478 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-15T21:32:36.903166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34521,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540204,"kb_used_data":187380,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281575316,"statfs":{"total":289910292480,"available":288333123584,"internally_reserved":0,"allocated":191877120,"data_stored":190849260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1385221123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1910,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1910,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.554924"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21259,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.360788+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:28.360788+0000","last_peered":"2024-09-15T21:32:28.360788+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:28.360788+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20811,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357508+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:28.357508+0000","last_peered":"2024-09-15T21:32:28.357508+0000","last_clean":"2024-09-15T21:32:28.357508+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:28.357508+0000","last_undegraded":"2024-09-15T21:32:28.357508+0000","last_fullsized":"2024-09-15T21:32:28.357508+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357242+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:28.357242+0000","last_peered":"2024-09-15T21:32:28.357242+0000","last_clean":"2024-09-15T21:32:28.357242+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:28.357242+0000","last_undegraded":"2024-09-15T21:32:28.357242+0000","last_fullsized":"2024-09-15T21:32:28.357242+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20669,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359725+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:28.359725+0000","last_peered":"2024-09-15T21:32:28.359725+0000","last_clean":"2024-09-15T21:32:28.359725+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:28.359725+0000","last_undegraded":"2024-09-15T21:32:28.359725+0000","last_fullsized":"2024-09-15T21:32:28.359725+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20803,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359810+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:28.359810+0000","last_peered":"2024-09-15T21:32:28.359810+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:28.359810+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21028,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359338+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:28.359338+0000","last_peered":"2024-09-15T21:32:28.359338+0000","last_clean":"2024-09-15T21:32:28.359338+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:28.359338+0000","last_undegraded":"2024-09-15T21:32:28.359338+0000","last_fullsized":"2024-09-15T21:32:28.359338+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356806+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:28.356806+0000","last_peered":"2024-09-15T21:32:28.356806+0000","last_clean":"2024-09-15T21:32:28.356806+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:28.356806+0000","last_undegraded":"2024-09-15T21:32:28.356806+0000","last_fullsized":"2024-09-15T21:32:28.356806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":23498,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:34.576437+0000","last_change":"2024-09-15T21:32:26.401928+0000","last_active":"2024-09-15T21:32:34.576437+0000","last_peered":"2024-09-15T21:32:34.576437+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:34.576437+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1426,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2596}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359473+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:28.359473+0000","last_peered":"2024-09-15T21:32:28.359473+0000","last_clean":"2024-09-15T21:32:28.359473+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:28.359473+0000","last_undegraded":"2024-09-15T21:32:28.359473+0000","last_fullsized":"2024-09-15T21:32:28.359473+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28983,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356845+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:28.356845+0000","last_peered":"2024-09-15T21:32:28.356845+0000","last_clean":"2024-09-15T21:32:28.356845+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:28.356845+0000","last_undegraded":"2024-09-15T21:32:28.356845+0000","last_fullsized":"2024-09-15T21:32:28.356845+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29050,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356478+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:28.356478+0000","last_peered":"2024-09-15T21:32:28.356478+0000","last_clean":"2024-09-15T21:32:28.356478+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:28.356478+0000","last_undegraded":"2024-09-15T21:32:28.356478+0000","last_fullsized":"2024-09-15T21:32:28.356478+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357418+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:28.357418+0000","last_peered":"2024-09-15T21:32:28.357418+0000","last_clean":"2024-09-15T21:32:28.357418+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:28.357418+0000","last_undegraded":"2024-09-15T21:32:28.357418+0000","last_fullsized":"2024-09-15T21:32:28.357418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359632+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:28.359632+0000","last_peered":"2024-09-15T21:32:28.359632+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:28.359632+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14954,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34520,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190394368,"data_stored":190285978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480644,"kb_used_data":119924,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93891196,"statfs":{"total":96636764160,"available":96144584704,"internally_reserved":0,"allocated":122802176,"data_stored":122462274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523676,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848164,"statfs":{"total":96636764160,"available":96100519936,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122310656,"data_stored":122274526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:41.480 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:41.777 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:41.778 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:41.790 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-15T21:32:40.903818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35226,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537804,"kb_used_data":184980,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281577716,"statfs":{"total":289910292480,"available":288335581184,"internally_reserved":0,"allocated":189419520,"data_stored":188391660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1385221123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2615,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001949"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21259,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.360788+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:28.360788+0000","last_peered":"2024-09-15T21:32:28.360788+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:28.360788+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20811,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357508+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:28.357508+0000","last_peered":"2024-09-15T21:32:28.357508+0000","last_clean":"2024-09-15T21:32:28.357508+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:28.357508+0000","last_undegraded":"2024-09-15T21:32:28.357508+0000","last_fullsized":"2024-09-15T21:32:28.357508+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357242+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:28.357242+0000","last_peered":"2024-09-15T21:32:28.357242+0000","last_clean":"2024-09-15T21:32:28.357242+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:28.357242+0000","last_undegraded":"2024-09-15T21:32:28.357242+0000","last_fullsized":"2024-09-15T21:32:28.357242+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20669,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359725+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:28.359725+0000","last_peered":"2024-09-15T21:32:28.359725+0000","last_clean":"2024-09-15T21:32:28.359725+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:28.359725+0000","last_undegraded":"2024-09-15T21:32:28.359725+0000","last_fullsized":"2024-09-15T21:32:28.359725+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20803,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359810+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:28.359810+0000","last_peered":"2024-09-15T21:32:28.359810+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:28.359810+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21028,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359338+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:28.359338+0000","last_peered":"2024-09-15T21:32:28.359338+0000","last_clean":"2024-09-15T21:32:28.359338+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:28.359338+0000","last_undegraded":"2024-09-15T21:32:28.359338+0000","last_fullsized":"2024-09-15T21:32:28.359338+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356806+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:28.356806+0000","last_peered":"2024-09-15T21:32:28.356806+0000","last_clean":"2024-09-15T21:32:28.356806+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:28.356806+0000","last_undegraded":"2024-09-15T21:32:28.356806+0000","last_fullsized":"2024-09-15T21:32:28.356806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":24908,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:39.579252+0000","last_change":"2024-09-15T21:32:26.401928+0000","last_active":"2024-09-15T21:32:39.579252+0000","last_peered":"2024-09-15T21:32:39.579252+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:39.579252+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1891,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1891}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359473+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:28.359473+0000","last_peered":"2024-09-15T21:32:28.359473+0000","last_clean":"2024-09-15T21:32:28.359473+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:28.359473+0000","last_undegraded":"2024-09-15T21:32:28.359473+0000","last_fullsized":"2024-09-15T21:32:28.359473+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28983,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356845+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:28.356845+0000","last_peered":"2024-09-15T21:32:28.356845+0000","last_clean":"2024-09-15T21:32:28.356845+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:28.356845+0000","last_undegraded":"2024-09-15T21:32:28.356845+0000","last_fullsized":"2024-09-15T21:32:28.356845+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29050,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356478+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:28.356478+0000","last_peered":"2024-09-15T21:32:28.356478+0000","last_clean":"2024-09-15T21:32:28.356478+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:28.356478+0000","last_undegraded":"2024-09-15T21:32:28.356478+0000","last_fullsized":"2024-09-15T21:32:28.356478+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357418+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:28.357418+0000","last_peered":"2024-09-15T21:32:28.357418+0000","last_clean":"2024-09-15T21:32:28.357418+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:28.357418+0000","last_undegraded":"2024-09-15T21:32:28.357418+0000","last_fullsized":"2024-09-15T21:32:28.357418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359632+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:28.359632+0000","last_peered":"2024-09-15T21:32:28.359632+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:28.359632+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35225,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":187936768,"data_stored":187828378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478244,"kb_used_data":117524,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93893596,"statfs":{"total":96636764160,"available":96147042304,"internally_reserved":0,"allocated":120344576,"data_stored":120004674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523676,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848164,"statfs":{"total":96636764160,"available":96100519936,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119853056,"data_stored":119816926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:41.792 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:42.094 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:42.094 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:42.107 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-15T21:32:40.903818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35226,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537804,"kb_used_data":184980,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281577716,"statfs":{"total":289910292480,"available":288335581184,"internally_reserved":0,"allocated":189419520,"data_stored":188391660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1385221123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2615,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001949"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21259,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.360788+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:28.360788+0000","last_peered":"2024-09-15T21:32:28.360788+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:28.360788+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20811,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357508+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:28.357508+0000","last_peered":"2024-09-15T21:32:28.357508+0000","last_clean":"2024-09-15T21:32:28.357508+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:28.357508+0000","last_undegraded":"2024-09-15T21:32:28.357508+0000","last_fullsized":"2024-09-15T21:32:28.357508+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357242+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:28.357242+0000","last_peered":"2024-09-15T21:32:28.357242+0000","last_clean":"2024-09-15T21:32:28.357242+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:28.357242+0000","last_undegraded":"2024-09-15T21:32:28.357242+0000","last_fullsized":"2024-09-15T21:32:28.357242+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20669,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359725+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:28.359725+0000","last_peered":"2024-09-15T21:32:28.359725+0000","last_clean":"2024-09-15T21:32:28.359725+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:28.359725+0000","last_undegraded":"2024-09-15T21:32:28.359725+0000","last_fullsized":"2024-09-15T21:32:28.359725+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20803,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359810+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:28.359810+0000","last_peered":"2024-09-15T21:32:28.359810+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:28.359810+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21028,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359338+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:28.359338+0000","last_peered":"2024-09-15T21:32:28.359338+0000","last_clean":"2024-09-15T21:32:28.359338+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:28.359338+0000","last_undegraded":"2024-09-15T21:32:28.359338+0000","last_fullsized":"2024-09-15T21:32:28.359338+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356806+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:28.356806+0000","last_peered":"2024-09-15T21:32:28.356806+0000","last_clean":"2024-09-15T21:32:28.356806+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:28.356806+0000","last_undegraded":"2024-09-15T21:32:28.356806+0000","last_fullsized":"2024-09-15T21:32:28.356806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":24908,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:39.579252+0000","last_change":"2024-09-15T21:32:26.401928+0000","last_active":"2024-09-15T21:32:39.579252+0000","last_peered":"2024-09-15T21:32:39.579252+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:39.579252+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1891,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1891}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359473+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:28.359473+0000","last_peered":"2024-09-15T21:32:28.359473+0000","last_clean":"2024-09-15T21:32:28.359473+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:28.359473+0000","last_undegraded":"2024-09-15T21:32:28.359473+0000","last_fullsized":"2024-09-15T21:32:28.359473+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28983,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356845+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:28.356845+0000","last_peered":"2024-09-15T21:32:28.356845+0000","last_clean":"2024-09-15T21:32:28.356845+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:28.356845+0000","last_undegraded":"2024-09-15T21:32:28.356845+0000","last_fullsized":"2024-09-15T21:32:28.356845+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29050,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356478+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:28.356478+0000","last_peered":"2024-09-15T21:32:28.356478+0000","last_clean":"2024-09-15T21:32:28.356478+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:28.356478+0000","last_undegraded":"2024-09-15T21:32:28.356478+0000","last_fullsized":"2024-09-15T21:32:28.356478+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357418+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:28.357418+0000","last_peered":"2024-09-15T21:32:28.357418+0000","last_clean":"2024-09-15T21:32:28.357418+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:28.357418+0000","last_undegraded":"2024-09-15T21:32:28.357418+0000","last_fullsized":"2024-09-15T21:32:28.357418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359632+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:28.359632+0000","last_peered":"2024-09-15T21:32:28.359632+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:28.359632+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35225,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":187936768,"data_stored":187828378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478244,"kb_used_data":117524,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93893596,"statfs":{"total":96636764160,"available":96147042304,"internally_reserved":0,"allocated":120344576,"data_stored":120004674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523676,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848164,"statfs":{"total":96636764160,"available":96100519936,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119853056,"data_stored":119816926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:42.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:42.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21259, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:28.360788+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:32:28.360788+0000', 'last_peered': '2024-09-15T21:32:28.360788+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:28.360788+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:42.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:42.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20803, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:28.359810+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:28.359810+0000', 'last_peered': '2024-09-15T21:32:28.359810+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:28.359810+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:42.108 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:32:42.108 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 24908, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:39.579252+0000', 'last_change': '2024-09-15T21:32:26.401928+0000', 'last_active': '2024-09-15T21:32:39.579252+0000', 'last_peered': '2024-09-15T21:32:39.579252+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:32:39.579252+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1891, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2131, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1891}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:42.109 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:42.109 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21036, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:28.359632+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:28.359632+0000', 'last_peered': '2024-09-15T21:32:28.359632+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:28.359632+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:42.109 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:32:42.533 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:42.534 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14249/24 objects degraded (59370.833%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":814,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1574711296,"bytes_avail":288335581184,"bytes_total":289910292480,"degraded_objects":14249,"degraded_total":24,"degraded_ratio":593.70833333333337,"recovering_objects_per_sec":217,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2615,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:32:42.548 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:32:42.548 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:42.868 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:42.869 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:42.885 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-15T21:32:40.903818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35226,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537804,"kb_used_data":184980,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281577716,"statfs":{"total":289910292480,"available":288335581184,"internally_reserved":0,"allocated":189419520,"data_stored":188391660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1385221123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2615,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2615,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001949"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21259,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.360788+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:28.360788+0000","last_peered":"2024-09-15T21:32:28.360788+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:28.360788+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20811,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357508+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:28.357508+0000","last_peered":"2024-09-15T21:32:28.357508+0000","last_clean":"2024-09-15T21:32:28.357508+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:28.357508+0000","last_undegraded":"2024-09-15T21:32:28.357508+0000","last_fullsized":"2024-09-15T21:32:28.357508+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357242+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:28.357242+0000","last_peered":"2024-09-15T21:32:28.357242+0000","last_clean":"2024-09-15T21:32:28.357242+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:28.357242+0000","last_undegraded":"2024-09-15T21:32:28.357242+0000","last_fullsized":"2024-09-15T21:32:28.357242+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20669,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359725+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:28.359725+0000","last_peered":"2024-09-15T21:32:28.359725+0000","last_clean":"2024-09-15T21:32:28.359725+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:28.359725+0000","last_undegraded":"2024-09-15T21:32:28.359725+0000","last_fullsized":"2024-09-15T21:32:28.359725+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20803,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359810+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:28.359810+0000","last_peered":"2024-09-15T21:32:28.359810+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:28.359810+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21028,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359338+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:28.359338+0000","last_peered":"2024-09-15T21:32:28.359338+0000","last_clean":"2024-09-15T21:32:28.359338+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:28.359338+0000","last_undegraded":"2024-09-15T21:32:28.359338+0000","last_fullsized":"2024-09-15T21:32:28.359338+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356806+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:28.356806+0000","last_peered":"2024-09-15T21:32:28.356806+0000","last_clean":"2024-09-15T21:32:28.356806+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:28.356806+0000","last_undegraded":"2024-09-15T21:32:28.356806+0000","last_fullsized":"2024-09-15T21:32:28.356806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":24908,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:39.579252+0000","last_change":"2024-09-15T21:32:26.401928+0000","last_active":"2024-09-15T21:32:39.579252+0000","last_peered":"2024-09-15T21:32:39.579252+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:39.579252+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1891,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2131,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1891}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359473+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:28.359473+0000","last_peered":"2024-09-15T21:32:28.359473+0000","last_clean":"2024-09-15T21:32:28.359473+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:28.359473+0000","last_undegraded":"2024-09-15T21:32:28.359473+0000","last_fullsized":"2024-09-15T21:32:28.359473+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28983,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356845+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:28.356845+0000","last_peered":"2024-09-15T21:32:28.356845+0000","last_clean":"2024-09-15T21:32:28.356845+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:28.356845+0000","last_undegraded":"2024-09-15T21:32:28.356845+0000","last_fullsized":"2024-09-15T21:32:28.356845+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29050,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356478+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:28.356478+0000","last_peered":"2024-09-15T21:32:28.356478+0000","last_clean":"2024-09-15T21:32:28.356478+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:28.356478+0000","last_undegraded":"2024-09-15T21:32:28.356478+0000","last_fullsized":"2024-09-15T21:32:28.356478+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357418+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:28.357418+0000","last_peered":"2024-09-15T21:32:28.357418+0000","last_clean":"2024-09-15T21:32:28.357418+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:28.357418+0000","last_undegraded":"2024-09-15T21:32:28.357418+0000","last_fullsized":"2024-09-15T21:32:28.357418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359632+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:28.359632+0000","last_peered":"2024-09-15T21:32:28.359632+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:28.359632+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14249,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35225,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":187936768,"data_stored":187828378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478244,"kb_used_data":117524,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93893596,"statfs":{"total":96636764160,"available":96147042304,"internally_reserved":0,"allocated":120344576,"data_stored":120004674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523676,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848164,"statfs":{"total":96636764160,"available":96100519936,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":119853056,"data_stored":119816926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:45.887 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:46.187 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:46.188 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:46.200 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-15T21:32:44.904439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36105,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1539564,"kb_used_data":182580,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281575956,"statfs":{"total":289910292480,"available":288333778944,"internally_reserved":0,"allocated":186961920,"data_stored":185934060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2533,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001905"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21259,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.360788+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:28.360788+0000","last_peered":"2024-09-15T21:32:28.360788+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:28.360788+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20811,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357508+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:28.357508+0000","last_peered":"2024-09-15T21:32:28.357508+0000","last_clean":"2024-09-15T21:32:28.357508+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:28.357508+0000","last_undegraded":"2024-09-15T21:32:28.357508+0000","last_fullsized":"2024-09-15T21:32:28.357508+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357242+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:28.357242+0000","last_peered":"2024-09-15T21:32:28.357242+0000","last_clean":"2024-09-15T21:32:28.357242+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:28.357242+0000","last_undegraded":"2024-09-15T21:32:28.357242+0000","last_fullsized":"2024-09-15T21:32:28.357242+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20669,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359725+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:28.359725+0000","last_peered":"2024-09-15T21:32:28.359725+0000","last_clean":"2024-09-15T21:32:28.359725+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:28.359725+0000","last_undegraded":"2024-09-15T21:32:28.359725+0000","last_fullsized":"2024-09-15T21:32:28.359725+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20803,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359810+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:28.359810+0000","last_peered":"2024-09-15T21:32:28.359810+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:28.359810+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21028,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359338+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:28.359338+0000","last_peered":"2024-09-15T21:32:28.359338+0000","last_clean":"2024-09-15T21:32:28.359338+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:28.359338+0000","last_undegraded":"2024-09-15T21:32:28.359338+0000","last_fullsized":"2024-09-15T21:32:28.359338+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356806+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:28.356806+0000","last_peered":"2024-09-15T21:32:28.356806+0000","last_clean":"2024-09-15T21:32:28.356806+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:28.356806+0000","last_undegraded":"2024-09-15T21:32:28.356806+0000","last_fullsized":"2024-09-15T21:32:28.356806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":26666,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:44.579433+0000","last_change":"2024-09-15T21:32:26.401928+0000","last_active":"2024-09-15T21:32:44.579433+0000","last_peered":"2024-09-15T21:32:44.579433+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:44.579433+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3010,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1012}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359473+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:28.359473+0000","last_peered":"2024-09-15T21:32:28.359473+0000","last_clean":"2024-09-15T21:32:28.359473+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:28.359473+0000","last_undegraded":"2024-09-15T21:32:28.359473+0000","last_fullsized":"2024-09-15T21:32:28.359473+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28983,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356845+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:28.356845+0000","last_peered":"2024-09-15T21:32:28.356845+0000","last_clean":"2024-09-15T21:32:28.356845+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:28.356845+0000","last_undegraded":"2024-09-15T21:32:28.356845+0000","last_fullsized":"2024-09-15T21:32:28.356845+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29050,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356478+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:28.356478+0000","last_peered":"2024-09-15T21:32:28.356478+0000","last_clean":"2024-09-15T21:32:28.356478+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:28.356478+0000","last_undegraded":"2024-09-15T21:32:28.356478+0000","last_fullsized":"2024-09-15T21:32:28.356478+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357418+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:28.357418+0000","last_peered":"2024-09-15T21:32:28.357418+0000","last_clean":"2024-09-15T21:32:28.357418+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:28.357418+0000","last_undegraded":"2024-09-15T21:32:28.357418+0000","last_fullsized":"2024-09-15T21:32:28.357418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359632+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:28.359632+0000","last_peered":"2024-09-15T21:32:28.359632+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:28.359632+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36104,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185479168,"data_stored":185370778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475844,"kb_used_data":115124,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93895996,"statfs":{"total":96636764160,"available":96149499904,"internally_reserved":0,"allocated":117886976,"data_stored":117547074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527836,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844004,"statfs":{"total":96636764160,"available":96096260096,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117395456,"data_stored":117359326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:46.201 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:46.556 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:46.557 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:46.569 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-15T21:32:44.904439+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36105,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1539564,"kb_used_data":182580,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281575956,"statfs":{"total":289910292480,"available":288333778944,"internally_reserved":0,"allocated":186961920,"data_stored":185934060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2533,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001905"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21259,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.360788+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:28.360788+0000","last_peered":"2024-09-15T21:32:28.360788+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:28.360788+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20811,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357508+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:28.357508+0000","last_peered":"2024-09-15T21:32:28.357508+0000","last_clean":"2024-09-15T21:32:28.357508+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:28.357508+0000","last_undegraded":"2024-09-15T21:32:28.357508+0000","last_fullsized":"2024-09-15T21:32:28.357508+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357242+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:28.357242+0000","last_peered":"2024-09-15T21:32:28.357242+0000","last_clean":"2024-09-15T21:32:28.357242+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:28.357242+0000","last_undegraded":"2024-09-15T21:32:28.357242+0000","last_fullsized":"2024-09-15T21:32:28.357242+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20669,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359725+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:28.359725+0000","last_peered":"2024-09-15T21:32:28.359725+0000","last_clean":"2024-09-15T21:32:28.359725+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:28.359725+0000","last_undegraded":"2024-09-15T21:32:28.359725+0000","last_fullsized":"2024-09-15T21:32:28.359725+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20803,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359810+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:28.359810+0000","last_peered":"2024-09-15T21:32:28.359810+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:28.359810+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21028,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359338+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:28.359338+0000","last_peered":"2024-09-15T21:32:28.359338+0000","last_clean":"2024-09-15T21:32:28.359338+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:28.359338+0000","last_undegraded":"2024-09-15T21:32:28.359338+0000","last_fullsized":"2024-09-15T21:32:28.359338+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356806+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:28.356806+0000","last_peered":"2024-09-15T21:32:28.356806+0000","last_clean":"2024-09-15T21:32:28.356806+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:28.356806+0000","last_undegraded":"2024-09-15T21:32:28.356806+0000","last_fullsized":"2024-09-15T21:32:28.356806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":26666,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:44.579433+0000","last_change":"2024-09-15T21:32:26.401928+0000","last_active":"2024-09-15T21:32:44.579433+0000","last_peered":"2024-09-15T21:32:44.579433+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:44.579433+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3010,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1012}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359473+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:28.359473+0000","last_peered":"2024-09-15T21:32:28.359473+0000","last_clean":"2024-09-15T21:32:28.359473+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:28.359473+0000","last_undegraded":"2024-09-15T21:32:28.359473+0000","last_fullsized":"2024-09-15T21:32:28.359473+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28983,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356845+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:28.356845+0000","last_peered":"2024-09-15T21:32:28.356845+0000","last_clean":"2024-09-15T21:32:28.356845+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:28.356845+0000","last_undegraded":"2024-09-15T21:32:28.356845+0000","last_fullsized":"2024-09-15T21:32:28.356845+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29050,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356478+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:28.356478+0000","last_peered":"2024-09-15T21:32:28.356478+0000","last_clean":"2024-09-15T21:32:28.356478+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:28.356478+0000","last_undegraded":"2024-09-15T21:32:28.356478+0000","last_fullsized":"2024-09-15T21:32:28.356478+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357418+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:28.357418+0000","last_peered":"2024-09-15T21:32:28.357418+0000","last_clean":"2024-09-15T21:32:28.357418+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:28.357418+0000","last_undegraded":"2024-09-15T21:32:28.357418+0000","last_fullsized":"2024-09-15T21:32:28.357418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359632+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:28.359632+0000","last_peered":"2024-09-15T21:32:28.359632+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:28.359632+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36104,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185479168,"data_stored":185370778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475844,"kb_used_data":115124,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93895996,"statfs":{"total":96636764160,"available":96149499904,"internally_reserved":0,"allocated":117886976,"data_stored":117547074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527836,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844004,"statfs":{"total":96636764160,"available":96096260096,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117395456,"data_stored":117359326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:46.570 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:46.571 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21259, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:28.360788+0000', 'last_change': '2024-09-15T21:21:14.905393+0000', 'last_active': '2024-09-15T21:32:28.360788+0000', 'last_peered': '2024-09-15T21:32:28.360788+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:28.360788+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4132, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4132}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:46.571 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:46.571 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20803, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:28.359810+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:28.359810+0000', 'last_peered': '2024-09-15T21:32:28.359810+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:28.359810+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:46.571 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:32:46.571 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 26666, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:44.579433+0000', 'last_change': '2024-09-15T21:32:26.401928+0000', 'last_active': '2024-09-15T21:32:44.579433+0000', 'last_peered': '2024-09-15T21:32:44.579433+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:32:44.579433+0000', 'last_undegraded': '2024-09-15T21:21:14.471845+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 42, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1012, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3010, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1012}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:46.571 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:46.571 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21036, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:28.359632+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:28.359632+0000', 'last_peered': '2024-09-15T21:32:28.359632+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:28.359632+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:46.572 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:32:46.946 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:46.947 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13370/24 objects degraded (55708.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":818,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1576513536,"bytes_avail":288333778944,"bytes_total":289910292480,"degraded_objects":13370,"degraded_total":24,"degraded_ratio":557.08333333333337,"recovering_objects_per_sec":211,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2533,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:32:46.960 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:32:46.960 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:47.273 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:47.274 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:47.286 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-15T21:32:46.904765+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36105,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531244,"kb_used_data":182580,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281584276,"statfs":{"total":289910292480,"available":288342298624,"internally_reserved":0,"allocated":186961920,"data_stored":185934060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1380961283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1584,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1584,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001925"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21259,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.360788+0000","last_change":"2024-09-15T21:21:14.905393+0000","last_active":"2024-09-15T21:32:28.360788+0000","last_peered":"2024-09-15T21:32:28.360788+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:28.360788+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4132,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20811,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357508+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:28.357508+0000","last_peered":"2024-09-15T21:32:28.357508+0000","last_clean":"2024-09-15T21:32:28.357508+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:28.357508+0000","last_undegraded":"2024-09-15T21:32:28.357508+0000","last_fullsized":"2024-09-15T21:32:28.357508+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357242+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:28.357242+0000","last_peered":"2024-09-15T21:32:28.357242+0000","last_clean":"2024-09-15T21:32:28.357242+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:28.357242+0000","last_undegraded":"2024-09-15T21:32:28.357242+0000","last_fullsized":"2024-09-15T21:32:28.357242+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20669,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359725+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:28.359725+0000","last_peered":"2024-09-15T21:32:28.359725+0000","last_clean":"2024-09-15T21:32:28.359725+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:28.359725+0000","last_undegraded":"2024-09-15T21:32:28.359725+0000","last_fullsized":"2024-09-15T21:32:28.359725+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20803,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359810+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:28.359810+0000","last_peered":"2024-09-15T21:32:28.359810+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:28.359810+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21028,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359338+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:28.359338+0000","last_peered":"2024-09-15T21:32:28.359338+0000","last_clean":"2024-09-15T21:32:28.359338+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:28.359338+0000","last_undegraded":"2024-09-15T21:32:28.359338+0000","last_fullsized":"2024-09-15T21:32:28.359338+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356806+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:28.356806+0000","last_peered":"2024-09-15T21:32:28.356806+0000","last_clean":"2024-09-15T21:32:28.356806+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:28.356806+0000","last_undegraded":"2024-09-15T21:32:28.356806+0000","last_fullsized":"2024-09-15T21:32:28.356806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":26666,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:44.579433+0000","last_change":"2024-09-15T21:32:26.401928+0000","last_active":"2024-09-15T21:32:44.579433+0000","last_peered":"2024-09-15T21:32:44.579433+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:44.579433+0000","last_undegraded":"2024-09-15T21:21:14.471845+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":42,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3010,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1012}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359473+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:28.359473+0000","last_peered":"2024-09-15T21:32:28.359473+0000","last_clean":"2024-09-15T21:32:28.359473+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:28.359473+0000","last_undegraded":"2024-09-15T21:32:28.359473+0000","last_fullsized":"2024-09-15T21:32:28.359473+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28983,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356845+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:28.356845+0000","last_peered":"2024-09-15T21:32:28.356845+0000","last_clean":"2024-09-15T21:32:28.356845+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:28.356845+0000","last_undegraded":"2024-09-15T21:32:28.356845+0000","last_fullsized":"2024-09-15T21:32:28.356845+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29050,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.356478+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:28.356478+0000","last_peered":"2024-09-15T21:32:28.356478+0000","last_clean":"2024-09-15T21:32:28.356478+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:28.356478+0000","last_undegraded":"2024-09-15T21:32:28.356478+0000","last_fullsized":"2024-09-15T21:32:28.356478+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.357418+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:28.357418+0000","last_peered":"2024-09-15T21:32:28.357418+0000","last_clean":"2024-09-15T21:32:28.357418+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:28.357418+0000","last_undegraded":"2024-09-15T21:32:28.357418+0000","last_fullsized":"2024-09-15T21:32:28.357418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21036,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:28.359632+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:28.359632+0000","last_peered":"2024-09-15T21:32:28.359632+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:28.359632+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36104,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185479168,"data_stored":185370778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475844,"kb_used_data":115124,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93895996,"statfs":{"total":96636764160,"available":96149499904,"internally_reserved":0,"allocated":117886976,"data_stored":117547074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519516,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852324,"statfs":{"total":96636764160,"available":96104779776,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117395456,"data_stored":117359326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:50.289 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:50.633 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:50.633 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:50.646 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-09-15T21:32:50.423443+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37178,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1528856,"kb_used_data":180188,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281586664,"statfs":{"total":289910292480,"available":288344743936,"internally_reserved":0,"allocated":184512512,"data_stored":183483295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1380961283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1952,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1952,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.519624"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21384,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:49.582396+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:32:49.582396+0000","last_peered":"2024-09-15T21:32:49.582396+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:49.582396+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4071}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20813,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433410+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:49.433410+0000","last_peered":"2024-09-15T21:32:49.433410+0000","last_clean":"2024-09-15T21:32:49.433410+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:49.433410+0000","last_undegraded":"2024-09-15T21:32:49.433410+0000","last_fullsized":"2024-09-15T21:32:49.433410+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20418,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433316+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:49.433316+0000","last_peered":"2024-09-15T21:32:49.433316+0000","last_clean":"2024-09-15T21:32:49.433316+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:49.433316+0000","last_undegraded":"2024-09-15T21:32:49.433316+0000","last_fullsized":"2024-09-15T21:32:49.433316+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.425976+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:49.425976+0000","last_peered":"2024-09-15T21:32:49.425976+0000","last_clean":"2024-09-15T21:32:49.425976+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:49.425976+0000","last_undegraded":"2024-09-15T21:32:49.425976+0000","last_fullsized":"2024-09-15T21:32:49.425976+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20805,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:49.425532+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:49.425532+0000","last_peered":"2024-09-15T21:32:49.425532+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:49.425532+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21030,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.424685+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:49.424685+0000","last_peered":"2024-09-15T21:32:49.424685+0000","last_clean":"2024-09-15T21:32:49.424685+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:49.424685+0000","last_undegraded":"2024-09-15T21:32:49.424685+0000","last_fullsized":"2024-09-15T21:32:49.424685+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433184+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:49.433184+0000","last_peered":"2024-09-15T21:32:49.433184+0000","last_clean":"2024-09-15T21:32:49.433184+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:49.433184+0000","last_undegraded":"2024-09-15T21:32:49.433184+0000","last_fullsized":"2024-09-15T21:32:49.433184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28692,"reported_epoch":59,"state":"peering","last_fresh":"2024-09-15T21:32:49.006130+0000","last_change":"2024-09-15T21:32:49.431540+0000","last_active":"2024-09-15T21:32:49.431540+0000","last_peered":"2024-09-15T21:32:49.006130+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:49.006130+0000","last_undegraded":"2024-09-15T21:32:49.006130+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.425503+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:49.425503+0000","last_peered":"2024-09-15T21:32:49.425503+0000","last_clean":"2024-09-15T21:32:49.425503+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:49.425503+0000","last_undegraded":"2024-09-15T21:32:49.425503+0000","last_fullsized":"2024-09-15T21:32:49.425503+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28986,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433158+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:49.433158+0000","last_peered":"2024-09-15T21:32:49.433158+0000","last_clean":"2024-09-15T21:32:49.433158+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:49.433158+0000","last_undegraded":"2024-09-15T21:32:49.433158+0000","last_fullsized":"2024-09-15T21:32:49.433158+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29053,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433064+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:49.433064+0000","last_peered":"2024-09-15T21:32:49.433064+0000","last_clean":"2024-09-15T21:32:49.433064+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:49.433064+0000","last_undegraded":"2024-09-15T21:32:49.433064+0000","last_fullsized":"2024-09-15T21:32:49.433064+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433437+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:49.433437+0000","last_peered":"2024-09-15T21:32:49.433437+0000","last_clean":"2024-09-15T21:32:49.433437+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:49.433437+0000","last_undegraded":"2024-09-15T21:32:49.433437+0000","last_fullsized":"2024-09-15T21:32:49.433437+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21039,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:49.427002+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:49.427002+0000","last_peered":"2024-09-15T21:32:49.427002+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:49.427002+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37177,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":183021568,"data_stored":182916791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473456,"kb_used_data":112732,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93898384,"statfs":{"total":96636764160,"available":96151945216,"internally_reserved":0,"allocated":115437568,"data_stored":115096309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519516,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852324,"statfs":{"total":96636764160,"available":96104779776,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114937856,"data_stored":114905339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:50.648 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:50.991 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:50.992 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:51.006 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-15T21:32:50.905462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37178,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1528856,"kb_used_data":180188,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281586664,"statfs":{"total":289910292480,"available":288344743936,"internally_reserved":0,"allocated":184512512,"data_stored":183483295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1380961283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1952,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1952,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001322"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21384,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:49.582396+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:32:49.582396+0000","last_peered":"2024-09-15T21:32:49.582396+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:49.582396+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4071}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20813,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433410+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:49.433410+0000","last_peered":"2024-09-15T21:32:49.433410+0000","last_clean":"2024-09-15T21:32:49.433410+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:49.433410+0000","last_undegraded":"2024-09-15T21:32:49.433410+0000","last_fullsized":"2024-09-15T21:32:49.433410+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20418,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433316+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:49.433316+0000","last_peered":"2024-09-15T21:32:49.433316+0000","last_clean":"2024-09-15T21:32:49.433316+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:49.433316+0000","last_undegraded":"2024-09-15T21:32:49.433316+0000","last_fullsized":"2024-09-15T21:32:49.433316+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.425976+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:49.425976+0000","last_peered":"2024-09-15T21:32:49.425976+0000","last_clean":"2024-09-15T21:32:49.425976+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:49.425976+0000","last_undegraded":"2024-09-15T21:32:49.425976+0000","last_fullsized":"2024-09-15T21:32:49.425976+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20805,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:49.425532+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:49.425532+0000","last_peered":"2024-09-15T21:32:49.425532+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:49.425532+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21030,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.424685+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:49.424685+0000","last_peered":"2024-09-15T21:32:49.424685+0000","last_clean":"2024-09-15T21:32:49.424685+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:49.424685+0000","last_undegraded":"2024-09-15T21:32:49.424685+0000","last_fullsized":"2024-09-15T21:32:49.424685+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433184+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:49.433184+0000","last_peered":"2024-09-15T21:32:49.433184+0000","last_clean":"2024-09-15T21:32:49.433184+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:49.433184+0000","last_undegraded":"2024-09-15T21:32:49.433184+0000","last_fullsized":"2024-09-15T21:32:49.433184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28692,"reported_epoch":59,"state":"peering","last_fresh":"2024-09-15T21:32:49.006130+0000","last_change":"2024-09-15T21:32:49.431540+0000","last_active":"2024-09-15T21:32:49.431540+0000","last_peered":"2024-09-15T21:32:49.006130+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:49.006130+0000","last_undegraded":"2024-09-15T21:32:49.006130+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.425503+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:49.425503+0000","last_peered":"2024-09-15T21:32:49.425503+0000","last_clean":"2024-09-15T21:32:49.425503+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:49.425503+0000","last_undegraded":"2024-09-15T21:32:49.425503+0000","last_fullsized":"2024-09-15T21:32:49.425503+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28986,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433158+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:49.433158+0000","last_peered":"2024-09-15T21:32:49.433158+0000","last_clean":"2024-09-15T21:32:49.433158+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:49.433158+0000","last_undegraded":"2024-09-15T21:32:49.433158+0000","last_fullsized":"2024-09-15T21:32:49.433158+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29053,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433064+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:49.433064+0000","last_peered":"2024-09-15T21:32:49.433064+0000","last_clean":"2024-09-15T21:32:49.433064+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:49.433064+0000","last_undegraded":"2024-09-15T21:32:49.433064+0000","last_fullsized":"2024-09-15T21:32:49.433064+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433437+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:49.433437+0000","last_peered":"2024-09-15T21:32:49.433437+0000","last_clean":"2024-09-15T21:32:49.433437+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:49.433437+0000","last_undegraded":"2024-09-15T21:32:49.433437+0000","last_fullsized":"2024-09-15T21:32:49.433437+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21039,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:49.427002+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:49.427002+0000","last_peered":"2024-09-15T21:32:49.427002+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:49.427002+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37177,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":183021568,"data_stored":182916791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473456,"kb_used_data":112732,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93898384,"statfs":{"total":96636764160,"available":96151945216,"internally_reserved":0,"allocated":115437568,"data_stored":115096309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519516,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852324,"statfs":{"total":96636764160,"available":96104779776,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114937856,"data_stored":114905339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:51.008 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:51.008 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 21384, 'reported_epoch': 60, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:49.582396+0000', 'last_change': '2024-09-15T21:32:49.006428+0000', 'last_active': '2024-09-15T21:32:49.582396+0000', 'last_peered': '2024-09-15T21:32:49.582396+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:49.582396+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4071, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 61, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4071}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:51.008 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:51.009 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20805, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:49.425532+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:49.425532+0000', 'last_peered': '2024-09-15T21:32:49.425532+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:49.425532+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:51.009 INFO:tasks.lost_unfound.ceph_manager:PG 2.2 is not active+clean 2024-09-15T21:32:51.009 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.2', 'version': "49'20482", 'reported_seq': 28692, 'reported_epoch': 59, 'state': 'peering', 'last_fresh': '2024-09-15T21:32:49.006130+0000', 'last_change': '2024-09-15T21:32:49.431540+0000', 'last_active': '2024-09-15T21:32:49.431540+0000', 'last_peered': '2024-09-15T21:32:49.006130+0000', 'last_clean': '2024-09-15T21:21:12.336162+0000', 'last_became_active': '2024-09-15T21:21:14.875043+0000', 'last_became_peered': '2024-09-15T21:21:14.875043+0000', 'last_unstale': '2024-09-15T21:32:49.006130+0000', 'last_undegraded': '2024-09-15T21:32:49.006130+0000', 'last_fullsized': '2024-09-15T21:21:14.353580+0000', 'mapping_epoch': 60, 'log_start': "45'10400", 'ondisk_log_start': "45'10400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3360", 'last_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:01.957162+0000', 'objects_scrubbed': 3319, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1158, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 27672, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4022, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:51.009 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:51.010 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21039, 'reported_epoch': 60, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:49.427002+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:49.427002+0000', 'last_peered': '2024-09-15T21:32:49.427002+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:49.427002+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:51.010 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:32:51.470 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:51.470 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12297/24 objects degraded (51237.500%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":823,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1565548544,"bytes_avail":288344743936,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":12297,"degraded_total":24,"degraded_ratio":512.375,"recovering_objects_per_sec":243,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1952,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:32:51.485 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:32:51.485 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:51.806 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:51.806 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:51.819 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-15T21:32:50.905462+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37178,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1528856,"kb_used_data":180188,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281586664,"statfs":{"total":289910292480,"available":288344743936,"internally_reserved":0,"allocated":184512512,"data_stored":183483295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1380961283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1952,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1952,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001322"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":21384,"reported_epoch":60,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:49.582396+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:32:49.582396+0000","last_peered":"2024-09-15T21:32:49.582396+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:49.582396+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4071,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":61,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4071}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20813,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433410+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:49.433410+0000","last_peered":"2024-09-15T21:32:49.433410+0000","last_clean":"2024-09-15T21:32:49.433410+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:49.433410+0000","last_undegraded":"2024-09-15T21:32:49.433410+0000","last_fullsized":"2024-09-15T21:32:49.433410+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359991+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:28.359991+0000","last_peered":"2024-09-15T21:32:28.359991+0000","last_clean":"2024-09-15T21:32:28.359991+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:28.359991+0000","last_undegraded":"2024-09-15T21:32:28.359991+0000","last_fullsized":"2024-09-15T21:32:28.359991+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20418,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433316+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:49.433316+0000","last_peered":"2024-09-15T21:32:49.433316+0000","last_clean":"2024-09-15T21:32:49.433316+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:49.433316+0000","last_undegraded":"2024-09-15T21:32:49.433316+0000","last_fullsized":"2024-09-15T21:32:49.433316+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20671,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.425976+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:49.425976+0000","last_peered":"2024-09-15T21:32:49.425976+0000","last_clean":"2024-09-15T21:32:49.425976+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:49.425976+0000","last_undegraded":"2024-09-15T21:32:49.425976+0000","last_fullsized":"2024-09-15T21:32:49.425976+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20805,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:49.425532+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:49.425532+0000","last_peered":"2024-09-15T21:32:49.425532+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:49.425532+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20830,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.442111+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:28.442111+0000","last_peered":"2024-09-15T21:32:28.442111+0000","last_clean":"2024-09-15T21:32:28.442111+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:28.442111+0000","last_undegraded":"2024-09-15T21:32:28.442111+0000","last_fullsized":"2024-09-15T21:32:28.442111+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21030,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.424685+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:49.424685+0000","last_peered":"2024-09-15T21:32:49.424685+0000","last_clean":"2024-09-15T21:32:49.424685+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:49.424685+0000","last_undegraded":"2024-09-15T21:32:49.424685+0000","last_fullsized":"2024-09-15T21:32:49.424685+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.358932+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:28.358932+0000","last_peered":"2024-09-15T21:32:28.358932+0000","last_clean":"2024-09-15T21:32:28.358932+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:28.358932+0000","last_undegraded":"2024-09-15T21:32:28.358932+0000","last_fullsized":"2024-09-15T21:32:28.358932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20675,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359123+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:28.359123+0000","last_peered":"2024-09-15T21:32:28.359123+0000","last_clean":"2024-09-15T21:32:28.359123+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:28.359123+0000","last_undegraded":"2024-09-15T21:32:28.359123+0000","last_fullsized":"2024-09-15T21:32:28.359123+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433184+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:49.433184+0000","last_peered":"2024-09-15T21:32:49.433184+0000","last_clean":"2024-09-15T21:32:49.433184+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:49.433184+0000","last_undegraded":"2024-09-15T21:32:49.433184+0000","last_fullsized":"2024-09-15T21:32:49.433184+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28692,"reported_epoch":59,"state":"peering","last_fresh":"2024-09-15T21:32:49.006130+0000","last_change":"2024-09-15T21:32:49.431540+0000","last_active":"2024-09-15T21:32:49.431540+0000","last_peered":"2024-09-15T21:32:49.006130+0000","last_clean":"2024-09-15T21:21:12.336162+0000","last_became_active":"2024-09-15T21:21:14.875043+0000","last_became_peered":"2024-09-15T21:21:14.875043+0000","last_unstale":"2024-09-15T21:32:49.006130+0000","last_undegraded":"2024-09-15T21:32:49.006130+0000","last_fullsized":"2024-09-15T21:21:14.353580+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":135,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.425503+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:49.425503+0000","last_peered":"2024-09-15T21:32:49.425503+0000","last_clean":"2024-09-15T21:32:49.425503+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:49.425503+0000","last_undegraded":"2024-09-15T21:32:49.425503+0000","last_fullsized":"2024-09-15T21:32:49.425503+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28986,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433158+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:49.433158+0000","last_peered":"2024-09-15T21:32:49.433158+0000","last_clean":"2024-09-15T21:32:49.433158+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:49.433158+0000","last_undegraded":"2024-09-15T21:32:49.433158+0000","last_fullsized":"2024-09-15T21:32:49.433158+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359050+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:28.359050+0000","last_peered":"2024-09-15T21:32:28.359050+0000","last_clean":"2024-09-15T21:32:28.359050+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:28.359050+0000","last_undegraded":"2024-09-15T21:32:28.359050+0000","last_fullsized":"2024-09-15T21:32:28.359050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29053,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433064+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:49.433064+0000","last_peered":"2024-09-15T21:32:49.433064+0000","last_clean":"2024-09-15T21:32:49.433064+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:49.433064+0000","last_undegraded":"2024-09-15T21:32:49.433064+0000","last_fullsized":"2024-09-15T21:32:49.433064+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20848,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359740+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:28.359740+0000","last_peered":"2024-09-15T21:32:28.359740+0000","last_clean":"2024-09-15T21:32:28.359740+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:28.359740+0000","last_undegraded":"2024-09-15T21:32:28.359740+0000","last_fullsized":"2024-09-15T21:32:28.359740+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359245+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:28.359245+0000","last_peered":"2024-09-15T21:32:28.359245+0000","last_clean":"2024-09-15T21:32:28.359245+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:28.359245+0000","last_undegraded":"2024-09-15T21:32:28.359245+0000","last_fullsized":"2024-09-15T21:32:28.359245+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21613,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359572+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:28.359572+0000","last_peered":"2024-09-15T21:32:28.359572+0000","last_clean":"2024-09-15T21:32:28.359572+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:28.359572+0000","last_undegraded":"2024-09-15T21:32:28.359572+0000","last_fullsized":"2024-09-15T21:32:28.359572+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-15T21:32:49.433437+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:49.433437+0000","last_peered":"2024-09-15T21:32:49.433437+0000","last_clean":"2024-09-15T21:32:49.433437+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:49.433437+0000","last_undegraded":"2024-09-15T21:32:49.433437+0000","last_fullsized":"2024-09-15T21:32:49.433437+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28729,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359893+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:28.359893+0000","last_peered":"2024-09-15T21:32:28.359893+0000","last_clean":"2024-09-15T21:32:28.359893+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:28.359893+0000","last_undegraded":"2024-09-15T21:32:28.359893+0000","last_fullsized":"2024-09-15T21:32:28.359893+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359494+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:28.359494+0000","last_peered":"2024-09-15T21:32:28.359494+0000","last_clean":"2024-09-15T21:32:28.359494+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:28.359494+0000","last_undegraded":"2024-09-15T21:32:28.359494+0000","last_fullsized":"2024-09-15T21:32:28.359494+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-15T21:32:28.359668+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:28.359668+0000","last_peered":"2024-09-15T21:32:28.359668+0000","last_clean":"2024-09-15T21:32:28.359668+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:28.359668+0000","last_undegraded":"2024-09-15T21:32:28.359668+0000","last_fullsized":"2024-09-15T21:32:28.359668+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21039,"reported_epoch":60,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:49.427002+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:49.427002+0000","last_peered":"2024-09-15T21:32:49.427002+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:49.427002+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37177,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":183021568,"data_stored":182916791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473456,"kb_used_data":112732,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93898384,"statfs":{"total":96636764160,"available":96151945216,"internally_reserved":0,"allocated":115437568,"data_stored":115096309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535884,"kb_used_data":66932,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835956,"statfs":{"total":96636764160,"available":96088018944,"internally_reserved":0,"allocated":68538368,"data_stored":68194389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519516,"kb_used_data":524,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852324,"statfs":{"total":96636764160,"available":96104779776,"internally_reserved":0,"allocated":536576,"data_stored":192597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114937856,"data_stored":114905339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:54.822 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:55.177 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:55.178 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:55.192 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-15T21:32:54.906158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38054,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534220,"kb_used_data":177232,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281581300,"statfs":{"total":289910292480,"available":288339251200,"internally_reserved":0,"allocated":181485568,"data_stored":180431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1949,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001393"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":23138,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:54.583859+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:32:54.583859+0000","last_peered":"2024-09-15T21:32:54.583859+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:54.583859+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":937,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3195}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21032,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425649+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425649+0000","last_peered":"2024-09-15T21:32:50.425649+0000","last_clean":"2024-09-15T21:32:50.425649+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425649+0000","last_undegraded":"2024-09-15T21:32:50.425649+0000","last_fullsized":"2024-09-15T21:32:50.425649+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21617,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425523+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425523+0000","last_peered":"2024-09-15T21:32:50.425523+0000","last_clean":"2024-09-15T21:32:50.425523+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425523+0000","last_undegraded":"2024-09-15T21:32:50.425523+0000","last_fullsized":"2024-09-15T21:32:50.425523+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38053,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":179953664,"data_stored":179848887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":470464,"kb_used_data":109744,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93901376,"statfs":{"total":96636764160,"available":96155009024,"internally_reserved":0,"allocated":112377856,"data_stored":112031606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527860,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843980,"statfs":{"total":96636764160,"available":96096235520,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111869952,"data_stored":111837435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:55.193 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:55.530 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:55.531 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:55.544 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-15T21:32:54.906158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38054,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534220,"kb_used_data":177232,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281581300,"statfs":{"total":289910292480,"available":288339251200,"internally_reserved":0,"allocated":181485568,"data_stored":180431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1949,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001393"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":23138,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:54.583859+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:32:54.583859+0000","last_peered":"2024-09-15T21:32:54.583859+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:54.583859+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":937,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3195}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21032,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425649+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425649+0000","last_peered":"2024-09-15T21:32:50.425649+0000","last_clean":"2024-09-15T21:32:50.425649+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425649+0000","last_undegraded":"2024-09-15T21:32:50.425649+0000","last_fullsized":"2024-09-15T21:32:50.425649+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21617,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425523+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425523+0000","last_peered":"2024-09-15T21:32:50.425523+0000","last_clean":"2024-09-15T21:32:50.425523+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425523+0000","last_undegraded":"2024-09-15T21:32:50.425523+0000","last_fullsized":"2024-09-15T21:32:50.425523+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38053,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":179953664,"data_stored":179848887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":470464,"kb_used_data":109744,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93901376,"statfs":{"total":96636764160,"available":96155009024,"internally_reserved":0,"allocated":112377856,"data_stored":112031606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527860,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843980,"statfs":{"total":96636764160,"available":96096235520,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111869952,"data_stored":111837435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:55.546 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:55.546 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 23138, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:54.583859+0000', 'last_change': '2024-09-15T21:32:49.006428+0000', 'last_active': '2024-09-15T21:32:54.583859+0000', 'last_peered': '2024-09-15T21:32:54.583859+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:54.583859+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 937, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3195}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:55.546 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:55.546 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20807, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:50.425651+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:50.425651+0000', 'last_peered': '2024-09-15T21:32:50.425651+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:50.425651+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:55.547 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:55.547 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21041, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:50.426062+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:50.426062+0000', 'last_peered': '2024-09-15T21:32:50.426062+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:50.426062+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:55.548 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:32:55.964 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:55.965 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11421/24 objects degraded (47587.500%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":827,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1571041280,"bytes_avail":288339251200,"bytes_total":289910292480,"degraded_objects":11421,"degraded_total":24,"degraded_ratio":475.875,"recovering_objects_per_sec":243,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1949,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:32:55.980 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:32:55.980 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:56.285 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:56.286 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:56.299 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-15T21:32:54.906158+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38054,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1534220,"kb_used_data":177232,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281581300,"statfs":{"total":289910292480,"available":288339251200,"internally_reserved":0,"allocated":181485568,"data_stored":180431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1949,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001393"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":23138,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:54.583859+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:32:54.583859+0000","last_peered":"2024-09-15T21:32:54.583859+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:54.583859+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":937,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3195}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21032,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425649+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425649+0000","last_peered":"2024-09-15T21:32:50.425649+0000","last_clean":"2024-09-15T21:32:50.425649+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425649+0000","last_undegraded":"2024-09-15T21:32:50.425649+0000","last_fullsized":"2024-09-15T21:32:50.425649+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21617,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425523+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425523+0000","last_peered":"2024-09-15T21:32:50.425523+0000","last_clean":"2024-09-15T21:32:50.425523+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425523+0000","last_undegraded":"2024-09-15T21:32:50.425523+0000","last_fullsized":"2024-09-15T21:32:50.425523+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38053,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":179953664,"data_stored":179848887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":470464,"kb_used_data":109744,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93901376,"statfs":{"total":96636764160,"available":96155009024,"internally_reserved":0,"allocated":112377856,"data_stored":112031606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527860,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843980,"statfs":{"total":96636764160,"available":96096235520,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111869952,"data_stored":111837435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:59.304 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:59.606 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:59.606 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:59.618 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-15T21:32:58.906904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38054,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525892,"kb_used_data":177232,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281589628,"statfs":{"total":289910292480,"available":288347779072,"internally_reserved":0,"allocated":181485568,"data_stored":180431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1380961283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1949,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.483505"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":23138,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:54.583859+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:32:54.583859+0000","last_peered":"2024-09-15T21:32:54.583859+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:54.583859+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":937,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3195}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21032,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425649+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425649+0000","last_peered":"2024-09-15T21:32:50.425649+0000","last_clean":"2024-09-15T21:32:50.425649+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425649+0000","last_undegraded":"2024-09-15T21:32:50.425649+0000","last_fullsized":"2024-09-15T21:32:50.425649+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21617,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425523+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425523+0000","last_peered":"2024-09-15T21:32:50.425523+0000","last_clean":"2024-09-15T21:32:50.425523+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425523+0000","last_undegraded":"2024-09-15T21:32:50.425523+0000","last_fullsized":"2024-09-15T21:32:50.425523+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38053,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":179953664,"data_stored":179848887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":470464,"kb_used_data":109744,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93901376,"statfs":{"total":96636764160,"available":96155009024,"internally_reserved":0,"allocated":112377856,"data_stored":112031606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519532,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852308,"statfs":{"total":96636764160,"available":96104763392,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111869952,"data_stored":111837435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:59.619 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:32:59.959 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:32:59.959 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:32:59.971 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-15T21:32:58.906904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38054,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525892,"kb_used_data":177232,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281589628,"statfs":{"total":289910292480,"available":288347779072,"internally_reserved":0,"allocated":181485568,"data_stored":180431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1380961283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1949,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.483505"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":23138,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:54.583859+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:32:54.583859+0000","last_peered":"2024-09-15T21:32:54.583859+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:54.583859+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":937,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3195}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21032,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425649+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425649+0000","last_peered":"2024-09-15T21:32:50.425649+0000","last_clean":"2024-09-15T21:32:50.425649+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425649+0000","last_undegraded":"2024-09-15T21:32:50.425649+0000","last_fullsized":"2024-09-15T21:32:50.425649+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21617,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425523+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425523+0000","last_peered":"2024-09-15T21:32:50.425523+0000","last_clean":"2024-09-15T21:32:50.425523+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425523+0000","last_undegraded":"2024-09-15T21:32:50.425523+0000","last_fullsized":"2024-09-15T21:32:50.425523+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38053,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":179953664,"data_stored":179848887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":470464,"kb_used_data":109744,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93901376,"statfs":{"total":96636764160,"available":96155009024,"internally_reserved":0,"allocated":112377856,"data_stored":112031606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519532,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852308,"statfs":{"total":96636764160,"available":96104763392,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111869952,"data_stored":111837435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:32:59.973 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:32:59.973 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 23138, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:54.583859+0000', 'last_change': '2024-09-15T21:32:49.006428+0000', 'last_active': '2024-09-15T21:32:54.583859+0000', 'last_peered': '2024-09-15T21:32:54.583859+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:54.583859+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3195, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 937, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3195}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:59.973 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:32:59.973 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20807, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:50.425651+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:50.425651+0000', 'last_peered': '2024-09-15T21:32:50.425651+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:50.425651+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:59.973 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:32:59.973 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21041, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:50.426062+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:50.426062+0000', 'last_peered': '2024-09-15T21:32:50.426062+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:50.426062+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:32:59.974 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:00.354 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:00.354 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11421/24 objects degraded (47587.500%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":832,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1562513408,"bytes_avail":288347779072,"bytes_total":289910292480,"degraded_objects":11421,"degraded_total":24,"degraded_ratio":475.875,"recovering_objects_per_sec":205,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1949,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:00.367 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:00.367 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:00.699 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:00.699 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:00.711 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":469,"stamp":"2024-09-15T21:32:58.906904+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38054,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525892,"kb_used_data":177232,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281589628,"statfs":{"total":289910292480,"available":288347779072,"internally_reserved":0,"allocated":181485568,"data_stored":180431438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1380961283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1949,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.483505"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":23138,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:54.583859+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:32:54.583859+0000","last_peered":"2024-09-15T21:32:54.583859+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:54.583859+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":937,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3195}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21032,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425649+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425649+0000","last_peered":"2024-09-15T21:32:50.425649+0000","last_clean":"2024-09-15T21:32:50.425649+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425649+0000","last_undegraded":"2024-09-15T21:32:50.425649+0000","last_fullsized":"2024-09-15T21:32:50.425649+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21617,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425523+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425523+0000","last_peered":"2024-09-15T21:32:50.425523+0000","last_clean":"2024-09-15T21:32:50.425523+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425523+0000","last_undegraded":"2024-09-15T21:32:50.425523+0000","last_fullsized":"2024-09-15T21:32:50.425523+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11421,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38053,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":179953664,"data_stored":179848887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":470464,"kb_used_data":109744,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93901376,"statfs":{"total":96636764160,"available":96155009024,"internally_reserved":0,"allocated":112377856,"data_stored":112031606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519532,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852308,"statfs":{"total":96636764160,"available":96104763392,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":111869952,"data_stored":111837435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:03.714 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:04.043 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:04.043 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:04.057 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-09-15T21:33:02.907582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10468,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39007,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531332,"kb_used_data":174352,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281584188,"statfs":{"total":289910292480,"available":288342208512,"internally_reserved":0,"allocated":178536448,"data_stored":177482318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1829,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1829,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002119"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":25044,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:59.584297+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:32:59.584297+0000","last_peered":"2024-09-15T21:32:59.584297+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:59.584297+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1890,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2242}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21032,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425649+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425649+0000","last_peered":"2024-09-15T21:32:50.425649+0000","last_clean":"2024-09-15T21:32:50.425649+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425649+0000","last_undegraded":"2024-09-15T21:32:50.425649+0000","last_fullsized":"2024-09-15T21:32:50.425649+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21617,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425523+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425523+0000","last_peered":"2024-09-15T21:32:50.425523+0000","last_clean":"2024-09-15T21:32:50.425523+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425523+0000","last_undegraded":"2024-09-15T21:32:50.425523+0000","last_fullsized":"2024-09-15T21:32:50.425523+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10468,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39006,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176881664,"data_stored":176776887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467584,"kb_used_data":106864,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93904256,"statfs":{"total":96636764160,"available":96157958144,"internally_reserved":0,"allocated":109428736,"data_stored":109082486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527852,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843988,"statfs":{"total":96636764160,"available":96096243712,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":108797952,"data_stored":108765435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:04.058 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:04.373 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:04.373 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:04.385 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-09-15T21:33:02.907582+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10468,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39007,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1531332,"kb_used_data":174352,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281584188,"statfs":{"total":289910292480,"available":288342208512,"internally_reserved":0,"allocated":178536448,"data_stored":177482318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1829,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1829,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002119"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":25044,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:59.584297+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:32:59.584297+0000","last_peered":"2024-09-15T21:32:59.584297+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:32:59.584297+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1890,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2242}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21032,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425649+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425649+0000","last_peered":"2024-09-15T21:32:50.425649+0000","last_clean":"2024-09-15T21:32:50.425649+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425649+0000","last_undegraded":"2024-09-15T21:32:50.425649+0000","last_fullsized":"2024-09-15T21:32:50.425649+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21617,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425523+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425523+0000","last_peered":"2024-09-15T21:32:50.425523+0000","last_clean":"2024-09-15T21:32:50.425523+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425523+0000","last_undegraded":"2024-09-15T21:32:50.425523+0000","last_fullsized":"2024-09-15T21:32:50.425523+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10468,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39006,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176881664,"data_stored":176776887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":467584,"kb_used_data":106864,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93904256,"statfs":{"total":96636764160,"available":96157958144,"internally_reserved":0,"allocated":109428736,"data_stored":109082486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527852,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843988,"statfs":{"total":96636764160,"available":96096243712,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":108797952,"data_stored":108765435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:04.387 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:33:04.387 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 25044, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:59.584297+0000', 'last_change': '2024-09-15T21:32:49.006428+0000', 'last_active': '2024-09-15T21:32:59.584297+0000', 'last_peered': '2024-09-15T21:32:59.584297+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:32:59.584297+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2242, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1890, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2242}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:04.387 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:33:04.387 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20807, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:50.425651+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:50.425651+0000', 'last_peered': '2024-09-15T21:32:50.425651+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:50.425651+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:04.387 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:33:04.387 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21041, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:50.426062+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:50.426062+0000', 'last_peered': '2024-09-15T21:32:50.426062+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:50.426062+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:04.388 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:04.802 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:04.803 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10468/24 objects degraded (43616.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":836,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1568083968,"bytes_avail":288342208512,"bytes_total":289910292480,"degraded_objects":10468,"degraded_total":24,"degraded_ratio":436.16666666666674,"recovering_objects_per_sec":152,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1829,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:04.817 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:04.818 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:05.116 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:05.116 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:05.129 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-15T21:33:04.907973+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39769,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1528332,"kb_used_data":171352,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281587188,"statfs":{"total":289910292480,"available":288345280512,"internally_reserved":0,"allocated":175464448,"data_stored":174410318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2591,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2591,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002136"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":26568,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:04.587464+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:33:04.587464+0000","last_peered":"2024-09-15T21:33:04.587464+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:33:04.587464+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2652,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1480}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21033,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425691+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425691+0000","last_peered":"2024-09-15T21:32:50.425691+0000","last_clean":"2024-09-15T21:32:50.425691+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425691+0000","last_undegraded":"2024-09-15T21:32:50.425691+0000","last_fullsized":"2024-09-15T21:32:50.425691+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21617,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425523+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425523+0000","last_peered":"2024-09-15T21:32:50.425523+0000","last_clean":"2024-09-15T21:32:50.425523+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425523+0000","last_undegraded":"2024-09-15T21:32:50.425523+0000","last_fullsized":"2024-09-15T21:32:50.425523+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39768,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173809664,"data_stored":173704887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464584,"kb_used_data":103864,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93907256,"statfs":{"total":96636764160,"available":96161030144,"internally_reserved":0,"allocated":106356736,"data_stored":106010486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527852,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843988,"statfs":{"total":96636764160,"available":96096243712,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105725952,"data_stored":105693435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:08.131 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:08.459 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:08.472 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-09-15T21:33:06.908284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39769,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520012,"kb_used_data":171352,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281595508,"statfs":{"total":289910292480,"available":288353800192,"internally_reserved":0,"allocated":175464448,"data_stored":174410318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1380961283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002126"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":26568,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:04.587464+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:33:04.587464+0000","last_peered":"2024-09-15T21:33:04.587464+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:33:04.587464+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2652,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1480}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21033,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425691+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425691+0000","last_peered":"2024-09-15T21:32:50.425691+0000","last_clean":"2024-09-15T21:32:50.425691+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425691+0000","last_undegraded":"2024-09-15T21:32:50.425691+0000","last_fullsized":"2024-09-15T21:32:50.425691+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21618,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425558+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425558+0000","last_peered":"2024-09-15T21:32:50.425558+0000","last_clean":"2024-09-15T21:32:50.425558+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425558+0000","last_undegraded":"2024-09-15T21:32:50.425558+0000","last_fullsized":"2024-09-15T21:32:50.425558+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39768,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173809664,"data_stored":173704887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464584,"kb_used_data":103864,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93907256,"statfs":{"total":96636764160,"available":96161030144,"internally_reserved":0,"allocated":106356736,"data_stored":106010486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519532,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852308,"statfs":{"total":96636764160,"available":96104763392,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105725952,"data_stored":105693435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:08.474 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:08.789 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:08.790 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:08.805 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":473,"stamp":"2024-09-15T21:33:06.908284+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39769,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520012,"kb_used_data":171352,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281595508,"statfs":{"total":289910292480,"available":288353800192,"internally_reserved":0,"allocated":175464448,"data_stored":174410318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1380961283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002126"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":26568,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:04.587464+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:33:04.587464+0000","last_peered":"2024-09-15T21:33:04.587464+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:33:04.587464+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2652,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1480}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21033,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425691+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425691+0000","last_peered":"2024-09-15T21:32:50.425691+0000","last_clean":"2024-09-15T21:32:50.425691+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425691+0000","last_undegraded":"2024-09-15T21:32:50.425691+0000","last_fullsized":"2024-09-15T21:32:50.425691+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21618,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425558+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425558+0000","last_peered":"2024-09-15T21:32:50.425558+0000","last_clean":"2024-09-15T21:32:50.425558+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425558+0000","last_undegraded":"2024-09-15T21:32:50.425558+0000","last_fullsized":"2024-09-15T21:32:50.425558+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39768,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173809664,"data_stored":173704887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464584,"kb_used_data":103864,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93907256,"statfs":{"total":96636764160,"available":96161030144,"internally_reserved":0,"allocated":106356736,"data_stored":106010486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519532,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852308,"statfs":{"total":96636764160,"available":96104763392,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105725952,"data_stored":105693435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:08.806 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:33:08.806 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 26568, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:04.587464+0000', 'last_change': '2024-09-15T21:32:49.006428+0000', 'last_active': '2024-09-15T21:33:04.587464+0000', 'last_peered': '2024-09-15T21:33:04.587464+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:33:04.587464+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1480, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2652, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1480}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:08.806 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:33:08.806 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20807, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:50.425651+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:50.425651+0000', 'last_peered': '2024-09-15T21:32:50.425651+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:50.425651+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:08.806 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:33:08.806 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21041, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:50.426062+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:50.426062+0000', 'last_peered': '2024-09-15T21:32:50.426062+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:50.426062+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:08.807 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:09.184 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:09.184 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9706/24 objects degraded (40441.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":840,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1556492288,"bytes_avail":288353800192,"bytes_total":289910292480,"degraded_objects":9706,"degraded_total":24,"degraded_ratio":404.41666666666674,"recovering_objects_per_sec":142,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1715,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:09.197 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:09.197 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:09.507 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:09.508 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:09.522 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-09-15T21:33:08.908581+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39769,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520012,"kb_used_data":171352,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281595508,"statfs":{"total":289910292480,"available":288353800192,"internally_reserved":0,"allocated":175464448,"data_stored":174410318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1380961283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1715,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1715,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001998"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":26568,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:04.587464+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:33:04.587464+0000","last_peered":"2024-09-15T21:33:04.587464+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:33:04.587464+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1480,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2652,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1480}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21033,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425691+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425691+0000","last_peered":"2024-09-15T21:32:50.425691+0000","last_clean":"2024-09-15T21:32:50.425691+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425691+0000","last_undegraded":"2024-09-15T21:32:50.425691+0000","last_fullsized":"2024-09-15T21:32:50.425691+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21618,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425558+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425558+0000","last_peered":"2024-09-15T21:32:50.425558+0000","last_clean":"2024-09-15T21:32:50.425558+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425558+0000","last_undegraded":"2024-09-15T21:32:50.425558+0000","last_fullsized":"2024-09-15T21:32:50.425558+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9706,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39768,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":173809664,"data_stored":173704887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":464584,"kb_used_data":103864,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93907256,"statfs":{"total":96636764160,"available":96161030144,"internally_reserved":0,"allocated":106356736,"data_stored":106010486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519532,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852308,"statfs":{"total":96636764160,"available":96104763392,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105725952,"data_stored":105693435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:12.525 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:12.823 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:12.823 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:12.835 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2024-09-15T21:33:12.353406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40767,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525332,"kb_used_data":168352,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281590188,"statfs":{"total":289910292480,"available":288348352512,"internally_reserved":0,"allocated":172392448,"data_stored":171338318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1760,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.446151"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":28564,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:09.589687+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:33:09.589687+0000","last_peered":"2024-09-15T21:33:09.589687+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:33:09.589687+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3650,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-482}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21033,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425691+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425691+0000","last_peered":"2024-09-15T21:32:50.425691+0000","last_clean":"2024-09-15T21:32:50.425691+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425691+0000","last_undegraded":"2024-09-15T21:32:50.425691+0000","last_fullsized":"2024-09-15T21:32:50.425691+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21618,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425558+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425558+0000","last_peered":"2024-09-15T21:32:50.425558+0000","last_clean":"2024-09-15T21:32:50.425558+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425558+0000","last_undegraded":"2024-09-15T21:32:50.425558+0000","last_fullsized":"2024-09-15T21:32:50.425558+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40766,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":170737664,"data_stored":170632887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461584,"kb_used_data":100864,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93910256,"statfs":{"total":96636764160,"available":96164102144,"internally_reserved":0,"allocated":103284736,"data_stored":102938486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527852,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843988,"statfs":{"total":96636764160,"available":96096243712,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102653952,"data_stored":102621435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:12.837 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:13.138 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:13.138 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:13.150 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-09-15T21:33:12.909270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40767,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525332,"kb_used_data":168352,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281590188,"statfs":{"total":289910292480,"available":288348352512,"internally_reserved":0,"allocated":172392448,"data_stored":171338318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1760,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001687"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":28564,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:09.589687+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:33:09.589687+0000","last_peered":"2024-09-15T21:33:09.589687+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:33:09.589687+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3650,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-482}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21033,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425691+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425691+0000","last_peered":"2024-09-15T21:32:50.425691+0000","last_clean":"2024-09-15T21:32:50.425691+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425691+0000","last_undegraded":"2024-09-15T21:32:50.425691+0000","last_fullsized":"2024-09-15T21:32:50.425691+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21618,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425558+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425558+0000","last_peered":"2024-09-15T21:32:50.425558+0000","last_clean":"2024-09-15T21:32:50.425558+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425558+0000","last_undegraded":"2024-09-15T21:32:50.425558+0000","last_fullsized":"2024-09-15T21:32:50.425558+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40766,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":170737664,"data_stored":170632887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461584,"kb_used_data":100864,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93910256,"statfs":{"total":96636764160,"available":96164102144,"internally_reserved":0,"allocated":103284736,"data_stored":102938486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527852,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843988,"statfs":{"total":96636764160,"available":96096243712,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102653952,"data_stored":102621435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:13.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.f is not active+clean 2024-09-15T21:33:13.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.f', 'version': "49'21152", 'reported_seq': 28564, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:09.589687+0000', 'last_change': '2024-09-15T21:32:49.006428+0000', 'last_active': '2024-09-15T21:33:09.589687+0000', 'last_peered': '2024-09-15T21:33:09.589687+0000', 'last_clean': '2024-09-15T21:21:12.309998+0000', 'last_became_active': '2024-09-15T21:21:14.897040+0000', 'last_became_peered': '2024-09-15T21:21:14.897040+0000', 'last_unstale': '2024-09-15T21:33:09.589687+0000', 'last_undegraded': '2024-09-15T21:21:14.479969+0000', 'last_fullsized': '2024-09-15T21:21:14.356468+0000', 'mapping_epoch': 42, 'log_start': "45'11100", 'ondisk_log_start': "45'11100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'objects_scrubbed': 0, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 482, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21152, 'num_write_kb': 42304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3650, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -482}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:13.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:33:13.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20807, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:50.425651+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:32:50.425651+0000', 'last_peered': '2024-09-15T21:32:50.425651+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:32:50.425651+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:13.152 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:33:13.152 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21041, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:32:50.426062+0000', 'last_change': '2024-09-15T21:21:14.906875+0000', 'last_active': '2024-09-15T21:32:50.426062+0000', 'last_peered': '2024-09-15T21:32:50.426062+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:32:50.426062+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4145}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:13.153 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:13.590 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:13.590 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8708/24 objects degraded (36283.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":845,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1561939968,"bytes_avail":288348352512,"bytes_total":289910292480,"degraded_objects":8708,"degraded_total":24,"degraded_ratio":362.83333333333326,"recovering_objects_per_sec":175,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1760,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:13.603 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:13.603 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:14.004 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:14.005 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:14.021 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-15T21:33:13.352899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40767,"num_bytes_recovered":32630687,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525332,"kb_used_data":168352,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281590188,"statfs":{"total":289910292480,"available":288348352512,"internally_reserved":0,"allocated":172392448,"data_stored":171338318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-998,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":998,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.444925"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":28564,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:09.589687+0000","last_change":"2024-09-15T21:32:49.006428+0000","last_active":"2024-09-15T21:33:09.589687+0000","last_peered":"2024-09-15T21:33:09.589687+0000","last_clean":"2024-09-15T21:21:12.309998+0000","last_became_active":"2024-09-15T21:21:14.897040+0000","last_became_peered":"2024-09-15T21:21:14.897040+0000","last_unstale":"2024-09-15T21:33:09.589687+0000","last_undegraded":"2024-09-15T21:21:14.479969+0000","last_fullsized":"2024-09-15T21:21:14.356468+0000","mapping_epoch":42,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3650,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-482}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20815,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703771+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:32:50.703771+0000","last_peered":"2024-09-15T21:32:50.703771+0000","last_clean":"2024-09-15T21:32:50.703771+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:32:50.703771+0000","last_undegraded":"2024-09-15T21:32:50.703771+0000","last_fullsized":"2024-09-15T21:32:50.703771+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29814,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425247+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:32:50.425247+0000","last_peered":"2024-09-15T21:32:50.425247+0000","last_clean":"2024-09-15T21:32:50.425247+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:32:50.425247+0000","last_undegraded":"2024-09-15T21:32:50.425247+0000","last_fullsized":"2024-09-15T21:32:50.425247+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703618+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:32:50.703618+0000","last_peered":"2024-09-15T21:32:50.703618+0000","last_clean":"2024-09-15T21:32:50.703618+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:32:50.703618+0000","last_undegraded":"2024-09-15T21:32:50.703618+0000","last_fullsized":"2024-09-15T21:32:50.703618+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20673,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.426150+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:32:50.426150+0000","last_peered":"2024-09-15T21:32:50.426150+0000","last_clean":"2024-09-15T21:32:50.426150+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:32:50.426150+0000","last_undegraded":"2024-09-15T21:32:50.426150+0000","last_fullsized":"2024-09-15T21:32:50.426150+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20807,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.425651+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:32:50.425651+0000","last_peered":"2024-09-15T21:32:50.425651+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:32:50.425651+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20834,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425001+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:32:50.425001+0000","last_peered":"2024-09-15T21:32:50.425001+0000","last_clean":"2024-09-15T21:32:50.425001+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:32:50.425001+0000","last_undegraded":"2024-09-15T21:32:50.425001+0000","last_fullsized":"2024-09-15T21:32:50.425001+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21033,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425691+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:32:50.425691+0000","last_peered":"2024-09-15T21:32:50.425691+0000","last_clean":"2024-09-15T21:32:50.425691+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:32:50.425691+0000","last_undegraded":"2024-09-15T21:32:50.425691+0000","last_fullsized":"2024-09-15T21:32:50.425691+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.424934+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:32:50.424934+0000","last_peered":"2024-09-15T21:32:50.424934+0000","last_clean":"2024-09-15T21:32:50.424934+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:32:50.424934+0000","last_undegraded":"2024-09-15T21:32:50.424934+0000","last_fullsized":"2024-09-15T21:32:50.424934+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20679,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425144+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:32:50.425144+0000","last_peered":"2024-09-15T21:32:50.425144+0000","last_clean":"2024-09-15T21:32:50.425144+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:32:50.425144+0000","last_undegraded":"2024-09-15T21:32:50.425144+0000","last_fullsized":"2024-09-15T21:32:50.425144+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703494+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:32:50.703494+0000","last_peered":"2024-09-15T21:32:50.703494+0000","last_clean":"2024-09-15T21:32:50.703494+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:32:50.703494+0000","last_undegraded":"2024-09-15T21:32:50.703494+0000","last_fullsized":"2024-09-15T21:32:50.703494+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28701,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.483047+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:32:50.483047+0000","last_peered":"2024-09-15T21:32:50.483047+0000","last_clean":"2024-09-15T21:32:50.483047+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:32:50.483047+0000","last_undegraded":"2024-09-15T21:32:50.483047+0000","last_fullsized":"2024-09-15T21:32:50.483047+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425537+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:32:50.425537+0000","last_peered":"2024-09-15T21:32:50.425537+0000","last_clean":"2024-09-15T21:32:50.425537+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:32:50.425537+0000","last_undegraded":"2024-09-15T21:32:50.425537+0000","last_fullsized":"2024-09-15T21:32:50.425537+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28988,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703520+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:32:50.703520+0000","last_peered":"2024-09-15T21:32:50.703520+0000","last_clean":"2024-09-15T21:32:50.703520+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:32:50.703520+0000","last_undegraded":"2024-09-15T21:32:50.703520+0000","last_fullsized":"2024-09-15T21:32:50.703520+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425051+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:32:50.425051+0000","last_peered":"2024-09-15T21:32:50.425051+0000","last_clean":"2024-09-15T21:32:50.425051+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:32:50.425051+0000","last_undegraded":"2024-09-15T21:32:50.425051+0000","last_fullsized":"2024-09-15T21:32:50.425051+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29055,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703386+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:32:50.703386+0000","last_peered":"2024-09-15T21:32:50.703386+0000","last_clean":"2024-09-15T21:32:50.703386+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:32:50.703386+0000","last_undegraded":"2024-09-15T21:32:50.703386+0000","last_fullsized":"2024-09-15T21:32:50.703386+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20852,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425191+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:32:50.425191+0000","last_peered":"2024-09-15T21:32:50.425191+0000","last_clean":"2024-09-15T21:32:50.425191+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:32:50.425191+0000","last_undegraded":"2024-09-15T21:32:50.425191+0000","last_fullsized":"2024-09-15T21:32:50.425191+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425147+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:32:50.425147+0000","last_peered":"2024-09-15T21:32:50.425147+0000","last_clean":"2024-09-15T21:32:50.425147+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:32:50.425147+0000","last_undegraded":"2024-09-15T21:32:50.425147+0000","last_fullsized":"2024-09-15T21:32:50.425147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21618,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425558+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:32:50.425558+0000","last_peered":"2024-09-15T21:32:50.425558+0000","last_clean":"2024-09-15T21:32:50.425558+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:32:50.425558+0000","last_undegraded":"2024-09-15T21:32:50.425558+0000","last_fullsized":"2024-09-15T21:32:50.425558+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.703775+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:32:50.703775+0000","last_peered":"2024-09-15T21:32:50.703775+0000","last_clean":"2024-09-15T21:32:50.703775+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:32:50.703775+0000","last_undegraded":"2024-09-15T21:32:50.703775+0000","last_fullsized":"2024-09-15T21:32:50.703775+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28733,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425338+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:32:50.425338+0000","last_peered":"2024-09-15T21:32:50.425338+0000","last_clean":"2024-09-15T21:32:50.425338+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:32:50.425338+0000","last_undegraded":"2024-09-15T21:32:50.425338+0000","last_fullsized":"2024-09-15T21:32:50.425338+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425435+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:32:50.425435+0000","last_peered":"2024-09-15T21:32:50.425435+0000","last_clean":"2024-09-15T21:32:50.425435+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:32:50.425435+0000","last_undegraded":"2024-09-15T21:32:50.425435+0000","last_fullsized":"2024-09-15T21:32:50.425435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-15T21:32:50.425105+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:32:50.425105+0000","last_peered":"2024-09-15T21:32:50.425105+0000","last_clean":"2024-09-15T21:32:50.425105+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:32:50.425105+0000","last_undegraded":"2024-09-15T21:32:50.425105+0000","last_fullsized":"2024-09-15T21:32:50.425105+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21041,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:32:50.426062+0000","last_change":"2024-09-15T21:21:14.906875+0000","last_active":"2024-09-15T21:32:50.426062+0000","last_peered":"2024-09-15T21:32:50.426062+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:32:50.426062+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4145},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40766,"num_bytes_recovered":32630668,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":170737664,"data_stored":170632887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461584,"kb_used_data":100864,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93910256,"statfs":{"total":96636764160,"available":96164102144,"internally_reserved":0,"allocated":103284736,"data_stored":102938486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535896,"kb_used_data":66948,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835944,"statfs":{"total":96636764160,"available":96088006656,"internally_reserved":0,"allocated":68554752,"data_stored":68200812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527852,"kb_used_data":540,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843988,"statfs":{"total":96636764160,"available":96096243712,"internally_reserved":0,"allocated":552960,"data_stored":199020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102653952,"data_stored":102621435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:17.022 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:17.345 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:17.346 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:17.358 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-15T21:33:16.909946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7838,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41637,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1509668,"kb_used_data":165156,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281605852,"statfs":{"total":289910292480,"available":288364392448,"internally_reserved":0,"allocated":169119744,"data_stored":168042807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1376701443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1868,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1868,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001364"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29539,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.430682+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.430682+0000","last_peered":"2024-09-15T21:33:13.430682+0000","last_clean":"2024-09-15T21:33:13.430682+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.430682+0000","last_undegraded":"2024-09-15T21:33:13.430682+0000","last_fullsized":"2024-09-15T21:33:13.430682+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20677,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356170+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356170+0000","last_peered":"2024-09-15T21:33:13.356170+0000","last_clean":"2024-09-15T21:33:13.356170+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356170+0000","last_undegraded":"2024-09-15T21:33:13.356170+0000","last_fullsized":"2024-09-15T21:33:13.356170+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20683,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357734+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357734+0000","last_peered":"2024-09-15T21:33:13.357734+0000","last_clean":"2024-09-15T21:33:13.357734+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357734+0000","last_undegraded":"2024-09-15T21:33:13.357734+0000","last_fullsized":"2024-09-15T21:33:13.357734+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28737,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357789+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357789+0000","last_peered":"2024-09-15T21:33:13.357789+0000","last_clean":"2024-09-15T21:33:13.357789+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357789+0000","last_undegraded":"2024-09-15T21:33:13.357789+0000","last_fullsized":"2024-09-15T21:33:13.357789+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21822,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:14.589836+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:14.589836+0000","last_peered":"2024-09-15T21:33:14.589836+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:14.589836+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7838,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41636,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167415808,"data_stored":167318257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458372,"kb_used_data":97644,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93913468,"statfs":{"total":96636764160,"available":96167391232,"internally_reserved":0,"allocated":99987456,"data_stored":99631195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515380,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856460,"statfs":{"total":96636764160,"available":96109015040,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":99340288,"data_stored":99307771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:17.360 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:17.726 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:17.727 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:17.740 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-15T21:33:16.909946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7838,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41637,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1509668,"kb_used_data":165156,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281605852,"statfs":{"total":289910292480,"available":288364392448,"internally_reserved":0,"allocated":169119744,"data_stored":168042807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1376701443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1868,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1868,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001364"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29539,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.430682+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.430682+0000","last_peered":"2024-09-15T21:33:13.430682+0000","last_clean":"2024-09-15T21:33:13.430682+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.430682+0000","last_undegraded":"2024-09-15T21:33:13.430682+0000","last_fullsized":"2024-09-15T21:33:13.430682+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20677,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356170+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356170+0000","last_peered":"2024-09-15T21:33:13.356170+0000","last_clean":"2024-09-15T21:33:13.356170+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356170+0000","last_undegraded":"2024-09-15T21:33:13.356170+0000","last_fullsized":"2024-09-15T21:33:13.356170+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20683,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357734+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357734+0000","last_peered":"2024-09-15T21:33:13.357734+0000","last_clean":"2024-09-15T21:33:13.357734+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357734+0000","last_undegraded":"2024-09-15T21:33:13.357734+0000","last_fullsized":"2024-09-15T21:33:13.357734+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28737,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357789+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357789+0000","last_peered":"2024-09-15T21:33:13.357789+0000","last_clean":"2024-09-15T21:33:13.357789+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357789+0000","last_undegraded":"2024-09-15T21:33:13.357789+0000","last_fullsized":"2024-09-15T21:33:13.357789+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21822,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:14.589836+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:14.589836+0000","last_peered":"2024-09-15T21:33:14.589836+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:14.589836+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7838,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41636,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167415808,"data_stored":167318257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458372,"kb_used_data":97644,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93913468,"statfs":{"total":96636764160,"available":96167391232,"internally_reserved":0,"allocated":99987456,"data_stored":99631195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515380,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856460,"statfs":{"total":96636764160,"available":96109015040,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":99340288,"data_stored":99307771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:17.742 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:33:17.742 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20811, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:13.356087+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:33:13.356087+0000', 'last_peered': '2024-09-15T21:33:13.356087+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:33:13.356087+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:17.742 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:33:17.742 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 21822, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:14.589836+0000', 'last_change': '2024-09-15T21:33:11.515205+0000', 'last_active': '2024-09-15T21:33:14.589836+0000', 'last_peered': '2024-09-15T21:33:14.589836+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:33:14.589836+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3761, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 388, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3759}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:17.743 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:18.160 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:18.160 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7838/24 objects degraded (32658.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":849,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1545900032,"bytes_avail":288364392448,"bytes_total":289910292480,"degraded_objects":7838,"degraded_total":24,"degraded_ratio":326.58333333333331,"recovering_objects_per_sec":233,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":1868,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:18.175 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:18.175 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:18.500 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:18.516 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-15T21:33:16.909946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7838,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41637,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1509668,"kb_used_data":165156,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281605852,"statfs":{"total":289910292480,"available":288364392448,"internally_reserved":0,"allocated":169119744,"data_stored":168042807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1376701443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1868,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1868,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001364"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29539,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.430682+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.430682+0000","last_peered":"2024-09-15T21:33:13.430682+0000","last_clean":"2024-09-15T21:33:13.430682+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.430682+0000","last_undegraded":"2024-09-15T21:33:13.430682+0000","last_fullsized":"2024-09-15T21:33:13.430682+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20677,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356170+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356170+0000","last_peered":"2024-09-15T21:33:13.356170+0000","last_clean":"2024-09-15T21:33:13.356170+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356170+0000","last_undegraded":"2024-09-15T21:33:13.356170+0000","last_fullsized":"2024-09-15T21:33:13.356170+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20683,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357734+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357734+0000","last_peered":"2024-09-15T21:33:13.357734+0000","last_clean":"2024-09-15T21:33:13.357734+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357734+0000","last_undegraded":"2024-09-15T21:33:13.357734+0000","last_fullsized":"2024-09-15T21:33:13.357734+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28737,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357789+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357789+0000","last_peered":"2024-09-15T21:33:13.357789+0000","last_clean":"2024-09-15T21:33:13.357789+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357789+0000","last_undegraded":"2024-09-15T21:33:13.357789+0000","last_fullsized":"2024-09-15T21:33:13.357789+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21822,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:14.589836+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:14.589836+0000","last_peered":"2024-09-15T21:33:14.589836+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:14.589836+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":388,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7838,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41636,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":167415808,"data_stored":167318257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":458372,"kb_used_data":97644,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93913468,"statfs":{"total":96636764160,"available":96167391232,"internally_reserved":0,"allocated":99987456,"data_stored":99631195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515380,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856460,"statfs":{"total":96636764160,"available":96109015040,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":99340288,"data_stored":99307771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:21.518 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:21.868 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:21.868 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:21.881 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-09-15T21:33:20.910639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42470,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1506060,"kb_used_data":161556,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281609460,"statfs":{"total":289910292480,"available":288368087040,"internally_reserved":0,"allocated":165433344,"data_stored":164356407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1376701443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1703,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1703,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.557232"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29539,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.430682+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.430682+0000","last_peered":"2024-09-15T21:33:13.430682+0000","last_clean":"2024-09-15T21:33:13.430682+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.430682+0000","last_undegraded":"2024-09-15T21:33:13.430682+0000","last_fullsized":"2024-09-15T21:33:13.430682+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20677,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356170+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356170+0000","last_peered":"2024-09-15T21:33:13.356170+0000","last_clean":"2024-09-15T21:33:13.356170+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356170+0000","last_undegraded":"2024-09-15T21:33:13.356170+0000","last_fullsized":"2024-09-15T21:33:13.356170+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20683,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357734+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357734+0000","last_peered":"2024-09-15T21:33:13.357734+0000","last_clean":"2024-09-15T21:33:13.357734+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357734+0000","last_undegraded":"2024-09-15T21:33:13.357734+0000","last_fullsized":"2024-09-15T21:33:13.357734+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28737,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357789+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357789+0000","last_peered":"2024-09-15T21:33:13.357789+0000","last_clean":"2024-09-15T21:33:13.357789+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357789+0000","last_undegraded":"2024-09-15T21:33:13.357789+0000","last_fullsized":"2024-09-15T21:33:13.357789+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":23488,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:19.589168+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:19.589168+0000","last_peered":"2024-09-15T21:33:19.589168+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:19.589168+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2928,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1221,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2926}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42469,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163729408,"data_stored":163631857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454764,"kb_used_data":94044,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93917076,"statfs":{"total":96636764160,"available":96171085824,"internally_reserved":0,"allocated":96301056,"data_stored":95944795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515380,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856460,"statfs":{"total":96636764160,"available":96109015040,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":95653888,"data_stored":95621371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:21.883 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:22.206 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:22.207 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:22.220 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-09-15T21:33:20.910639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42470,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1506060,"kb_used_data":161556,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281609460,"statfs":{"total":289910292480,"available":288368087040,"internally_reserved":0,"allocated":165433344,"data_stored":164356407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1376701443},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1703,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1703,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.557232"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29539,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.430682+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.430682+0000","last_peered":"2024-09-15T21:33:13.430682+0000","last_clean":"2024-09-15T21:33:13.430682+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.430682+0000","last_undegraded":"2024-09-15T21:33:13.430682+0000","last_fullsized":"2024-09-15T21:33:13.430682+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20677,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356170+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356170+0000","last_peered":"2024-09-15T21:33:13.356170+0000","last_clean":"2024-09-15T21:33:13.356170+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356170+0000","last_undegraded":"2024-09-15T21:33:13.356170+0000","last_fullsized":"2024-09-15T21:33:13.356170+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20683,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357734+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357734+0000","last_peered":"2024-09-15T21:33:13.357734+0000","last_clean":"2024-09-15T21:33:13.357734+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357734+0000","last_undegraded":"2024-09-15T21:33:13.357734+0000","last_fullsized":"2024-09-15T21:33:13.357734+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28737,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357789+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357789+0000","last_peered":"2024-09-15T21:33:13.357789+0000","last_clean":"2024-09-15T21:33:13.357789+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357789+0000","last_undegraded":"2024-09-15T21:33:13.357789+0000","last_fullsized":"2024-09-15T21:33:13.357789+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":23488,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:19.589168+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:19.589168+0000","last_peered":"2024-09-15T21:33:19.589168+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:19.589168+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2928,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1221,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2926}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42469,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163729408,"data_stored":163631857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454764,"kb_used_data":94044,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93917076,"statfs":{"total":96636764160,"available":96171085824,"internally_reserved":0,"allocated":96301056,"data_stored":95944795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515380,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856460,"statfs":{"total":96636764160,"available":96109015040,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":527167151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":95653888,"data_stored":95621371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:22.222 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:33:22.222 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20811, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:13.356087+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:33:13.356087+0000', 'last_peered': '2024-09-15T21:33:13.356087+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:33:13.356087+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:22.222 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:33:22.222 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 23488, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:19.589168+0000', 'last_change': '2024-09-15T21:33:11.515205+0000', 'last_active': '2024-09-15T21:33:19.589168+0000', 'last_peered': '2024-09-15T21:33:19.589168+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:33:19.589168+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2928, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1221, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2926}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:22.222 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:22.602 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:22.602 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7005/24 objects degraded (29187.500%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":854,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1542205440,"bytes_avail":288368087040,"bytes_total":289910292480,"degraded_objects":7005,"degraded_total":24,"degraded_ratio":291.875,"recovering_objects_per_sec":199,"recovering_bytes_per_sec":56,"recovering_keys_per_sec":0,"num_objects_recovered":1703,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:22.615 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:22.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:22.914 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:22.915 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:22.928 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":483,"stamp":"2024-09-15T21:33:22.911042+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42470,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514380,"kb_used_data":161556,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281601140,"statfs":{"total":289910292480,"available":288359567360,"internally_reserved":0,"allocated":165433344,"data_stored":164356407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1385221123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1703,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1703,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001772"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29539,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.430682+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.430682+0000","last_peered":"2024-09-15T21:33:13.430682+0000","last_clean":"2024-09-15T21:33:13.430682+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.430682+0000","last_undegraded":"2024-09-15T21:33:13.430682+0000","last_fullsized":"2024-09-15T21:33:13.430682+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20677,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356170+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356170+0000","last_peered":"2024-09-15T21:33:13.356170+0000","last_clean":"2024-09-15T21:33:13.356170+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356170+0000","last_undegraded":"2024-09-15T21:33:13.356170+0000","last_fullsized":"2024-09-15T21:33:13.356170+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357772+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357772+0000","last_peered":"2024-09-15T21:33:13.357772+0000","last_clean":"2024-09-15T21:33:13.357772+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357772+0000","last_undegraded":"2024-09-15T21:33:13.357772+0000","last_fullsized":"2024-09-15T21:33:13.357772+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28737,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357789+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357789+0000","last_peered":"2024-09-15T21:33:13.357789+0000","last_clean":"2024-09-15T21:33:13.357789+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357789+0000","last_undegraded":"2024-09-15T21:33:13.357789+0000","last_fullsized":"2024-09-15T21:33:13.357789+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":23488,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:19.589168+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:19.589168+0000","last_peered":"2024-09-15T21:33:19.589168+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:19.589168+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2928,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1221,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2926}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7005,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42469,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163729408,"data_stored":163631857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454764,"kb_used_data":94044,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93917076,"statfs":{"total":96636764160,"available":96171085824,"internally_reserved":0,"allocated":96301056,"data_stored":95944795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523700,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848140,"statfs":{"total":96636764160,"available":96100495360,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":95653888,"data_stored":95621371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:25.930 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:26.235 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:26.249 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2024-09-15T21:33:24.911418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43109,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1510900,"kb_used_data":158076,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281604620,"statfs":{"total":289910292480,"available":288363130880,"internally_reserved":0,"allocated":161869824,"data_stored":160792887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1385221123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.558519"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29539,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.430682+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.430682+0000","last_peered":"2024-09-15T21:33:13.430682+0000","last_clean":"2024-09-15T21:33:13.430682+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.430682+0000","last_undegraded":"2024-09-15T21:33:13.430682+0000","last_fullsized":"2024-09-15T21:33:13.430682+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20678,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356223+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356223+0000","last_peered":"2024-09-15T21:33:13.356223+0000","last_clean":"2024-09-15T21:33:13.356223+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356223+0000","last_undegraded":"2024-09-15T21:33:13.356223+0000","last_fullsized":"2024-09-15T21:33:13.356223+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357772+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357772+0000","last_peered":"2024-09-15T21:33:13.357772+0000","last_clean":"2024-09-15T21:33:13.357772+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357772+0000","last_undegraded":"2024-09-15T21:33:13.357772+0000","last_fullsized":"2024-09-15T21:33:13.357772+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28737,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357789+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357789+0000","last_peered":"2024-09-15T21:33:13.357789+0000","last_clean":"2024-09-15T21:33:13.357789+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357789+0000","last_undegraded":"2024-09-15T21:33:13.357789+0000","last_fullsized":"2024-09-15T21:33:13.357789+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":24766,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:24.588364+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:24.588364+0000","last_peered":"2024-09-15T21:33:24.588364+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:24.588364+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1860,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43108,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160043008,"data_stored":159945457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451284,"kb_used_data":90564,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93920556,"statfs":{"total":96636764160,"available":96174649344,"internally_reserved":0,"allocated":92737536,"data_stored":92381275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523700,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848140,"statfs":{"total":96636764160,"available":96100495360,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91967488,"data_stored":91934971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:26.251 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:26.564 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:26.565 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:26.578 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":484,"stamp":"2024-09-15T21:33:24.911418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43109,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1510900,"kb_used_data":158076,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281604620,"statfs":{"total":289910292480,"available":288363130880,"internally_reserved":0,"allocated":161869824,"data_stored":160792887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1385221123},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2342,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2342,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.558519"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29539,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.430682+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.430682+0000","last_peered":"2024-09-15T21:33:13.430682+0000","last_clean":"2024-09-15T21:33:13.430682+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.430682+0000","last_undegraded":"2024-09-15T21:33:13.430682+0000","last_fullsized":"2024-09-15T21:33:13.430682+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20678,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356223+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356223+0000","last_peered":"2024-09-15T21:33:13.356223+0000","last_clean":"2024-09-15T21:33:13.356223+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356223+0000","last_undegraded":"2024-09-15T21:33:13.356223+0000","last_fullsized":"2024-09-15T21:33:13.356223+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357772+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357772+0000","last_peered":"2024-09-15T21:33:13.357772+0000","last_clean":"2024-09-15T21:33:13.357772+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357772+0000","last_undegraded":"2024-09-15T21:33:13.357772+0000","last_fullsized":"2024-09-15T21:33:13.357772+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28737,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357789+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357789+0000","last_peered":"2024-09-15T21:33:13.357789+0000","last_clean":"2024-09-15T21:33:13.357789+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357789+0000","last_undegraded":"2024-09-15T21:33:13.357789+0000","last_fullsized":"2024-09-15T21:33:13.357789+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":24766,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:24.588364+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:24.588364+0000","last_peered":"2024-09-15T21:33:24.588364+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:24.588364+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1860,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43108,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160043008,"data_stored":159945457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451284,"kb_used_data":90564,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93920556,"statfs":{"total":96636764160,"available":96174649344,"internally_reserved":0,"allocated":92737536,"data_stored":92381275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523700,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848140,"statfs":{"total":96636764160,"available":96100495360,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":535686831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91967488,"data_stored":91934971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:26.579 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:33:26.580 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20811, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:13.356087+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:33:13.356087+0000', 'last_peered': '2024-09-15T21:33:13.356087+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:33:13.356087+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:26.580 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:33:26.580 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 24766, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:24.588364+0000', 'last_change': '2024-09-15T21:33:11.515205+0000', 'last_active': '2024-09-15T21:33:24.588364+0000', 'last_peered': '2024-09-15T21:33:24.588364+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:33:24.588364+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2289, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1860, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2287}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:26.580 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:26.945 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:26.945 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6366/24 objects degraded (26525.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":858,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1547161600,"bytes_avail":288363130880,"bytes_total":289910292480,"degraded_objects":6366,"degraded_total":24,"degraded_ratio":265.25,"recovering_objects_per_sec":202,"recovering_bytes_per_sec":41,"recovering_keys_per_sec":0,"num_objects_recovered":2342,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:26.960 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:26.960 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:27.258 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:27.259 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:27.271 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-15T21:33:26.911736+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43109,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515060,"kb_used_data":158076,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281600460,"statfs":{"total":289910292480,"available":288358871040,"internally_reserved":0,"allocated":161869824,"data_stored":160792887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1472,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002122"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29539,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.430682+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.430682+0000","last_peered":"2024-09-15T21:33:13.430682+0000","last_clean":"2024-09-15T21:33:13.430682+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.430682+0000","last_undegraded":"2024-09-15T21:33:13.430682+0000","last_fullsized":"2024-09-15T21:33:13.430682+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20678,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356223+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356223+0000","last_peered":"2024-09-15T21:33:13.356223+0000","last_clean":"2024-09-15T21:33:13.356223+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356223+0000","last_undegraded":"2024-09-15T21:33:13.356223+0000","last_fullsized":"2024-09-15T21:33:13.356223+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357772+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357772+0000","last_peered":"2024-09-15T21:33:13.357772+0000","last_clean":"2024-09-15T21:33:13.357772+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357772+0000","last_undegraded":"2024-09-15T21:33:13.357772+0000","last_fullsized":"2024-09-15T21:33:13.357772+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28738,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357825+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357825+0000","last_peered":"2024-09-15T21:33:13.357825+0000","last_clean":"2024-09-15T21:33:13.357825+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357825+0000","last_undegraded":"2024-09-15T21:33:13.357825+0000","last_fullsized":"2024-09-15T21:33:13.357825+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":24766,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:24.588364+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:24.588364+0000","last_peered":"2024-09-15T21:33:24.588364+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:24.588364+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1860,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43108,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160043008,"data_stored":159945457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451284,"kb_used_data":90564,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93920556,"statfs":{"total":96636764160,"available":96174649344,"internally_reserved":0,"allocated":92737536,"data_stored":92381275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527860,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843980,"statfs":{"total":96636764160,"available":96096235520,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91967488,"data_stored":91934971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:30.274 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:30.598 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:30.599 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:30.614 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":486,"stamp":"2024-09-15T21:33:28.912120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43109,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1515060,"kb_used_data":158076,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281600460,"statfs":{"total":289910292480,"available":288358871040,"internally_reserved":0,"allocated":161869824,"data_stored":160792887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1472,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1472,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002173"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29539,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.430682+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.430682+0000","last_peered":"2024-09-15T21:33:13.430682+0000","last_clean":"2024-09-15T21:33:13.430682+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.430682+0000","last_undegraded":"2024-09-15T21:33:13.430682+0000","last_fullsized":"2024-09-15T21:33:13.430682+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20678,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356223+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356223+0000","last_peered":"2024-09-15T21:33:13.356223+0000","last_clean":"2024-09-15T21:33:13.356223+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356223+0000","last_undegraded":"2024-09-15T21:33:13.356223+0000","last_fullsized":"2024-09-15T21:33:13.356223+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357772+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357772+0000","last_peered":"2024-09-15T21:33:13.357772+0000","last_clean":"2024-09-15T21:33:13.357772+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357772+0000","last_undegraded":"2024-09-15T21:33:13.357772+0000","last_fullsized":"2024-09-15T21:33:13.357772+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28738,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357825+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357825+0000","last_peered":"2024-09-15T21:33:13.357825+0000","last_clean":"2024-09-15T21:33:13.357825+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357825+0000","last_undegraded":"2024-09-15T21:33:13.357825+0000","last_fullsized":"2024-09-15T21:33:13.357825+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":24766,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:24.588364+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:24.588364+0000","last_peered":"2024-09-15T21:33:24.588364+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:24.588364+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1860,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2287}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43108,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":160043008,"data_stored":159945457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451284,"kb_used_data":90564,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93920556,"statfs":{"total":96636764160,"available":96174649344,"internally_reserved":0,"allocated":92737536,"data_stored":92381275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527860,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843980,"statfs":{"total":96636764160,"available":96096235520,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":91967488,"data_stored":91934971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:30.615 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:30.920 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:30.920 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:30.933 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-15T21:33:30.912406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43879,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511340,"kb_used_data":154356,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281604180,"statfs":{"total":289910292480,"available":288362680320,"internally_reserved":0,"allocated":158060544,"data_stored":156983607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002046"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29539,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.430682+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.430682+0000","last_peered":"2024-09-15T21:33:13.430682+0000","last_clean":"2024-09-15T21:33:13.430682+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.430682+0000","last_undegraded":"2024-09-15T21:33:13.430682+0000","last_fullsized":"2024-09-15T21:33:13.430682+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20678,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356223+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356223+0000","last_peered":"2024-09-15T21:33:13.356223+0000","last_clean":"2024-09-15T21:33:13.356223+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356223+0000","last_undegraded":"2024-09-15T21:33:13.356223+0000","last_fullsized":"2024-09-15T21:33:13.356223+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357772+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357772+0000","last_peered":"2024-09-15T21:33:13.357772+0000","last_clean":"2024-09-15T21:33:13.357772+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357772+0000","last_undegraded":"2024-09-15T21:33:13.357772+0000","last_fullsized":"2024-09-15T21:33:13.357772+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28738,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357825+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357825+0000","last_peered":"2024-09-15T21:33:13.357825+0000","last_clean":"2024-09-15T21:33:13.357825+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357825+0000","last_undegraded":"2024-09-15T21:33:13.357825+0000","last_fullsized":"2024-09-15T21:33:13.357825+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":26306,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:29.593131+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:29.593131+0000","last_peered":"2024-09-15T21:33:29.593131+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:29.593131+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1519,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2630,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1517}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43878,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":156356608,"data_stored":156259057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447564,"kb_used_data":86844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93924276,"statfs":{"total":96636764160,"available":96178458624,"internally_reserved":0,"allocated":88928256,"data_stored":88571995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527860,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843980,"statfs":{"total":96636764160,"available":96096235520,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":88281088,"data_stored":88248571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:30.934 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:33:30.934 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20811, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:13.356087+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:33:13.356087+0000', 'last_peered': '2024-09-15T21:33:13.356087+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:33:13.356087+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:30.934 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:33:30.934 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 26306, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:29.593131+0000', 'last_change': '2024-09-15T21:33:11.515205+0000', 'last_active': '2024-09-15T21:33:29.593131+0000', 'last_peered': '2024-09-15T21:33:29.593131+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:33:29.593131+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1519, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2630, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1517}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:30.935 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:31.349 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:31.349 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6366/24 objects degraded (26525.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":863,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1551421440,"bytes_avail":288358871040,"bytes_total":289910292480,"degraded_objects":6366,"degraded_total":24,"degraded_ratio":265.25,"recovering_objects_per_sec":122,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1472,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:31.363 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:31.363 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:31.679 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:31.680 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:31.692 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-15T21:33:30.912406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43879,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511340,"kb_used_data":154356,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281604180,"statfs":{"total":289910292480,"available":288362680320,"internally_reserved":0,"allocated":158060544,"data_stored":156983607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1389480963},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002046"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29539,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.430682+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.430682+0000","last_peered":"2024-09-15T21:33:13.430682+0000","last_clean":"2024-09-15T21:33:13.430682+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.430682+0000","last_undegraded":"2024-09-15T21:33:13.430682+0000","last_fullsized":"2024-09-15T21:33:13.430682+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20678,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356223+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356223+0000","last_peered":"2024-09-15T21:33:13.356223+0000","last_clean":"2024-09-15T21:33:13.356223+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356223+0000","last_undegraded":"2024-09-15T21:33:13.356223+0000","last_fullsized":"2024-09-15T21:33:13.356223+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357772+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357772+0000","last_peered":"2024-09-15T21:33:13.357772+0000","last_clean":"2024-09-15T21:33:13.357772+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357772+0000","last_undegraded":"2024-09-15T21:33:13.357772+0000","last_fullsized":"2024-09-15T21:33:13.357772+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28738,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357825+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357825+0000","last_peered":"2024-09-15T21:33:13.357825+0000","last_clean":"2024-09-15T21:33:13.357825+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357825+0000","last_undegraded":"2024-09-15T21:33:13.357825+0000","last_fullsized":"2024-09-15T21:33:13.357825+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":26306,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:29.593131+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:29.593131+0000","last_peered":"2024-09-15T21:33:29.593131+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:29.593131+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1519,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2630,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1517}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5596,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43878,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":156356608,"data_stored":156259057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447564,"kb_used_data":86844,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93924276,"statfs":{"total":96636764160,"available":96178458624,"internally_reserved":0,"allocated":88928256,"data_stored":88571995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527860,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93843980,"statfs":{"total":96636764160,"available":96096235520,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":539946671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":88281088,"data_stored":88248571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:34.695 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:35.005 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:35.005 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:35.018 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-15T21:33:34.913035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44858,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1499420,"kb_used_data":150756,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281616100,"statfs":{"total":289910292480,"available":288374886400,"internally_reserved":0,"allocated":154374144,"data_stored":153297207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1380961283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2388,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001992"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29540,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.432927+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.432927+0000","last_peered":"2024-09-15T21:33:13.432927+0000","last_clean":"2024-09-15T21:33:13.432927+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.432927+0000","last_undegraded":"2024-09-15T21:33:13.432927+0000","last_fullsized":"2024-09-15T21:33:13.432927+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20678,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356223+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356223+0000","last_peered":"2024-09-15T21:33:13.356223+0000","last_clean":"2024-09-15T21:33:13.356223+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356223+0000","last_undegraded":"2024-09-15T21:33:13.356223+0000","last_fullsized":"2024-09-15T21:33:13.356223+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357772+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357772+0000","last_peered":"2024-09-15T21:33:13.357772+0000","last_clean":"2024-09-15T21:33:13.357772+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357772+0000","last_undegraded":"2024-09-15T21:33:13.357772+0000","last_fullsized":"2024-09-15T21:33:13.357772+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28738,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357825+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357825+0000","last_peered":"2024-09-15T21:33:13.357825+0000","last_clean":"2024-09-15T21:33:13.357825+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357825+0000","last_undegraded":"2024-09-15T21:33:13.357825+0000","last_fullsized":"2024-09-15T21:33:13.357825+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":28264,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:34.592479+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:34.592479+0000","last_peered":"2024-09-15T21:33:34.592479+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:34.592479+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3609,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-538}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44857,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152670208,"data_stored":152572657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443964,"kb_used_data":83244,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93927876,"statfs":{"total":96636764160,"available":96182145024,"internally_reserved":0,"allocated":85241856,"data_stored":84885595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519540,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852300,"statfs":{"total":96636764160,"available":96104755200,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84594688,"data_stored":84562171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:35.019 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:35.319 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:35.320 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:35.333 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-15T21:33:34.913035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44858,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1499420,"kb_used_data":150756,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281616100,"statfs":{"total":289910292480,"available":288374886400,"internally_reserved":0,"allocated":154374144,"data_stored":153297207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1380961283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2388,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001992"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29540,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.432927+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.432927+0000","last_peered":"2024-09-15T21:33:13.432927+0000","last_clean":"2024-09-15T21:33:13.432927+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.432927+0000","last_undegraded":"2024-09-15T21:33:13.432927+0000","last_fullsized":"2024-09-15T21:33:13.432927+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20678,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356223+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356223+0000","last_peered":"2024-09-15T21:33:13.356223+0000","last_clean":"2024-09-15T21:33:13.356223+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356223+0000","last_undegraded":"2024-09-15T21:33:13.356223+0000","last_fullsized":"2024-09-15T21:33:13.356223+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357772+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357772+0000","last_peered":"2024-09-15T21:33:13.357772+0000","last_clean":"2024-09-15T21:33:13.357772+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357772+0000","last_undegraded":"2024-09-15T21:33:13.357772+0000","last_fullsized":"2024-09-15T21:33:13.357772+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28738,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357825+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357825+0000","last_peered":"2024-09-15T21:33:13.357825+0000","last_clean":"2024-09-15T21:33:13.357825+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357825+0000","last_undegraded":"2024-09-15T21:33:13.357825+0000","last_fullsized":"2024-09-15T21:33:13.357825+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":28264,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:34.592479+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:34.592479+0000","last_peered":"2024-09-15T21:33:34.592479+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:34.592479+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3609,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-538}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44857,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152670208,"data_stored":152572657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443964,"kb_used_data":83244,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93927876,"statfs":{"total":96636764160,"available":96182145024,"internally_reserved":0,"allocated":85241856,"data_stored":84885595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519540,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852300,"statfs":{"total":96636764160,"available":96104755200,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84594688,"data_stored":84562171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:35.334 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:33:35.334 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20811, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:13.356087+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:33:13.356087+0000', 'last_peered': '2024-09-15T21:33:13.356087+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:33:13.356087+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:35.334 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:33:35.334 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 28264, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:34.592479+0000', 'last_change': '2024-09-15T21:33:11.515205+0000', 'last_active': '2024-09-15T21:33:34.592479+0000', 'last_peered': '2024-09-15T21:33:34.592479+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:33:34.592479+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 540, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3609, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -538}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:35.335 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:35.713 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:35.713 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4617/24 objects degraded (19237.500%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":867,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1535406080,"bytes_avail":288374886400,"bytes_total":289910292480,"degraded_objects":4617,"degraded_total":24,"degraded_ratio":192.375,"recovering_objects_per_sec":198,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2388,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:35.727 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:35.727 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:36.079 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:36.079 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:36.092 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-15T21:33:34.913035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44858,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1499420,"kb_used_data":150756,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281616100,"statfs":{"total":289910292480,"available":288374886400,"internally_reserved":0,"allocated":154374144,"data_stored":153297207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1380961283},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2388,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2388,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001992"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29540,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.432927+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.432927+0000","last_peered":"2024-09-15T21:33:13.432927+0000","last_clean":"2024-09-15T21:33:13.432927+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.432927+0000","last_undegraded":"2024-09-15T21:33:13.432927+0000","last_fullsized":"2024-09-15T21:33:13.432927+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20678,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356223+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356223+0000","last_peered":"2024-09-15T21:33:13.356223+0000","last_clean":"2024-09-15T21:33:13.356223+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356223+0000","last_undegraded":"2024-09-15T21:33:13.356223+0000","last_fullsized":"2024-09-15T21:33:13.356223+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357772+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357772+0000","last_peered":"2024-09-15T21:33:13.357772+0000","last_clean":"2024-09-15T21:33:13.357772+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357772+0000","last_undegraded":"2024-09-15T21:33:13.357772+0000","last_fullsized":"2024-09-15T21:33:13.357772+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28738,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357825+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357825+0000","last_peered":"2024-09-15T21:33:13.357825+0000","last_clean":"2024-09-15T21:33:13.357825+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357825+0000","last_undegraded":"2024-09-15T21:33:13.357825+0000","last_fullsized":"2024-09-15T21:33:13.357825+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":28264,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:34.592479+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:34.592479+0000","last_peered":"2024-09-15T21:33:34.592479+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:34.592479+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3609,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-538}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44857,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152670208,"data_stored":152572657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443964,"kb_used_data":83244,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93927876,"statfs":{"total":96636764160,"available":96182145024,"internally_reserved":0,"allocated":85241856,"data_stored":84885595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519540,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":518971,"kb_avail":93852300,"statfs":{"total":96636764160,"available":96104755200,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":531426991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84594688,"data_stored":84562171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:39.095 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:39.434 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:39.435 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:39.448 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-15T21:33:38.913741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44858,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525788,"kb_used_data":150756,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281589732,"statfs":{"total":289910292480,"available":288347885568,"internally_reserved":0,"allocated":154374144,"data_stored":153297207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":979,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001335"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29540,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.432927+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.432927+0000","last_peered":"2024-09-15T21:33:13.432927+0000","last_clean":"2024-09-15T21:33:13.432927+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.432927+0000","last_undegraded":"2024-09-15T21:33:13.432927+0000","last_fullsized":"2024-09-15T21:33:13.432927+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20678,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356223+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356223+0000","last_peered":"2024-09-15T21:33:13.356223+0000","last_clean":"2024-09-15T21:33:13.356223+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356223+0000","last_undegraded":"2024-09-15T21:33:13.356223+0000","last_fullsized":"2024-09-15T21:33:13.356223+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357772+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357772+0000","last_peered":"2024-09-15T21:33:13.357772+0000","last_clean":"2024-09-15T21:33:13.357772+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357772+0000","last_undegraded":"2024-09-15T21:33:13.357772+0000","last_fullsized":"2024-09-15T21:33:13.357772+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28738,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357825+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357825+0000","last_peered":"2024-09-15T21:33:13.357825+0000","last_clean":"2024-09-15T21:33:13.357825+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357825+0000","last_undegraded":"2024-09-15T21:33:13.357825+0000","last_fullsized":"2024-09-15T21:33:13.357825+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":28264,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:34.592479+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:34.592479+0000","last_peered":"2024-09-15T21:33:34.592479+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:34.592479+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3609,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-538}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44857,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152670208,"data_stored":152572657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443964,"kb_used_data":83244,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93927876,"statfs":{"total":96636764160,"available":96182145024,"internally_reserved":0,"allocated":85241856,"data_stored":84885595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545908,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825932,"statfs":{"total":96636764160,"available":96077754368,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84594688,"data_stored":84562171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:39.449 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:39.783 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:39.783 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:39.799 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-15T21:33:38.913741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44858,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525788,"kb_used_data":150756,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281589732,"statfs":{"total":289910292480,"available":288347885568,"internally_reserved":0,"allocated":154374144,"data_stored":153297207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":979,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001335"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29540,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.432927+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.432927+0000","last_peered":"2024-09-15T21:33:13.432927+0000","last_clean":"2024-09-15T21:33:13.432927+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.432927+0000","last_undegraded":"2024-09-15T21:33:13.432927+0000","last_fullsized":"2024-09-15T21:33:13.432927+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20678,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356223+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356223+0000","last_peered":"2024-09-15T21:33:13.356223+0000","last_clean":"2024-09-15T21:33:13.356223+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356223+0000","last_undegraded":"2024-09-15T21:33:13.356223+0000","last_fullsized":"2024-09-15T21:33:13.356223+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357772+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357772+0000","last_peered":"2024-09-15T21:33:13.357772+0000","last_clean":"2024-09-15T21:33:13.357772+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357772+0000","last_undegraded":"2024-09-15T21:33:13.357772+0000","last_fullsized":"2024-09-15T21:33:13.357772+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28738,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357825+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357825+0000","last_peered":"2024-09-15T21:33:13.357825+0000","last_clean":"2024-09-15T21:33:13.357825+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357825+0000","last_undegraded":"2024-09-15T21:33:13.357825+0000","last_fullsized":"2024-09-15T21:33:13.357825+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":28264,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:34.592479+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:34.592479+0000","last_peered":"2024-09-15T21:33:34.592479+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:34.592479+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3609,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-538}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44857,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152670208,"data_stored":152572657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443964,"kb_used_data":83244,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93927876,"statfs":{"total":96636764160,"available":96182145024,"internally_reserved":0,"allocated":85241856,"data_stored":84885595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545908,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825932,"statfs":{"total":96636764160,"available":96077754368,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84594688,"data_stored":84562171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:39.800 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:33:39.800 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 20811, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:13.356087+0000', 'last_change': '2024-09-15T21:21:14.922936+0000', 'last_active': '2024-09-15T21:33:13.356087+0000', 'last_peered': '2024-09-15T21:33:13.356087+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:33:13.356087+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4077, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4075}, {'shards': '0,2', 'objects': 1}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:39.801 INFO:tasks.lost_unfound.ceph_manager:PG 2.4 is not active+clean 2024-09-15T21:33:39.801 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.4', 'version': "49'20866", 'reported_seq': 28264, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:34.592479+0000', 'last_change': '2024-09-15T21:33:11.515205+0000', 'last_active': '2024-09-15T21:33:34.592479+0000', 'last_peered': '2024-09-15T21:33:34.592479+0000', 'last_clean': '2024-09-15T21:21:12.310233+0000', 'last_became_active': '2024-09-15T21:21:14.895821+0000', 'last_became_peered': '2024-09-15T21:21:14.895821+0000', 'last_unstale': '2024-09-15T21:33:34.592479+0000', 'last_undegraded': '2024-09-15T21:21:14.463908+0000', 'last_fullsized': '2024-09-15T21:21:14.352483+0000', 'mapping_epoch': 42, 'log_start': "45'10800", 'ondisk_log_start': "45'10800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:20:59.797974+0000', 'objects_scrubbed': 3286, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 989, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 540, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17570, 'num_write_kb': 28572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3609, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -538}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:39.801 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:40.202 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:40.202 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4617/24 objects degraded (19237.500%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":871,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1562406912,"bytes_avail":288347885568,"bytes_total":289910292480,"degraded_objects":4617,"degraded_total":24,"degraded_ratio":192.375,"recovering_objects_per_sec":122,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":979,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:40.216 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:40.216 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:40.586 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:40.586 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:40.599 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":493,"stamp":"2024-09-15T21:33:38.913741+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44858,"num_bytes_recovered":32631170,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525788,"kb_used_data":150756,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281589732,"statfs":{"total":289910292480,"available":288347885568,"internally_reserved":0,"allocated":154374144,"data_stored":153297207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-979,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":979,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001335"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29540,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.432927+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:13.432927+0000","last_peered":"2024-09-15T21:33:13.432927+0000","last_clean":"2024-09-15T21:33:13.432927+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:13.432927+0000","last_undegraded":"2024-09-15T21:33:13.432927+0000","last_fullsized":"2024-09-15T21:33:13.432927+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20819,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355080+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:13.355080+0000","last_peered":"2024-09-15T21:33:13.355080+0000","last_clean":"2024-09-15T21:33:13.355080+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:13.355080+0000","last_undegraded":"2024-09-15T21:33:13.355080+0000","last_fullsized":"2024-09-15T21:33:13.355080+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357700+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:13.357700+0000","last_peered":"2024-09-15T21:33:13.357700+0000","last_clean":"2024-09-15T21:33:13.357700+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:13.357700+0000","last_undegraded":"2024-09-15T21:33:13.357700+0000","last_fullsized":"2024-09-15T21:33:13.357700+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354926+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:13.354926+0000","last_peered":"2024-09-15T21:33:13.354926+0000","last_clean":"2024-09-15T21:33:13.354926+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:13.354926+0000","last_undegraded":"2024-09-15T21:33:13.354926+0000","last_fullsized":"2024-09-15T21:33:13.354926+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20678,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356223+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:13.356223+0000","last_peered":"2024-09-15T21:33:13.356223+0000","last_clean":"2024-09-15T21:33:13.356223+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:13.356223+0000","last_undegraded":"2024-09-15T21:33:13.356223+0000","last_fullsized":"2024-09-15T21:33:13.356223+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20811,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:13.356087+0000","last_change":"2024-09-15T21:21:14.922936+0000","last_active":"2024-09-15T21:33:13.356087+0000","last_peered":"2024-09-15T21:33:13.356087+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:13.356087+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4077,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4075},{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20838,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357506+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:13.357506+0000","last_peered":"2024-09-15T21:33:13.357506+0000","last_clean":"2024-09-15T21:33:13.357506+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:13.357506+0000","last_undegraded":"2024-09-15T21:33:13.357506+0000","last_fullsized":"2024-09-15T21:33:13.357506+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21037,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355973+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:13.355973+0000","last_peered":"2024-09-15T21:33:13.355973+0000","last_clean":"2024-09-15T21:33:13.355973+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:13.355973+0000","last_undegraded":"2024-09-15T21:33:13.355973+0000","last_fullsized":"2024-09-15T21:33:13.355973+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357424+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:13.357424+0000","last_peered":"2024-09-15T21:33:13.357424+0000","last_clean":"2024-09-15T21:33:13.357424+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:13.357424+0000","last_undegraded":"2024-09-15T21:33:13.357424+0000","last_fullsized":"2024-09-15T21:33:13.357424+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20684,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357772+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:13.357772+0000","last_peered":"2024-09-15T21:33:13.357772+0000","last_clean":"2024-09-15T21:33:13.357772+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:13.357772+0000","last_undegraded":"2024-09-15T21:33:13.357772+0000","last_fullsized":"2024-09-15T21:33:13.357772+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354538+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:13.354538+0000","last_peered":"2024-09-15T21:33:13.354538+0000","last_clean":"2024-09-15T21:33:13.354538+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:13.354538+0000","last_undegraded":"2024-09-15T21:33:13.354538+0000","last_fullsized":"2024-09-15T21:33:13.354538+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28705,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.356330+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:13.356330+0000","last_peered":"2024-09-15T21:33:13.356330+0000","last_clean":"2024-09-15T21:33:13.356330+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:13.356330+0000","last_undegraded":"2024-09-15T21:33:13.356330+0000","last_fullsized":"2024-09-15T21:33:13.356330+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355976+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:13.355976+0000","last_peered":"2024-09-15T21:33:13.355976+0000","last_clean":"2024-09-15T21:33:13.355976+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:13.355976+0000","last_undegraded":"2024-09-15T21:33:13.355976+0000","last_fullsized":"2024-09-15T21:33:13.355976+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28992,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354625+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:13.354625+0000","last_peered":"2024-09-15T21:33:13.354625+0000","last_clean":"2024-09-15T21:33:13.354625+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:13.354625+0000","last_undegraded":"2024-09-15T21:33:13.354625+0000","last_fullsized":"2024-09-15T21:33:13.354625+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":119,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357641+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:13.357641+0000","last_peered":"2024-09-15T21:33:13.357641+0000","last_clean":"2024-09-15T21:33:13.357641+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:13.357641+0000","last_undegraded":"2024-09-15T21:33:13.357641+0000","last_fullsized":"2024-09-15T21:33:13.357641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29059,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.354265+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:13.354265+0000","last_peered":"2024-09-15T21:33:13.354265+0000","last_clean":"2024-09-15T21:33:13.354265+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:13.354265+0000","last_undegraded":"2024-09-15T21:33:13.354265+0000","last_fullsized":"2024-09-15T21:33:13.354265+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20856,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358149+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:13.358149+0000","last_peered":"2024-09-15T21:33:13.358149+0000","last_clean":"2024-09-15T21:33:13.358149+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:13.358149+0000","last_undegraded":"2024-09-15T21:33:13.358149+0000","last_fullsized":"2024-09-15T21:33:13.358149+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.361066+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:13.361066+0000","last_peered":"2024-09-15T21:33:13.361066+0000","last_clean":"2024-09-15T21:33:13.361066+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:13.361066+0000","last_undegraded":"2024-09-15T21:33:13.361066+0000","last_fullsized":"2024-09-15T21:33:13.361066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21622,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357949+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:13.357949+0000","last_peered":"2024-09-15T21:33:13.357949+0000","last_clean":"2024-09-15T21:33:13.357949+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:13.357949+0000","last_undegraded":"2024-09-15T21:33:13.357949+0000","last_fullsized":"2024-09-15T21:33:13.357949+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.355076+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:13.355076+0000","last_peered":"2024-09-15T21:33:13.355076+0000","last_clean":"2024-09-15T21:33:13.355076+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:13.355076+0000","last_undegraded":"2024-09-15T21:33:13.355076+0000","last_fullsized":"2024-09-15T21:33:13.355076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28738,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357825+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:13.357825+0000","last_peered":"2024-09-15T21:33:13.357825+0000","last_clean":"2024-09-15T21:33:13.357825+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:13.357825+0000","last_undegraded":"2024-09-15T21:33:13.357825+0000","last_fullsized":"2024-09-15T21:33:13.357825+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":109,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.357877+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:13.357877+0000","last_peered":"2024-09-15T21:33:13.357877+0000","last_clean":"2024-09-15T21:33:13.357877+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:13.357877+0000","last_undegraded":"2024-09-15T21:33:13.357877+0000","last_fullsized":"2024-09-15T21:33:13.357877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-15T21:33:13.358050+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:13.358050+0000","last_peered":"2024-09-15T21:33:13.358050+0000","last_clean":"2024-09-15T21:33:13.358050+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:13.358050+0000","last_undegraded":"2024-09-15T21:33:13.358050+0000","last_fullsized":"2024-09-15T21:33:13.358050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":28264,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:34.592479+0000","last_change":"2024-09-15T21:33:11.515205+0000","last_active":"2024-09-15T21:33:34.592479+0000","last_peered":"2024-09-15T21:33:34.592479+0000","last_clean":"2024-09-15T21:21:12.310233+0000","last_became_active":"2024-09-15T21:21:14.895821+0000","last_became_peered":"2024-09-15T21:21:14.895821+0000","last_unstale":"2024-09-15T21:33:34.592479+0000","last_undegraded":"2024-09-15T21:21:14.463908+0000","last_fullsized":"2024-09-15T21:21:14.352483+0000","mapping_epoch":42,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3609,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-538}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4617,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44857,"num_bytes_recovered":32631151,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152670208,"data_stored":152572657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":443964,"kb_used_data":83244,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93927876,"statfs":{"total":96636764160,"available":96182145024,"internally_reserved":0,"allocated":85241856,"data_stored":84885595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535916,"kb_used_data":66964,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835924,"statfs":{"total":96636764160,"available":96087986176,"internally_reserved":0,"allocated":68571136,"data_stored":68207185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545908,"kb_used_data":548,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825932,"statfs":{"total":96636764160,"available":96077754368,"internally_reserved":0,"allocated":561152,"data_stored":204427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":84594688,"data_stored":84562171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:43.601 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:43.917 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:43.918 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:43.929 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-09-15T21:33:42.914497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45762,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1509760,"kb_used_data":147204,"kb_used_omap":12,"kb_used_meta":1362483,"kb_avail":281605760,"statfs":{"total":289910292480,"available":288364298240,"internally_reserved":0,"allocated":150736896,"data_stored":149637002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1395182595},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-904,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":904,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001461"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496462+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496462+0000","last_peered":"2024-09-15T21:33:38.496462+0000","last_clean":"2024-09-15T21:33:38.496462+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496462+0000","last_undegraded":"2024-09-15T21:33:38.496462+0000","last_fullsized":"2024-09-15T21:33:38.496462+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":21544,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:39.596186+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:39.596186+0000","last_peered":"2024-09-15T21:33:39.596186+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:39.596186+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":364,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3712}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355944+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355944+0000","last_peered":"2024-09-15T21:33:38.355944+0000","last_clean":"2024-09-15T21:33:38.355944+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355944+0000","last_undegraded":"2024-09-15T21:33:38.355944+0000","last_fullsized":"2024-09-15T21:33:38.355944+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20860,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357558+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357558+0000","last_peered":"2024-09-15T21:33:38.357558+0000","last_clean":"2024-09-15T21:33:38.357558+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357558+0000","last_undegraded":"2024-09-15T21:33:38.357558+0000","last_fullsized":"2024-09-15T21:33:38.357558+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356149+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356149+0000","last_peered":"2024-09-15T21:33:38.356149+0000","last_clean":"2024-09-15T21:33:38.356149+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356149+0000","last_undegraded":"2024-09-15T21:33:38.356149+0000","last_fullsized":"2024-09-15T21:33:38.356149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357542+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357542+0000","last_peered":"2024-09-15T21:33:38.357542+0000","last_clean":"2024-09-15T21:33:38.357542+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357542+0000","last_undegraded":"2024-09-15T21:33:38.357542+0000","last_fullsized":"2024-09-15T21:33:38.357542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45761,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":148647723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":440392,"kb_used_data":79664,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93931448,"statfs":{"total":96636764160,"available":96185802752,"internally_reserved":0,"allocated":81575936,"data_stored":81213227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533440,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":532859,"kb_avail":93838400,"statfs":{"total":96636764160,"available":96090521600,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":545648303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":80666624,"data_stored":80637720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:43.931 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:44.230 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:44.230 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:44.243 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":495,"stamp":"2024-09-15T21:33:42.914497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45762,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1509760,"kb_used_data":147204,"kb_used_omap":12,"kb_used_meta":1362483,"kb_avail":281605760,"statfs":{"total":289910292480,"available":288364298240,"internally_reserved":0,"allocated":150736896,"data_stored":149637002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1395182595},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-904,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":904,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001461"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496462+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496462+0000","last_peered":"2024-09-15T21:33:38.496462+0000","last_clean":"2024-09-15T21:33:38.496462+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496462+0000","last_undegraded":"2024-09-15T21:33:38.496462+0000","last_fullsized":"2024-09-15T21:33:38.496462+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":21544,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:39.596186+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:39.596186+0000","last_peered":"2024-09-15T21:33:39.596186+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:39.596186+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":364,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3712}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355944+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355944+0000","last_peered":"2024-09-15T21:33:38.355944+0000","last_clean":"2024-09-15T21:33:38.355944+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355944+0000","last_undegraded":"2024-09-15T21:33:38.355944+0000","last_fullsized":"2024-09-15T21:33:38.355944+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20860,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357558+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357558+0000","last_peered":"2024-09-15T21:33:38.357558+0000","last_clean":"2024-09-15T21:33:38.357558+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357558+0000","last_undegraded":"2024-09-15T21:33:38.357558+0000","last_fullsized":"2024-09-15T21:33:38.357558+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356149+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356149+0000","last_peered":"2024-09-15T21:33:38.356149+0000","last_clean":"2024-09-15T21:33:38.356149+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356149+0000","last_undegraded":"2024-09-15T21:33:38.356149+0000","last_fullsized":"2024-09-15T21:33:38.356149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357542+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357542+0000","last_peered":"2024-09-15T21:33:38.357542+0000","last_clean":"2024-09-15T21:33:38.357542+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357542+0000","last_undegraded":"2024-09-15T21:33:38.357542+0000","last_fullsized":"2024-09-15T21:33:38.357542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45761,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":148738048,"data_stored":148647723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":440392,"kb_used_data":79664,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93931448,"statfs":{"total":96636764160,"available":96185802752,"internally_reserved":0,"allocated":81575936,"data_stored":81213227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533440,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":532859,"kb_avail":93838400,"statfs":{"total":96636764160,"available":96090521600,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":545648303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":80666624,"data_stored":80637720,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:44.244 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:33:44.244 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 21544, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:39.596186+0000', 'last_change': '2024-09-15T21:33:36.869763+0000', 'last_active': '2024-09-15T21:33:39.596186+0000', 'last_peered': '2024-09-15T21:33:39.596186+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:33:39.596186+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3713, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 364, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3712}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:44.244 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:44.643 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:44.643 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3713/24 objects degraded (15470.833%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":876,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1545994240,"bytes_avail":288364298240,"bytes_total":289910292480,"degraded_objects":3713,"degraded_total":24,"degraded_ratio":154.70833333333334,"recovering_objects_per_sec":112,"recovering_bytes_per_sec":60,"recovering_keys_per_sec":0,"num_objects_recovered":904,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:44.656 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:44.657 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:44.969 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:44.970 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:44.982 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-15T21:33:44.914848+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46449,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1505672,"kb_used_data":143124,"kb_used_omap":12,"kb_used_meta":1362483,"kb_avail":281609848,"statfs":{"total":289910292480,"available":288368484352,"internally_reserved":0,"allocated":146558976,"data_stored":145459082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1395182595},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1591,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1591,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001413"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496462+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496462+0000","last_peered":"2024-09-15T21:33:38.496462+0000","last_clean":"2024-09-15T21:33:38.496462+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496462+0000","last_undegraded":"2024-09-15T21:33:38.496462+0000","last_fullsized":"2024-09-15T21:33:38.496462+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":22918,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:44.590405+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:44.590405+0000","last_peered":"2024-09-15T21:33:44.590405+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:44.590405+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1051,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3025}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":123,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355944+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355944+0000","last_peered":"2024-09-15T21:33:38.355944+0000","last_clean":"2024-09-15T21:33:38.355944+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355944+0000","last_undegraded":"2024-09-15T21:33:38.355944+0000","last_fullsized":"2024-09-15T21:33:38.355944+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20860,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357558+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357558+0000","last_peered":"2024-09-15T21:33:38.357558+0000","last_clean":"2024-09-15T21:33:38.357558+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357558+0000","last_undegraded":"2024-09-15T21:33:38.357558+0000","last_fullsized":"2024-09-15T21:33:38.357558+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356149+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356149+0000","last_peered":"2024-09-15T21:33:38.356149+0000","last_clean":"2024-09-15T21:33:38.356149+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356149+0000","last_undegraded":"2024-09-15T21:33:38.356149+0000","last_fullsized":"2024-09-15T21:33:38.356149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357542+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357542+0000","last_peered":"2024-09-15T21:33:38.357542+0000","last_clean":"2024-09-15T21:33:38.357542+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357542+0000","last_undegraded":"2024-09-15T21:33:38.357542+0000","last_fullsized":"2024-09-15T21:33:38.357542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46448,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144560128,"data_stored":144469803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436304,"kb_used_data":75584,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93935536,"statfs":{"total":96636764160,"available":96189988864,"internally_reserved":0,"allocated":77398016,"data_stored":77035307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533440,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":532859,"kb_avail":93838400,"statfs":{"total":96636764160,"available":96090521600,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":545648303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":76488704,"data_stored":76459800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:47.984 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:48.297 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:48.298 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:48.310 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2024-09-15T21:33:46.915244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46449,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1509832,"kb_used_data":143124,"kb_used_omap":12,"kb_used_meta":1366643,"kb_avail":281605688,"statfs":{"total":289910292480,"available":288364224512,"internally_reserved":0,"allocated":146558976,"data_stored":145459082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1399442435},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1591,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1591,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.559425"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496462+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496462+0000","last_peered":"2024-09-15T21:33:38.496462+0000","last_clean":"2024-09-15T21:33:38.496462+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496462+0000","last_undegraded":"2024-09-15T21:33:38.496462+0000","last_fullsized":"2024-09-15T21:33:38.496462+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":22918,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:44.590405+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:44.590405+0000","last_peered":"2024-09-15T21:33:44.590405+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:44.590405+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1051,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3025}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20860,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357558+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357558+0000","last_peered":"2024-09-15T21:33:38.357558+0000","last_clean":"2024-09-15T21:33:38.357558+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357558+0000","last_undegraded":"2024-09-15T21:33:38.357558+0000","last_fullsized":"2024-09-15T21:33:38.357558+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357542+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357542+0000","last_peered":"2024-09-15T21:33:38.357542+0000","last_clean":"2024-09-15T21:33:38.357542+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357542+0000","last_undegraded":"2024-09-15T21:33:38.357542+0000","last_fullsized":"2024-09-15T21:33:38.357542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46448,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144560128,"data_stored":144469803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436304,"kb_used_data":75584,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93935536,"statfs":{"total":96636764160,"available":96189988864,"internally_reserved":0,"allocated":77398016,"data_stored":77035307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537600,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":537019,"kb_avail":93834240,"statfs":{"total":96636764160,"available":96086261760,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":549908143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":76488704,"data_stored":76459800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:48.311 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:48.632 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:48.633 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:48.645 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":497,"stamp":"2024-09-15T21:33:46.915244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46449,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1509832,"kb_used_data":143124,"kb_used_omap":12,"kb_used_meta":1366643,"kb_avail":281605688,"statfs":{"total":289910292480,"available":288364224512,"internally_reserved":0,"allocated":146558976,"data_stored":145459082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1399442435},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1591,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1591,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.559425"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496462+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496462+0000","last_peered":"2024-09-15T21:33:38.496462+0000","last_clean":"2024-09-15T21:33:38.496462+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496462+0000","last_undegraded":"2024-09-15T21:33:38.496462+0000","last_fullsized":"2024-09-15T21:33:38.496462+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":22918,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:44.590405+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:44.590405+0000","last_peered":"2024-09-15T21:33:44.590405+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:44.590405+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1051,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3025}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20860,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357558+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357558+0000","last_peered":"2024-09-15T21:33:38.357558+0000","last_clean":"2024-09-15T21:33:38.357558+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357558+0000","last_undegraded":"2024-09-15T21:33:38.357558+0000","last_fullsized":"2024-09-15T21:33:38.357558+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357542+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357542+0000","last_peered":"2024-09-15T21:33:38.357542+0000","last_clean":"2024-09-15T21:33:38.357542+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357542+0000","last_undegraded":"2024-09-15T21:33:38.357542+0000","last_fullsized":"2024-09-15T21:33:38.357542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46448,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144560128,"data_stored":144469803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436304,"kb_used_data":75584,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93935536,"statfs":{"total":96636764160,"available":96189988864,"internally_reserved":0,"allocated":77398016,"data_stored":77035307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537600,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":537019,"kb_avail":93834240,"statfs":{"total":96636764160,"available":96086261760,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":549908143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":76488704,"data_stored":76459800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:48.647 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:33:48.647 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 22918, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:44.590405+0000', 'last_change': '2024-09-15T21:33:36.869763+0000', 'last_active': '2024-09-15T21:33:44.590405+0000', 'last_peered': '2024-09-15T21:33:44.590405+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:33:44.590405+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1051, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3025}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:48.647 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:49.028 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:49.028 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3026/24 objects degraded (12608.333%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":880,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1546067968,"bytes_avail":288364224512,"bytes_total":289910292480,"degraded_objects":3026,"degraded_total":24,"degraded_ratio":126.08333333333331,"recovering_objects_per_sec":166,"recovering_bytes_per_sec":50,"recovering_keys_per_sec":0,"num_objects_recovered":1591,"num_bytes_recovered":483,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:49.041 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:49.041 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:49.422 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:49.422 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:49.434 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-15T21:33:48.915537+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46449,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1509832,"kb_used_data":143124,"kb_used_omap":12,"kb_used_meta":1366643,"kb_avail":281605688,"statfs":{"total":289910292480,"available":288364224512,"internally_reserved":0,"allocated":146558976,"data_stored":145459082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1399442435},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1591,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1591,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.560622"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496462+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496462+0000","last_peered":"2024-09-15T21:33:38.496462+0000","last_clean":"2024-09-15T21:33:38.496462+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496462+0000","last_undegraded":"2024-09-15T21:33:38.496462+0000","last_fullsized":"2024-09-15T21:33:38.496462+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":22918,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:44.590405+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:44.590405+0000","last_peered":"2024-09-15T21:33:44.590405+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:44.590405+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1051,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3025}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20860,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357558+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357558+0000","last_peered":"2024-09-15T21:33:38.357558+0000","last_clean":"2024-09-15T21:33:38.357558+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357558+0000","last_undegraded":"2024-09-15T21:33:38.357558+0000","last_fullsized":"2024-09-15T21:33:38.357558+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357542+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357542+0000","last_peered":"2024-09-15T21:33:38.357542+0000","last_clean":"2024-09-15T21:33:38.357542+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357542+0000","last_undegraded":"2024-09-15T21:33:38.357542+0000","last_fullsized":"2024-09-15T21:33:38.357542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46448,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144560128,"data_stored":144469803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":436304,"kb_used_data":75584,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93935536,"statfs":{"total":96636764160,"available":96189988864,"internally_reserved":0,"allocated":77398016,"data_stored":77035307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537600,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":537019,"kb_avail":93834240,"statfs":{"total":96636764160,"available":96086261760,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":549908143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":76488704,"data_stored":76459800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:52.437 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:52.756 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:52.756 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:52.769 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2024-09-15T21:33:50.915836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47077,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1505632,"kb_used_data":138924,"kb_used_omap":12,"kb_used_meta":1366643,"kb_avail":281609888,"statfs":{"total":289910292480,"available":288368525312,"internally_reserved":0,"allocated":142258176,"data_stored":141158282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1399442435},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2219,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2219,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002095"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20429,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496503+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496503+0000","last_peered":"2024-09-15T21:33:38.496503+0000","last_clean":"2024-09-15T21:33:38.496503+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496503+0000","last_undegraded":"2024-09-15T21:33:38.496503+0000","last_fullsized":"2024-09-15T21:33:38.496503+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":24174,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:49.593709+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:49.593709+0000","last_peered":"2024-09-15T21:33:49.593709+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:49.593709+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1679,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2397}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20860,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357558+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357558+0000","last_peered":"2024-09-15T21:33:38.357558+0000","last_clean":"2024-09-15T21:33:38.357558+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357558+0000","last_undegraded":"2024-09-15T21:33:38.357558+0000","last_fullsized":"2024-09-15T21:33:38.357558+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357542+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357542+0000","last_peered":"2024-09-15T21:33:38.357542+0000","last_clean":"2024-09-15T21:33:38.357542+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357542+0000","last_undegraded":"2024-09-15T21:33:38.357542+0000","last_fullsized":"2024-09-15T21:33:38.357542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47076,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":140259328,"data_stored":140169003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432104,"kb_used_data":71384,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93939736,"statfs":{"total":96636764160,"available":96194289664,"internally_reserved":0,"allocated":73097216,"data_stored":72734507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537600,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":537019,"kb_avail":93834240,"statfs":{"total":96636764160,"available":96086261760,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":549908143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72187904,"data_stored":72159000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:52.770 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:53.080 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:53.081 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:53.094 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-15T21:33:52.916197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47077,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513952,"kb_used_data":138924,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281601568,"statfs":{"total":289910292480,"available":288360005632,"internally_reserved":0,"allocated":142258176,"data_stored":141158282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1855,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1855,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002097"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20429,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496503+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496503+0000","last_peered":"2024-09-15T21:33:38.496503+0000","last_clean":"2024-09-15T21:33:38.496503+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496503+0000","last_undegraded":"2024-09-15T21:33:38.496503+0000","last_fullsized":"2024-09-15T21:33:38.496503+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":24174,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:49.593709+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:49.593709+0000","last_peered":"2024-09-15T21:33:49.593709+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:49.593709+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1679,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2397}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20860,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357558+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357558+0000","last_peered":"2024-09-15T21:33:38.357558+0000","last_clean":"2024-09-15T21:33:38.357558+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357558+0000","last_undegraded":"2024-09-15T21:33:38.357558+0000","last_fullsized":"2024-09-15T21:33:38.357558+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357542+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357542+0000","last_peered":"2024-09-15T21:33:38.357542+0000","last_clean":"2024-09-15T21:33:38.357542+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357542+0000","last_undegraded":"2024-09-15T21:33:38.357542+0000","last_fullsized":"2024-09-15T21:33:38.357542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47076,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":140259328,"data_stored":140169003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432104,"kb_used_data":71384,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93939736,"statfs":{"total":96636764160,"available":96194289664,"internally_reserved":0,"allocated":73097216,"data_stored":72734507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545920,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825920,"statfs":{"total":96636764160,"available":96077742080,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72187904,"data_stored":72159000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:53.095 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:33:53.095 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 24174, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:49.593709+0000', 'last_change': '2024-09-15T21:33:36.869763+0000', 'last_active': '2024-09-15T21:33:49.593709+0000', 'last_peered': '2024-09-15T21:33:49.593709+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:33:49.593709+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2398, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1679, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2397}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:53.096 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:53.524 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:53.524 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2398/24 objects degraded (9991.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":885,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1550286848,"bytes_avail":288360005632,"bytes_total":289910292480,"degraded_objects":2398,"degraded_total":24,"degraded_ratio":99.916666666666686,"recovering_objects_per_sec":154,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1855,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:53.537 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:53.537 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:53.857 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:53.858 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:53.870 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-15T21:33:52.916197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47077,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513952,"kb_used_data":138924,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281601568,"statfs":{"total":289910292480,"available":288360005632,"internally_reserved":0,"allocated":142258176,"data_stored":141158282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1855,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1855,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002097"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20429,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496503+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496503+0000","last_peered":"2024-09-15T21:33:38.496503+0000","last_clean":"2024-09-15T21:33:38.496503+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496503+0000","last_undegraded":"2024-09-15T21:33:38.496503+0000","last_fullsized":"2024-09-15T21:33:38.496503+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":24174,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:49.593709+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:49.593709+0000","last_peered":"2024-09-15T21:33:49.593709+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:49.593709+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1679,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2397}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20860,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357558+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357558+0000","last_peered":"2024-09-15T21:33:38.357558+0000","last_clean":"2024-09-15T21:33:38.357558+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357558+0000","last_undegraded":"2024-09-15T21:33:38.357558+0000","last_fullsized":"2024-09-15T21:33:38.357558+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357542+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357542+0000","last_peered":"2024-09-15T21:33:38.357542+0000","last_clean":"2024-09-15T21:33:38.357542+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357542+0000","last_undegraded":"2024-09-15T21:33:38.357542+0000","last_fullsized":"2024-09-15T21:33:38.357542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2398,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47076,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":140259328,"data_stored":140169003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432104,"kb_used_data":71384,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93939736,"statfs":{"total":96636764160,"available":96194289664,"internally_reserved":0,"allocated":73097216,"data_stored":72734507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545920,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825920,"statfs":{"total":96636764160,"available":96077742080,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72187904,"data_stored":72159000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:56.872 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:57.203 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:57.203 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:57.216 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-09-15T21:33:56.916893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47830,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1497272,"kb_used_data":134724,"kb_used_omap":12,"kb_used_meta":1362483,"kb_avail":281618248,"statfs":{"total":289910292480,"available":288377085952,"internally_reserved":0,"allocated":137957376,"data_stored":136857482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1395182595},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1381,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1381,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002044"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20429,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496503+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496503+0000","last_peered":"2024-09-15T21:33:38.496503+0000","last_clean":"2024-09-15T21:33:38.496503+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496503+0000","last_undegraded":"2024-09-15T21:33:38.496503+0000","last_fullsized":"2024-09-15T21:33:38.496503+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":25680,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:54.593847+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:54.593847+0000","last_peered":"2024-09-15T21:33:54.593847+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:54.593847+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2432,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1644}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20861,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357606+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357606+0000","last_peered":"2024-09-15T21:33:38.357606+0000","last_clean":"2024-09-15T21:33:38.357606+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357606+0000","last_undegraded":"2024-09-15T21:33:38.357606+0000","last_fullsized":"2024-09-15T21:33:38.357606+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357583+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357583+0000","last_peered":"2024-09-15T21:33:38.357583+0000","last_clean":"2024-09-15T21:33:38.357583+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357583+0000","last_undegraded":"2024-09-15T21:33:38.357583+0000","last_fullsized":"2024-09-15T21:33:38.357583+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47829,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135958528,"data_stored":135868203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427904,"kb_used_data":67184,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93943936,"statfs":{"total":96636764160,"available":96198590464,"internally_reserved":0,"allocated":68796416,"data_stored":68433707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533440,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":532859,"kb_avail":93838400,"statfs":{"total":96636764160,"available":96090521600,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":545648303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":67887104,"data_stored":67858200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:57.217 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:57.526 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:57.527 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:57.539 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-09-15T21:33:56.916893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47830,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1497272,"kb_used_data":134724,"kb_used_omap":12,"kb_used_meta":1362483,"kb_avail":281618248,"statfs":{"total":289910292480,"available":288377085952,"internally_reserved":0,"allocated":137957376,"data_stored":136857482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1395182595},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1381,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1381,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002044"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20429,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496503+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496503+0000","last_peered":"2024-09-15T21:33:38.496503+0000","last_clean":"2024-09-15T21:33:38.496503+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496503+0000","last_undegraded":"2024-09-15T21:33:38.496503+0000","last_fullsized":"2024-09-15T21:33:38.496503+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":25680,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:54.593847+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:54.593847+0000","last_peered":"2024-09-15T21:33:54.593847+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:54.593847+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2432,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1644}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20861,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357606+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357606+0000","last_peered":"2024-09-15T21:33:38.357606+0000","last_clean":"2024-09-15T21:33:38.357606+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357606+0000","last_undegraded":"2024-09-15T21:33:38.357606+0000","last_fullsized":"2024-09-15T21:33:38.357606+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357583+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357583+0000","last_peered":"2024-09-15T21:33:38.357583+0000","last_clean":"2024-09-15T21:33:38.357583+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357583+0000","last_undegraded":"2024-09-15T21:33:38.357583+0000","last_fullsized":"2024-09-15T21:33:38.357583+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47829,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135958528,"data_stored":135868203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427904,"kb_used_data":67184,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93943936,"statfs":{"total":96636764160,"available":96198590464,"internally_reserved":0,"allocated":68796416,"data_stored":68433707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533440,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":532859,"kb_avail":93838400,"statfs":{"total":96636764160,"available":96090521600,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":545648303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":67887104,"data_stored":67858200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:33:57.542 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:33:57.543 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 25680, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:54.593847+0000', 'last_change': '2024-09-15T21:33:36.869763+0000', 'last_active': '2024-09-15T21:33:54.593847+0000', 'last_peered': '2024-09-15T21:33:54.593847+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:33:54.593847+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1645, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2432, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1644}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:33:57.543 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:33:57.918 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:57.918 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1645/24 objects degraded (6854.167%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":889,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1533206528,"bytes_avail":288377085952,"bytes_total":289910292480,"degraded_objects":1645,"degraded_total":24,"degraded_ratio":68.541666666666671,"recovering_objects_per_sec":115,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1381,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:33:57.934 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:33:57.934 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:33:58.271 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:33:58.272 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:33:58.285 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-09-15T21:33:56.916893+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47830,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1497272,"kb_used_data":134724,"kb_used_omap":12,"kb_used_meta":1362483,"kb_avail":281618248,"statfs":{"total":289910292480,"available":288377085952,"internally_reserved":0,"allocated":137957376,"data_stored":136857482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1395182595},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1381,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1381,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002044"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20429,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496503+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496503+0000","last_peered":"2024-09-15T21:33:38.496503+0000","last_clean":"2024-09-15T21:33:38.496503+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496503+0000","last_undegraded":"2024-09-15T21:33:38.496503+0000","last_fullsized":"2024-09-15T21:33:38.496503+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":25680,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:54.593847+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:54.593847+0000","last_peered":"2024-09-15T21:33:54.593847+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:54.593847+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2432,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1644}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20861,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357606+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357606+0000","last_peered":"2024-09-15T21:33:38.357606+0000","last_clean":"2024-09-15T21:33:38.357606+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357606+0000","last_undegraded":"2024-09-15T21:33:38.357606+0000","last_fullsized":"2024-09-15T21:33:38.357606+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357583+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357583+0000","last_peered":"2024-09-15T21:33:38.357583+0000","last_clean":"2024-09-15T21:33:38.357583+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357583+0000","last_undegraded":"2024-09-15T21:33:38.357583+0000","last_fullsized":"2024-09-15T21:33:38.357583+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47829,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":135958528,"data_stored":135868203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427904,"kb_used_data":67184,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93943936,"statfs":{"total":96636764160,"available":96198590464,"internally_reserved":0,"allocated":68796416,"data_stored":68433707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533440,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":532859,"kb_avail":93838400,"statfs":{"total":96636764160,"available":96090521600,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":545648303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":67887104,"data_stored":67858200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:01.287 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:01.605 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:01.605 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:01.617 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":504,"stamp":"2024-09-15T21:34:00.917533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48816,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493072,"kb_used_data":130524,"kb_used_omap":12,"kb_used_meta":1362483,"kb_avail":281622448,"statfs":{"total":289910292480,"available":288381386752,"internally_reserved":0,"allocated":133656576,"data_stored":132556682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1395182595},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001995"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20429,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496503+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496503+0000","last_peered":"2024-09-15T21:33:38.496503+0000","last_clean":"2024-09-15T21:33:38.496503+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496503+0000","last_undegraded":"2024-09-15T21:33:38.496503+0000","last_fullsized":"2024-09-15T21:33:38.496503+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":27652,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:59.596037+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:59.596037+0000","last_peered":"2024-09-15T21:33:59.596037+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:59.596037+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3418,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20861,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357606+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357606+0000","last_peered":"2024-09-15T21:33:38.357606+0000","last_clean":"2024-09-15T21:33:38.357606+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357606+0000","last_undegraded":"2024-09-15T21:33:38.357606+0000","last_fullsized":"2024-09-15T21:33:38.357606+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357583+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357583+0000","last_peered":"2024-09-15T21:33:38.357583+0000","last_clean":"2024-09-15T21:33:38.357583+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357583+0000","last_undegraded":"2024-09-15T21:33:38.357583+0000","last_fullsized":"2024-09-15T21:33:38.357583+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48815,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131657728,"data_stored":131571016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423704,"kb_used_data":62984,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93948136,"statfs":{"total":96636764160,"available":96202891264,"internally_reserved":0,"allocated":64495616,"data_stored":64132907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533440,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":532859,"kb_avail":93838400,"statfs":{"total":96636764160,"available":96090521600,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":545648303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":63586304,"data_stored":63561013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:01.619 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:01.962 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:01.974 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":504,"stamp":"2024-09-15T21:34:00.917533+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48816,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493072,"kb_used_data":130524,"kb_used_omap":12,"kb_used_meta":1362483,"kb_avail":281622448,"statfs":{"total":289910292480,"available":288381386752,"internally_reserved":0,"allocated":133656576,"data_stored":132556682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1395182595},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2367,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2367,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001995"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20429,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496503+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496503+0000","last_peered":"2024-09-15T21:33:38.496503+0000","last_clean":"2024-09-15T21:33:38.496503+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496503+0000","last_undegraded":"2024-09-15T21:33:38.496503+0000","last_fullsized":"2024-09-15T21:33:38.496503+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":27652,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:59.596037+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:59.596037+0000","last_peered":"2024-09-15T21:33:59.596037+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:59.596037+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3418,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20861,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357606+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357606+0000","last_peered":"2024-09-15T21:33:38.357606+0000","last_clean":"2024-09-15T21:33:38.357606+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357606+0000","last_undegraded":"2024-09-15T21:33:38.357606+0000","last_fullsized":"2024-09-15T21:33:38.357606+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357583+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357583+0000","last_peered":"2024-09-15T21:33:38.357583+0000","last_clean":"2024-09-15T21:33:38.357583+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357583+0000","last_undegraded":"2024-09-15T21:33:38.357583+0000","last_fullsized":"2024-09-15T21:33:38.357583+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48815,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131657728,"data_stored":131571016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423704,"kb_used_data":62984,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93948136,"statfs":{"total":96636764160,"available":96202891264,"internally_reserved":0,"allocated":64495616,"data_stored":64132907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533440,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":532859,"kb_avail":93838400,"statfs":{"total":96636764160,"available":96090521600,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":545648303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":63586304,"data_stored":63561013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:01.976 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:34:01.976 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 27652, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:59.596037+0000', 'last_change': '2024-09-15T21:33:36.869763+0000', 'last_active': '2024-09-15T21:33:59.596037+0000', 'last_peered': '2024-09-15T21:33:59.596037+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:33:59.596037+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 659, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3418, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -658}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:34:01.976 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:34:02.362 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:02.362 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 659/24 objects degraded (2745.833%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":894,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1528905728,"bytes_avail":288381386752,"bytes_total":289910292480,"degraded_objects":659,"degraded_total":24,"degraded_ratio":27.458333333333329,"recovering_objects_per_sec":197,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2367,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:34:02.375 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:34:02.376 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:02.662 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:02.663 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:02.675 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":505,"stamp":"2024-09-15T21:34:02.486475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48816,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1501392,"kb_used_data":130524,"kb_used_omap":12,"kb_used_meta":1370803,"kb_avail":281614128,"statfs":{"total":289910292480,"available":288372867072,"internally_reserved":0,"allocated":133656576,"data_stored":132556682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1403702275},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1739,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1739,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.570638"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29544,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357174+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:33:38.357174+0000","last_peered":"2024-09-15T21:33:38.357174+0000","last_clean":"2024-09-15T21:33:38.357174+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:33:38.357174+0000","last_undegraded":"2024-09-15T21:33:38.357174+0000","last_fullsized":"2024-09-15T21:33:38.357174+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20823,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496695+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:33:38.496695+0000","last_peered":"2024-09-15T21:33:38.496695+0000","last_clean":"2024-09-15T21:33:38.496695+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:33:38.496695+0000","last_undegraded":"2024-09-15T21:33:38.496695+0000","last_fullsized":"2024-09-15T21:33:38.496695+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20429,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496503+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:33:38.496503+0000","last_peered":"2024-09-15T21:33:38.496503+0000","last_clean":"2024-09-15T21:33:38.496503+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:33:38.496503+0000","last_undegraded":"2024-09-15T21:33:38.496503+0000","last_fullsized":"2024-09-15T21:33:38.496503+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20682,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356965+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:33:38.356965+0000","last_peered":"2024-09-15T21:33:38.356965+0000","last_clean":"2024-09-15T21:33:38.356965+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:33:38.356965+0000","last_undegraded":"2024-09-15T21:33:38.356965+0000","last_fullsized":"2024-09-15T21:33:38.356965+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":27652,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:59.596037+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:59.596037+0000","last_peered":"2024-09-15T21:33:59.596037+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:59.596037+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3418,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21041,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356566+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:33:38.356566+0000","last_peered":"2024-09-15T21:33:38.356566+0000","last_clean":"2024-09-15T21:33:38.356566+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:33:38.356566+0000","last_undegraded":"2024-09-15T21:33:38.356566+0000","last_fullsized":"2024-09-15T21:33:38.356566+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496398+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:33:38.496398+0000","last_peered":"2024-09-15T21:33:38.496398+0000","last_clean":"2024-09-15T21:33:38.496398+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:33:38.496398+0000","last_undegraded":"2024-09-15T21:33:38.496398+0000","last_fullsized":"2024-09-15T21:33:38.496398+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28709,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.358086+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:33:38.358086+0000","last_peered":"2024-09-15T21:33:38.358086+0000","last_clean":"2024-09-15T21:33:38.358086+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:33:38.358086+0000","last_undegraded":"2024-09-15T21:33:38.358086+0000","last_fullsized":"2024-09-15T21:33:38.358086+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":145,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357330+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:33:38.357330+0000","last_peered":"2024-09-15T21:33:38.357330+0000","last_clean":"2024-09-15T21:33:38.357330+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:33:38.357330+0000","last_undegraded":"2024-09-15T21:33:38.357330+0000","last_fullsized":"2024-09-15T21:33:38.357330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":28996,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496455+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:33:38.496455+0000","last_peered":"2024-09-15T21:33:38.496455+0000","last_clean":"2024-09-15T21:33:38.496455+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:33:38.496455+0000","last_undegraded":"2024-09-15T21:33:38.496455+0000","last_fullsized":"2024-09-15T21:33:38.496455+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29063,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496324+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:33:38.496324+0000","last_peered":"2024-09-15T21:33:38.496324+0000","last_clean":"2024-09-15T21:33:38.496324+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:33:38.496324+0000","last_undegraded":"2024-09-15T21:33:38.496324+0000","last_fullsized":"2024-09-15T21:33:38.496324+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20861,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357606+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357606+0000","last_peered":"2024-09-15T21:33:38.357606+0000","last_clean":"2024-09-15T21:33:38.357606+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357606+0000","last_undegraded":"2024-09-15T21:33:38.357606+0000","last_fullsized":"2024-09-15T21:33:38.357606+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.496719+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:33:38.496719+0000","last_peered":"2024-09-15T21:33:38.496719+0000","last_clean":"2024-09-15T21:33:38.496719+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:33:38.496719+0000","last_undegraded":"2024-09-15T21:33:38.496719+0000","last_fullsized":"2024-09-15T21:33:38.496719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357583+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357583+0000","last_peered":"2024-09-15T21:33:38.357583+0000","last_clean":"2024-09-15T21:33:38.357583+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357583+0000","last_undegraded":"2024-09-15T21:33:38.357583+0000","last_fullsized":"2024-09-15T21:33:38.357583+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48815,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131657728,"data_stored":131571016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":423704,"kb_used_data":62984,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93948136,"statfs":{"total":96636764160,"available":96202891264,"internally_reserved":0,"allocated":64495616,"data_stored":64132907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535928,"kb_used_data":66980,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93835912,"statfs":{"total":96636764160,"available":96087973888,"internally_reserved":0,"allocated":68587520,"data_stored":68213508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541760,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":541179,"kb_avail":93830080,"statfs":{"total":96636764160,"available":96082001920,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":554167983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":68042752,"data_stored":68006622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":63586304,"data_stored":63561013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:05.678 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:05.971 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:05.982 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-15T21:34:04.918449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48816,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496276,"kb_used_data":125404,"kb_used_omap":12,"kb_used_meta":1370803,"kb_avail":281619244,"statfs":{"total":289910292480,"available":288378105856,"internally_reserved":0,"allocated":128413696,"data_stored":127302259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1403702275},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":986,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001556"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29548,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491875+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:34:03.491875+0000","last_peered":"2024-09-15T21:34:03.491875+0000","last_clean":"2024-09-15T21:34:03.491875+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:34:03.491875+0000","last_undegraded":"2024-09-15T21:34:03.491875+0000","last_fullsized":"2024-09-15T21:34:03.491875+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20827,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347113+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:34:04.347113+0000","last_peered":"2024-09-15T21:34:04.347113+0000","last_clean":"2024-09-15T21:34:04.347113+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:34:04.347113+0000","last_undegraded":"2024-09-15T21:34:04.347113+0000","last_fullsized":"2024-09-15T21:34:04.347113+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347155+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:34:04.347155+0000","last_peered":"2024-09-15T21:34:04.347155+0000","last_clean":"2024-09-15T21:34:04.347155+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:34:04.347155+0000","last_undegraded":"2024-09-15T21:34:04.347155+0000","last_fullsized":"2024-09-15T21:34:04.347155+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20686,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491487+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:34:03.491487+0000","last_peered":"2024-09-15T21:34:03.491487+0000","last_clean":"2024-09-15T21:34:03.491487+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:34:03.491487+0000","last_undegraded":"2024-09-15T21:34:03.491487+0000","last_fullsized":"2024-09-15T21:34:03.491487+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":27652,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:59.596037+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:59.596037+0000","last_peered":"2024-09-15T21:33:59.596037+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:59.596037+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3418,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21045,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491479+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:34:03.491479+0000","last_peered":"2024-09-15T21:34:03.491479+0000","last_clean":"2024-09-15T21:34:03.491479+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:34:03.491479+0000","last_undegraded":"2024-09-15T21:34:03.491479+0000","last_fullsized":"2024-09-15T21:34:03.491479+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346961+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:34:04.346961+0000","last_peered":"2024-09-15T21:34:04.346961+0000","last_clean":"2024-09-15T21:34:04.346961+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:34:04.346961+0000","last_undegraded":"2024-09-15T21:34:04.346961+0000","last_fullsized":"2024-09-15T21:34:04.346961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28713,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491776+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:34:03.491776+0000","last_peered":"2024-09-15T21:34:03.491776+0000","last_clean":"2024-09-15T21:34:03.491776+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:34:03.491776+0000","last_undegraded":"2024-09-15T21:34:03.491776+0000","last_fullsized":"2024-09-15T21:34:03.491776+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491687+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:34:03.491687+0000","last_peered":"2024-09-15T21:34:03.491687+0000","last_clean":"2024-09-15T21:34:03.491687+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:34:03.491687+0000","last_undegraded":"2024-09-15T21:34:03.491687+0000","last_fullsized":"2024-09-15T21:34:03.491687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":29000,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347029+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:34:04.347029+0000","last_peered":"2024-09-15T21:34:04.347029+0000","last_clean":"2024-09-15T21:34:04.347029+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:34:04.347029+0000","last_undegraded":"2024-09-15T21:34:04.347029+0000","last_fullsized":"2024-09-15T21:34:04.347029+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29067,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346908+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:34:04.346908+0000","last_peered":"2024-09-15T21:34:04.346908+0000","last_clean":"2024-09-15T21:34:04.346908+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:34:04.346908+0000","last_undegraded":"2024-09-15T21:34:04.346908+0000","last_fullsized":"2024-09-15T21:34:04.346908+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20861,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357606+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357606+0000","last_peered":"2024-09-15T21:33:38.357606+0000","last_clean":"2024-09-15T21:33:38.357606+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357606+0000","last_undegraded":"2024-09-15T21:33:38.357606+0000","last_fullsized":"2024-09-15T21:33:38.357606+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347268+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:34:04.347268+0000","last_peered":"2024-09-15T21:34:04.347268+0000","last_clean":"2024-09-15T21:34:04.347268+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:34:04.347268+0000","last_undegraded":"2024-09-15T21:34:04.347268+0000","last_fullsized":"2024-09-15T21:34:04.347268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357583+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357583+0000","last_peered":"2024-09-15T21:33:38.357583+0000","last_clean":"2024-09-15T21:33:38.357583+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357583+0000","last_undegraded":"2024-09-15T21:33:38.357583+0000","last_fullsized":"2024-09-15T21:33:38.357583+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48815,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125779968,"data_stored":125693256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419512,"kb_used_data":58792,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93952328,"statfs":{"total":96636764160,"available":96207183872,"internally_reserved":0,"allocated":60203008,"data_stored":59838867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535004,"kb_used_data":66052,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836836,"statfs":{"total":96636764160,"available":96088920064,"internally_reserved":0,"allocated":67637248,"data_stored":67253125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541760,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":541179,"kb_avail":93830080,"statfs":{"total":96636764160,"available":96082001920,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":554167983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66584576,"data_stored":66548446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":59166720,"data_stored":59141429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:05.984 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:06.263 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:06.263 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:06.275 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":508,"stamp":"2024-09-15T21:34:04.918449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48816,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496276,"kb_used_data":125404,"kb_used_omap":12,"kb_used_meta":1370803,"kb_avail":281619244,"statfs":{"total":289910292480,"available":288378105856,"internally_reserved":0,"allocated":128413696,"data_stored":127302259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1403702275},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":986,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001556"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29548,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491875+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:34:03.491875+0000","last_peered":"2024-09-15T21:34:03.491875+0000","last_clean":"2024-09-15T21:34:03.491875+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:34:03.491875+0000","last_undegraded":"2024-09-15T21:34:03.491875+0000","last_fullsized":"2024-09-15T21:34:03.491875+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20827,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347113+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:34:04.347113+0000","last_peered":"2024-09-15T21:34:04.347113+0000","last_clean":"2024-09-15T21:34:04.347113+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:34:04.347113+0000","last_undegraded":"2024-09-15T21:34:04.347113+0000","last_fullsized":"2024-09-15T21:34:04.347113+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357386+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:33:38.357386+0000","last_peered":"2024-09-15T21:33:38.357386+0000","last_clean":"2024-09-15T21:33:38.357386+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:33:38.357386+0000","last_undegraded":"2024-09-15T21:33:38.357386+0000","last_fullsized":"2024-09-15T21:33:38.357386+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347155+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:34:04.347155+0000","last_peered":"2024-09-15T21:34:04.347155+0000","last_clean":"2024-09-15T21:34:04.347155+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:34:04.347155+0000","last_undegraded":"2024-09-15T21:34:04.347155+0000","last_fullsized":"2024-09-15T21:34:04.347155+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20686,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491487+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:34:03.491487+0000","last_peered":"2024-09-15T21:34:03.491487+0000","last_clean":"2024-09-15T21:34:03.491487+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:34:03.491487+0000","last_undegraded":"2024-09-15T21:34:03.491487+0000","last_fullsized":"2024-09-15T21:34:03.491487+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":27652,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-15T21:33:59.596037+0000","last_change":"2024-09-15T21:33:36.869763+0000","last_active":"2024-09-15T21:33:59.596037+0000","last_peered":"2024-09-15T21:33:59.596037+0000","last_clean":"2024-09-15T21:21:12.333436+0000","last_became_active":"2024-09-15T21:21:14.908243+0000","last_became_peered":"2024-09-15T21:21:14.908243+0000","last_unstale":"2024-09-15T21:33:59.596037+0000","last_undegraded":"2024-09-15T21:21:14.433971+0000","last_fullsized":"2024-09-15T21:21:14.356277+0000","mapping_epoch":42,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3418,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-658}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20842,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356019+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:33:38.356019+0000","last_peered":"2024-09-15T21:33:38.356019+0000","last_clean":"2024-09-15T21:33:38.356019+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:33:38.356019+0000","last_undegraded":"2024-09-15T21:33:38.356019+0000","last_fullsized":"2024-09-15T21:33:38.356019+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21045,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491479+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:34:03.491479+0000","last_peered":"2024-09-15T21:34:03.491479+0000","last_clean":"2024-09-15T21:34:03.491479+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:34:03.491479+0000","last_undegraded":"2024-09-15T21:34:03.491479+0000","last_fullsized":"2024-09-15T21:34:03.491479+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356102+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:33:38.356102+0000","last_peered":"2024-09-15T21:33:38.356102+0000","last_clean":"2024-09-15T21:33:38.356102+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:33:38.356102+0000","last_undegraded":"2024-09-15T21:33:38.356102+0000","last_fullsized":"2024-09-15T21:33:38.356102+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20688,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356044+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:33:38.356044+0000","last_peered":"2024-09-15T21:33:38.356044+0000","last_clean":"2024-09-15T21:33:38.356044+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:33:38.356044+0000","last_undegraded":"2024-09-15T21:33:38.356044+0000","last_fullsized":"2024-09-15T21:33:38.356044+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346961+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:34:04.346961+0000","last_peered":"2024-09-15T21:34:04.346961+0000","last_clean":"2024-09-15T21:34:04.346961+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:34:04.346961+0000","last_undegraded":"2024-09-15T21:34:04.346961+0000","last_fullsized":"2024-09-15T21:34:04.346961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28713,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491776+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:34:03.491776+0000","last_peered":"2024-09-15T21:34:03.491776+0000","last_clean":"2024-09-15T21:34:03.491776+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:34:03.491776+0000","last_undegraded":"2024-09-15T21:34:03.491776+0000","last_fullsized":"2024-09-15T21:34:03.491776+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491687+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:34:03.491687+0000","last_peered":"2024-09-15T21:34:03.491687+0000","last_clean":"2024-09-15T21:34:03.491687+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:34:03.491687+0000","last_undegraded":"2024-09-15T21:34:03.491687+0000","last_fullsized":"2024-09-15T21:34:03.491687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":29000,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347029+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:34:04.347029+0000","last_peered":"2024-09-15T21:34:04.347029+0000","last_clean":"2024-09-15T21:34:04.347029+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:34:04.347029+0000","last_undegraded":"2024-09-15T21:34:04.347029+0000","last_fullsized":"2024-09-15T21:34:04.347029+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.355993+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:33:38.355993+0000","last_peered":"2024-09-15T21:33:38.355993+0000","last_clean":"2024-09-15T21:33:38.355993+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:33:38.355993+0000","last_undegraded":"2024-09-15T21:33:38.355993+0000","last_fullsized":"2024-09-15T21:33:38.355993+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29067,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346908+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:34:04.346908+0000","last_peered":"2024-09-15T21:34:04.346908+0000","last_clean":"2024-09-15T21:34:04.346908+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:34:04.346908+0000","last_undegraded":"2024-09-15T21:34:04.346908+0000","last_fullsized":"2024-09-15T21:34:04.346908+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20861,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357606+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:33:38.357606+0000","last_peered":"2024-09-15T21:33:38.357606+0000","last_clean":"2024-09-15T21:33:38.357606+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:33:38.357606+0000","last_undegraded":"2024-09-15T21:33:38.357606+0000","last_fullsized":"2024-09-15T21:33:38.357606+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.356187+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:33:38.356187+0000","last_peered":"2024-09-15T21:33:38.356187+0000","last_clean":"2024-09-15T21:33:38.356187+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:33:38.356187+0000","last_undegraded":"2024-09-15T21:33:38.356187+0000","last_fullsized":"2024-09-15T21:33:38.356187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21626,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357627+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:33:38.357627+0000","last_peered":"2024-09-15T21:33:38.357627+0000","last_clean":"2024-09-15T21:33:38.357627+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:33:38.357627+0000","last_undegraded":"2024-09-15T21:33:38.357627+0000","last_fullsized":"2024-09-15T21:33:38.357627+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347268+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:34:04.347268+0000","last_peered":"2024-09-15T21:34:04.347268+0000","last_clean":"2024-09-15T21:34:04.347268+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:34:04.347268+0000","last_undegraded":"2024-09-15T21:34:04.347268+0000","last_fullsized":"2024-09-15T21:34:04.347268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28742,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357687+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:33:38.357687+0000","last_peered":"2024-09-15T21:33:38.357687+0000","last_clean":"2024-09-15T21:33:38.357687+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:33:38.357687+0000","last_undegraded":"2024-09-15T21:33:38.357687+0000","last_fullsized":"2024-09-15T21:33:38.357687+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357583+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:33:38.357583+0000","last_peered":"2024-09-15T21:33:38.357583+0000","last_clean":"2024-09-15T21:33:38.357583+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:33:38.357583+0000","last_undegraded":"2024-09-15T21:33:38.357583+0000","last_fullsized":"2024-09-15T21:33:38.357583+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":118,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.357685+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:33:38.357685+0000","last_peered":"2024-09-15T21:33:38.357685+0000","last_clean":"2024-09-15T21:33:38.357685+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:33:38.357685+0000","last_undegraded":"2024-09-15T21:33:38.357685+0000","last_fullsized":"2024-09-15T21:33:38.357685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21027,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-15T21:33:38.440063+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:33:38.440063+0000","last_peered":"2024-09-15T21:33:38.440063+0000","last_clean":"2024-09-15T21:33:38.440063+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:33:38.440063+0000","last_undegraded":"2024-09-15T21:33:38.440063+0000","last_fullsized":"2024-09-15T21:33:38.440063+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":48815,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125779968,"data_stored":125693256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419512,"kb_used_data":58792,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93952328,"statfs":{"total":96636764160,"available":96207183872,"internally_reserved":0,"allocated":60203008,"data_stored":59838867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535004,"kb_used_data":66052,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836836,"statfs":{"total":96636764160,"available":96088920064,"internally_reserved":0,"allocated":67637248,"data_stored":67253125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541760,"kb_used_data":560,"kb_used_omap":4,"kb_used_meta":541179,"kb_avail":93830080,"statfs":{"total":96636764160,"available":96082001920,"internally_reserved":0,"allocated":573440,"data_stored":210267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":554167983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66584576,"data_stored":66548446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":59166720,"data_stored":59141429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:06.276 INFO:tasks.lost_unfound.ceph_manager:PG 2.a is not active+clean 2024-09-15T21:34:06.277 INFO:tasks.lost_unfound.ceph_manager:{'pgid': '2.a', 'version': "49'20635", 'reported_seq': 27652, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-15T21:33:59.596037+0000', 'last_change': '2024-09-15T21:33:36.869763+0000', 'last_active': '2024-09-15T21:33:59.596037+0000', 'last_peered': '2024-09-15T21:33:59.596037+0000', 'last_clean': '2024-09-15T21:21:12.333436+0000', 'last_became_active': '2024-09-15T21:21:14.908243+0000', 'last_became_peered': '2024-09-15T21:21:14.908243+0000', 'last_unstale': '2024-09-15T21:33:59.596037+0000', 'last_undegraded': '2024-09-15T21:21:14.433971+0000', 'last_fullsized': '2024-09-15T21:21:14.356277+0000', 'mapping_epoch': 42, 'log_start': "45'10600", 'ondisk_log_start': "45'10600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3368", 'last_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-15T21:19:34.719751+0000', 'last_clean_scrub_stamp': '2024-09-15T21:21:00.739021+0000', 'objects_scrubbed': 3320, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 968, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 483, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 659, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17311, 'num_write_kb': 27984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3418, 'num_bytes_recovered': 483, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -658}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-15T21:34:06.277 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-15T21:34:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:06.632 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 659/24 objects degraded (2745.833%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":898,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":3,"num_up_osds":3,"osd_up_since":1726435272,"num_in_osds":3,"osd_in_since":1726435265,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12,"data_bytes":4849,"bytes_used":1532186624,"bytes_avail":288378105856,"bytes_total":289910292480,"degraded_objects":659,"degraded_total":24,"degraded_ratio":27.458333333333329,"recovering_objects_per_sec":123,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":986,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-15T21:19:07:253728+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-15T21:22:10.704700+0000","services":{}},"progress_events":{}} 2024-09-15T21:34:06.644 INFO:tasks.lost_unfound.ceph_manager:making progress, resetting timeout 2024-09-15T21:34:06.645 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:06.928 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:06.939 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-09-15T21:34:06.918775+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49475,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500460,"kb_used_data":125420,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281615060,"statfs":{"total":289910292480,"available":288373821440,"internally_reserved":0,"allocated":128430080,"data_stored":127308532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1645,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1645,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001514"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29548,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491875+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:34:03.491875+0000","last_peered":"2024-09-15T21:34:03.491875+0000","last_clean":"2024-09-15T21:34:03.491875+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:34:03.491875+0000","last_undegraded":"2024-09-15T21:34:03.491875+0000","last_fullsized":"2024-09-15T21:34:03.491875+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20827,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347113+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:34:04.347113+0000","last_peered":"2024-09-15T21:34:04.347113+0000","last_clean":"2024-09-15T21:34:04.347113+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:34:04.347113+0000","last_undegraded":"2024-09-15T21:34:04.347113+0000","last_fullsized":"2024-09-15T21:34:04.347113+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490840+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:34:03.490840+0000","last_peered":"2024-09-15T21:34:03.490840+0000","last_clean":"2024-09-15T21:34:03.490840+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:34:03.490840+0000","last_undegraded":"2024-09-15T21:34:03.490840+0000","last_fullsized":"2024-09-15T21:34:03.490840+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347155+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:34:04.347155+0000","last_peered":"2024-09-15T21:34:04.347155+0000","last_clean":"2024-09-15T21:34:04.347155+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:34:04.347155+0000","last_undegraded":"2024-09-15T21:34:04.347155+0000","last_fullsized":"2024-09-15T21:34:04.347155+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20686,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491487+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:34:03.491487+0000","last_peered":"2024-09-15T21:34:03.491487+0000","last_clean":"2024-09-15T21:34:03.491487+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:34:03.491487+0000","last_undegraded":"2024-09-15T21:34:03.491487+0000","last_fullsized":"2024-09-15T21:34:03.491487+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.565894+0000","last_change":"2024-09-15T21:34:03.565354+0000","last_active":"2024-09-15T21:34:03.565894+0000","last_peered":"2024-09-15T21:34:03.565894+0000","last_clean":"2024-09-15T21:34:03.565894+0000","last_became_active":"2024-09-15T21:34:03.492946+0000","last_became_peered":"2024-09-15T21:34:03.492946+0000","last_unstale":"2024-09-15T21:34:03.565894+0000","last_undegraded":"2024-09-15T21:34:03.565894+0000","last_fullsized":"2024-09-15T21:34:03.565894+0000","mapping_epoch":66,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:26.940164+0000","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20846,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490723+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:34:03.490723+0000","last_peered":"2024-09-15T21:34:03.490723+0000","last_clean":"2024-09-15T21:34:03.490723+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:34:03.490723+0000","last_undegraded":"2024-09-15T21:34:03.490723+0000","last_fullsized":"2024-09-15T21:34:03.490723+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21045,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491479+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:34:03.491479+0000","last_peered":"2024-09-15T21:34:03.491479+0000","last_clean":"2024-09-15T21:34:03.491479+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:34:03.491479+0000","last_undegraded":"2024-09-15T21:34:03.491479+0000","last_fullsized":"2024-09-15T21:34:03.491479+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490643+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:34:03.490643+0000","last_peered":"2024-09-15T21:34:03.490643+0000","last_clean":"2024-09-15T21:34:03.490643+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:34:03.490643+0000","last_undegraded":"2024-09-15T21:34:03.490643+0000","last_fullsized":"2024-09-15T21:34:03.490643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20692,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491207+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:34:03.491207+0000","last_peered":"2024-09-15T21:34:03.491207+0000","last_clean":"2024-09-15T21:34:03.491207+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:34:03.491207+0000","last_undegraded":"2024-09-15T21:34:03.491207+0000","last_fullsized":"2024-09-15T21:34:03.491207+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346961+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:34:04.346961+0000","last_peered":"2024-09-15T21:34:04.346961+0000","last_clean":"2024-09-15T21:34:04.346961+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:34:04.346961+0000","last_undegraded":"2024-09-15T21:34:04.346961+0000","last_fullsized":"2024-09-15T21:34:04.346961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28713,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491776+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:34:03.491776+0000","last_peered":"2024-09-15T21:34:03.491776+0000","last_clean":"2024-09-15T21:34:03.491776+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:34:03.491776+0000","last_undegraded":"2024-09-15T21:34:03.491776+0000","last_fullsized":"2024-09-15T21:34:03.491776+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491687+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:34:03.491687+0000","last_peered":"2024-09-15T21:34:03.491687+0000","last_clean":"2024-09-15T21:34:03.491687+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:34:03.491687+0000","last_undegraded":"2024-09-15T21:34:03.491687+0000","last_fullsized":"2024-09-15T21:34:03.491687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":29000,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347029+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:34:04.347029+0000","last_peered":"2024-09-15T21:34:04.347029+0000","last_clean":"2024-09-15T21:34:04.347029+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:34:04.347029+0000","last_undegraded":"2024-09-15T21:34:04.347029+0000","last_fullsized":"2024-09-15T21:34:04.347029+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491109+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:34:03.491109+0000","last_peered":"2024-09-15T21:34:03.491109+0000","last_clean":"2024-09-15T21:34:03.491109+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:34:03.491109+0000","last_undegraded":"2024-09-15T21:34:03.491109+0000","last_fullsized":"2024-09-15T21:34:03.491109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29067,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346908+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:34:04.346908+0000","last_peered":"2024-09-15T21:34:04.346908+0000","last_clean":"2024-09-15T21:34:04.346908+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:34:04.346908+0000","last_undegraded":"2024-09-15T21:34:04.346908+0000","last_fullsized":"2024-09-15T21:34:04.346908+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20865,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491007+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:34:03.491007+0000","last_peered":"2024-09-15T21:34:03.491007+0000","last_clean":"2024-09-15T21:34:03.491007+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:34:03.491007+0000","last_undegraded":"2024-09-15T21:34:03.491007+0000","last_fullsized":"2024-09-15T21:34:03.491007+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491261+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:34:03.491261+0000","last_peered":"2024-09-15T21:34:03.491261+0000","last_clean":"2024-09-15T21:34:03.491261+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:34:03.491261+0000","last_undegraded":"2024-09-15T21:34:03.491261+0000","last_fullsized":"2024-09-15T21:34:03.491261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21630,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491119+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:34:03.491119+0000","last_peered":"2024-09-15T21:34:03.491119+0000","last_clean":"2024-09-15T21:34:03.491119+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:34:03.491119+0000","last_undegraded":"2024-09-15T21:34:03.491119+0000","last_fullsized":"2024-09-15T21:34:03.491119+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347268+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:34:04.347268+0000","last_peered":"2024-09-15T21:34:04.347268+0000","last_clean":"2024-09-15T21:34:04.347268+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:34:04.347268+0000","last_undegraded":"2024-09-15T21:34:04.347268+0000","last_fullsized":"2024-09-15T21:34:04.347268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28746,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490956+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:34:03.490956+0000","last_peered":"2024-09-15T21:34:03.490956+0000","last_clean":"2024-09-15T21:34:03.490956+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:34:03.490956+0000","last_undegraded":"2024-09-15T21:34:03.490956+0000","last_fullsized":"2024-09-15T21:34:03.490956+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490974+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:34:03.490974+0000","last_peered":"2024-09-15T21:34:03.490974+0000","last_clean":"2024-09-15T21:34:03.490974+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:34:03.490974+0000","last_undegraded":"2024-09-15T21:34:03.490974+0000","last_fullsized":"2024-09-15T21:34:03.490974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21031,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49474,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125779968,"data_stored":125693256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419512,"kb_used_data":58792,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93952328,"statfs":{"total":96636764160,"available":96207183872,"internally_reserved":0,"allocated":60203008,"data_stored":59838867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535004,"kb_used_data":66052,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836836,"statfs":{"total":96636764160,"available":96088920064,"internally_reserved":0,"allocated":67637248,"data_stored":67253125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545944,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825896,"statfs":{"total":96636764160,"available":96077717504,"internally_reserved":0,"allocated":589824,"data_stored":216540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66584576,"data_stored":66548446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":59166720,"data_stored":59141429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:09.942 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:10.238 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:10.250 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-09-15T21:34:08.919154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49475,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500460,"kb_used_data":125420,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281615060,"statfs":{"total":289910292480,"available":288373821440,"internally_reserved":0,"allocated":128430080,"data_stored":127308532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":659,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001621"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29548,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491875+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:34:03.491875+0000","last_peered":"2024-09-15T21:34:03.491875+0000","last_clean":"2024-09-15T21:34:03.491875+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:34:03.491875+0000","last_undegraded":"2024-09-15T21:34:03.491875+0000","last_fullsized":"2024-09-15T21:34:03.491875+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20827,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347113+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:34:04.347113+0000","last_peered":"2024-09-15T21:34:04.347113+0000","last_clean":"2024-09-15T21:34:04.347113+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:34:04.347113+0000","last_undegraded":"2024-09-15T21:34:04.347113+0000","last_fullsized":"2024-09-15T21:34:04.347113+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490840+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:34:03.490840+0000","last_peered":"2024-09-15T21:34:03.490840+0000","last_clean":"2024-09-15T21:34:03.490840+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:34:03.490840+0000","last_undegraded":"2024-09-15T21:34:03.490840+0000","last_fullsized":"2024-09-15T21:34:03.490840+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347155+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:34:04.347155+0000","last_peered":"2024-09-15T21:34:04.347155+0000","last_clean":"2024-09-15T21:34:04.347155+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:34:04.347155+0000","last_undegraded":"2024-09-15T21:34:04.347155+0000","last_fullsized":"2024-09-15T21:34:04.347155+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20686,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491487+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:34:03.491487+0000","last_peered":"2024-09-15T21:34:03.491487+0000","last_clean":"2024-09-15T21:34:03.491487+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:34:03.491487+0000","last_undegraded":"2024-09-15T21:34:03.491487+0000","last_fullsized":"2024-09-15T21:34:03.491487+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.565894+0000","last_change":"2024-09-15T21:34:03.565354+0000","last_active":"2024-09-15T21:34:03.565894+0000","last_peered":"2024-09-15T21:34:03.565894+0000","last_clean":"2024-09-15T21:34:03.565894+0000","last_became_active":"2024-09-15T21:34:03.492946+0000","last_became_peered":"2024-09-15T21:34:03.492946+0000","last_unstale":"2024-09-15T21:34:03.565894+0000","last_undegraded":"2024-09-15T21:34:03.565894+0000","last_fullsized":"2024-09-15T21:34:03.565894+0000","mapping_epoch":66,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:26.940164+0000","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20846,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490723+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:34:03.490723+0000","last_peered":"2024-09-15T21:34:03.490723+0000","last_clean":"2024-09-15T21:34:03.490723+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:34:03.490723+0000","last_undegraded":"2024-09-15T21:34:03.490723+0000","last_fullsized":"2024-09-15T21:34:03.490723+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21045,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491479+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:34:03.491479+0000","last_peered":"2024-09-15T21:34:03.491479+0000","last_clean":"2024-09-15T21:34:03.491479+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:34:03.491479+0000","last_undegraded":"2024-09-15T21:34:03.491479+0000","last_fullsized":"2024-09-15T21:34:03.491479+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490643+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:34:03.490643+0000","last_peered":"2024-09-15T21:34:03.490643+0000","last_clean":"2024-09-15T21:34:03.490643+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:34:03.490643+0000","last_undegraded":"2024-09-15T21:34:03.490643+0000","last_fullsized":"2024-09-15T21:34:03.490643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20692,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491207+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:34:03.491207+0000","last_peered":"2024-09-15T21:34:03.491207+0000","last_clean":"2024-09-15T21:34:03.491207+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:34:03.491207+0000","last_undegraded":"2024-09-15T21:34:03.491207+0000","last_fullsized":"2024-09-15T21:34:03.491207+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346961+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:34:04.346961+0000","last_peered":"2024-09-15T21:34:04.346961+0000","last_clean":"2024-09-15T21:34:04.346961+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:34:04.346961+0000","last_undegraded":"2024-09-15T21:34:04.346961+0000","last_fullsized":"2024-09-15T21:34:04.346961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28713,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491776+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:34:03.491776+0000","last_peered":"2024-09-15T21:34:03.491776+0000","last_clean":"2024-09-15T21:34:03.491776+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:34:03.491776+0000","last_undegraded":"2024-09-15T21:34:03.491776+0000","last_fullsized":"2024-09-15T21:34:03.491776+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491687+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:34:03.491687+0000","last_peered":"2024-09-15T21:34:03.491687+0000","last_clean":"2024-09-15T21:34:03.491687+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:34:03.491687+0000","last_undegraded":"2024-09-15T21:34:03.491687+0000","last_fullsized":"2024-09-15T21:34:03.491687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":29000,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347029+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:34:04.347029+0000","last_peered":"2024-09-15T21:34:04.347029+0000","last_clean":"2024-09-15T21:34:04.347029+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:34:04.347029+0000","last_undegraded":"2024-09-15T21:34:04.347029+0000","last_fullsized":"2024-09-15T21:34:04.347029+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491109+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:34:03.491109+0000","last_peered":"2024-09-15T21:34:03.491109+0000","last_clean":"2024-09-15T21:34:03.491109+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:34:03.491109+0000","last_undegraded":"2024-09-15T21:34:03.491109+0000","last_fullsized":"2024-09-15T21:34:03.491109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29067,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346908+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:34:04.346908+0000","last_peered":"2024-09-15T21:34:04.346908+0000","last_clean":"2024-09-15T21:34:04.346908+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:34:04.346908+0000","last_undegraded":"2024-09-15T21:34:04.346908+0000","last_fullsized":"2024-09-15T21:34:04.346908+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20865,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491007+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:34:03.491007+0000","last_peered":"2024-09-15T21:34:03.491007+0000","last_clean":"2024-09-15T21:34:03.491007+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:34:03.491007+0000","last_undegraded":"2024-09-15T21:34:03.491007+0000","last_fullsized":"2024-09-15T21:34:03.491007+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491261+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:34:03.491261+0000","last_peered":"2024-09-15T21:34:03.491261+0000","last_clean":"2024-09-15T21:34:03.491261+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:34:03.491261+0000","last_undegraded":"2024-09-15T21:34:03.491261+0000","last_fullsized":"2024-09-15T21:34:03.491261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21630,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491119+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:34:03.491119+0000","last_peered":"2024-09-15T21:34:03.491119+0000","last_clean":"2024-09-15T21:34:03.491119+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:34:03.491119+0000","last_undegraded":"2024-09-15T21:34:03.491119+0000","last_fullsized":"2024-09-15T21:34:03.491119+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347268+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:34:04.347268+0000","last_peered":"2024-09-15T21:34:04.347268+0000","last_clean":"2024-09-15T21:34:04.347268+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:34:04.347268+0000","last_undegraded":"2024-09-15T21:34:04.347268+0000","last_fullsized":"2024-09-15T21:34:04.347268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28746,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490956+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:34:03.490956+0000","last_peered":"2024-09-15T21:34:03.490956+0000","last_clean":"2024-09-15T21:34:03.490956+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:34:03.490956+0000","last_undegraded":"2024-09-15T21:34:03.490956+0000","last_fullsized":"2024-09-15T21:34:03.490956+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490974+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:34:03.490974+0000","last_peered":"2024-09-15T21:34:03.490974+0000","last_clean":"2024-09-15T21:34:03.490974+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:34:03.490974+0000","last_undegraded":"2024-09-15T21:34:03.490974+0000","last_fullsized":"2024-09-15T21:34:03.490974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21031,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49474,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125779968,"data_stored":125693256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419512,"kb_used_data":58792,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93952328,"statfs":{"total":96636764160,"available":96207183872,"internally_reserved":0,"allocated":60203008,"data_stored":59838867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535004,"kb_used_data":66052,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836836,"statfs":{"total":96636764160,"available":96088920064,"internally_reserved":0,"allocated":67637248,"data_stored":67253125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545944,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825896,"statfs":{"total":96636764160,"available":96077717504,"internally_reserved":0,"allocated":589824,"data_stored":216540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66584576,"data_stored":66548446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":59166720,"data_stored":59141429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:10.251 INFO:tasks.lost_unfound.ceph_manager:clean! 2024-09-15T21:34:10.252 INFO:tasks.lost_unfound.ceph_manager:waiting for clean 2024-09-15T21:34:10.252 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:10.536 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:10.537 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:10.548 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-09-15T21:34:08.919154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49475,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500460,"kb_used_data":125420,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281615060,"statfs":{"total":289910292480,"available":288373821440,"internally_reserved":0,"allocated":128430080,"data_stored":127308532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":659,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001621"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29548,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491875+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:34:03.491875+0000","last_peered":"2024-09-15T21:34:03.491875+0000","last_clean":"2024-09-15T21:34:03.491875+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:34:03.491875+0000","last_undegraded":"2024-09-15T21:34:03.491875+0000","last_fullsized":"2024-09-15T21:34:03.491875+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20827,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347113+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:34:04.347113+0000","last_peered":"2024-09-15T21:34:04.347113+0000","last_clean":"2024-09-15T21:34:04.347113+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:34:04.347113+0000","last_undegraded":"2024-09-15T21:34:04.347113+0000","last_fullsized":"2024-09-15T21:34:04.347113+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490840+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:34:03.490840+0000","last_peered":"2024-09-15T21:34:03.490840+0000","last_clean":"2024-09-15T21:34:03.490840+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:34:03.490840+0000","last_undegraded":"2024-09-15T21:34:03.490840+0000","last_fullsized":"2024-09-15T21:34:03.490840+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347155+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:34:04.347155+0000","last_peered":"2024-09-15T21:34:04.347155+0000","last_clean":"2024-09-15T21:34:04.347155+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:34:04.347155+0000","last_undegraded":"2024-09-15T21:34:04.347155+0000","last_fullsized":"2024-09-15T21:34:04.347155+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20686,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491487+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:34:03.491487+0000","last_peered":"2024-09-15T21:34:03.491487+0000","last_clean":"2024-09-15T21:34:03.491487+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:34:03.491487+0000","last_undegraded":"2024-09-15T21:34:03.491487+0000","last_fullsized":"2024-09-15T21:34:03.491487+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.565894+0000","last_change":"2024-09-15T21:34:03.565354+0000","last_active":"2024-09-15T21:34:03.565894+0000","last_peered":"2024-09-15T21:34:03.565894+0000","last_clean":"2024-09-15T21:34:03.565894+0000","last_became_active":"2024-09-15T21:34:03.492946+0000","last_became_peered":"2024-09-15T21:34:03.492946+0000","last_unstale":"2024-09-15T21:34:03.565894+0000","last_undegraded":"2024-09-15T21:34:03.565894+0000","last_fullsized":"2024-09-15T21:34:03.565894+0000","mapping_epoch":66,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:26.940164+0000","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20846,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490723+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:34:03.490723+0000","last_peered":"2024-09-15T21:34:03.490723+0000","last_clean":"2024-09-15T21:34:03.490723+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:34:03.490723+0000","last_undegraded":"2024-09-15T21:34:03.490723+0000","last_fullsized":"2024-09-15T21:34:03.490723+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21045,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491479+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:34:03.491479+0000","last_peered":"2024-09-15T21:34:03.491479+0000","last_clean":"2024-09-15T21:34:03.491479+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:34:03.491479+0000","last_undegraded":"2024-09-15T21:34:03.491479+0000","last_fullsized":"2024-09-15T21:34:03.491479+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490643+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:34:03.490643+0000","last_peered":"2024-09-15T21:34:03.490643+0000","last_clean":"2024-09-15T21:34:03.490643+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:34:03.490643+0000","last_undegraded":"2024-09-15T21:34:03.490643+0000","last_fullsized":"2024-09-15T21:34:03.490643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20692,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491207+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:34:03.491207+0000","last_peered":"2024-09-15T21:34:03.491207+0000","last_clean":"2024-09-15T21:34:03.491207+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:34:03.491207+0000","last_undegraded":"2024-09-15T21:34:03.491207+0000","last_fullsized":"2024-09-15T21:34:03.491207+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346961+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:34:04.346961+0000","last_peered":"2024-09-15T21:34:04.346961+0000","last_clean":"2024-09-15T21:34:04.346961+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:34:04.346961+0000","last_undegraded":"2024-09-15T21:34:04.346961+0000","last_fullsized":"2024-09-15T21:34:04.346961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28713,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491776+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:34:03.491776+0000","last_peered":"2024-09-15T21:34:03.491776+0000","last_clean":"2024-09-15T21:34:03.491776+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:34:03.491776+0000","last_undegraded":"2024-09-15T21:34:03.491776+0000","last_fullsized":"2024-09-15T21:34:03.491776+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491687+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:34:03.491687+0000","last_peered":"2024-09-15T21:34:03.491687+0000","last_clean":"2024-09-15T21:34:03.491687+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:34:03.491687+0000","last_undegraded":"2024-09-15T21:34:03.491687+0000","last_fullsized":"2024-09-15T21:34:03.491687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":29000,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347029+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:34:04.347029+0000","last_peered":"2024-09-15T21:34:04.347029+0000","last_clean":"2024-09-15T21:34:04.347029+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:34:04.347029+0000","last_undegraded":"2024-09-15T21:34:04.347029+0000","last_fullsized":"2024-09-15T21:34:04.347029+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491109+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:34:03.491109+0000","last_peered":"2024-09-15T21:34:03.491109+0000","last_clean":"2024-09-15T21:34:03.491109+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:34:03.491109+0000","last_undegraded":"2024-09-15T21:34:03.491109+0000","last_fullsized":"2024-09-15T21:34:03.491109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29067,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346908+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:34:04.346908+0000","last_peered":"2024-09-15T21:34:04.346908+0000","last_clean":"2024-09-15T21:34:04.346908+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:34:04.346908+0000","last_undegraded":"2024-09-15T21:34:04.346908+0000","last_fullsized":"2024-09-15T21:34:04.346908+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20865,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491007+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:34:03.491007+0000","last_peered":"2024-09-15T21:34:03.491007+0000","last_clean":"2024-09-15T21:34:03.491007+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:34:03.491007+0000","last_undegraded":"2024-09-15T21:34:03.491007+0000","last_fullsized":"2024-09-15T21:34:03.491007+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491261+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:34:03.491261+0000","last_peered":"2024-09-15T21:34:03.491261+0000","last_clean":"2024-09-15T21:34:03.491261+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:34:03.491261+0000","last_undegraded":"2024-09-15T21:34:03.491261+0000","last_fullsized":"2024-09-15T21:34:03.491261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21630,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491119+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:34:03.491119+0000","last_peered":"2024-09-15T21:34:03.491119+0000","last_clean":"2024-09-15T21:34:03.491119+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:34:03.491119+0000","last_undegraded":"2024-09-15T21:34:03.491119+0000","last_fullsized":"2024-09-15T21:34:03.491119+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347268+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:34:04.347268+0000","last_peered":"2024-09-15T21:34:04.347268+0000","last_clean":"2024-09-15T21:34:04.347268+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:34:04.347268+0000","last_undegraded":"2024-09-15T21:34:04.347268+0000","last_fullsized":"2024-09-15T21:34:04.347268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28746,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490956+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:34:03.490956+0000","last_peered":"2024-09-15T21:34:03.490956+0000","last_clean":"2024-09-15T21:34:03.490956+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:34:03.490956+0000","last_undegraded":"2024-09-15T21:34:03.490956+0000","last_fullsized":"2024-09-15T21:34:03.490956+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490974+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:34:03.490974+0000","last_peered":"2024-09-15T21:34:03.490974+0000","last_clean":"2024-09-15T21:34:03.490974+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:34:03.490974+0000","last_undegraded":"2024-09-15T21:34:03.490974+0000","last_fullsized":"2024-09-15T21:34:03.490974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21031,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49474,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125779968,"data_stored":125693256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419512,"kb_used_data":58792,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93952328,"statfs":{"total":96636764160,"available":96207183872,"internally_reserved":0,"allocated":60203008,"data_stored":59838867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535004,"kb_used_data":66052,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836836,"statfs":{"total":96636764160,"available":96088920064,"internally_reserved":0,"allocated":67637248,"data_stored":67253125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545944,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825896,"statfs":{"total":96636764160,"available":96077717504,"internally_reserved":0,"allocated":589824,"data_stored":216540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66584576,"data_stored":66548446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":59166720,"data_stored":59141429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:10.550 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:10.834 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:10.834 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:10.846 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":510,"stamp":"2024-09-15T21:34:08.919154+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49475,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500460,"kb_used_data":125420,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281615060,"statfs":{"total":289910292480,"available":288373821440,"internally_reserved":0,"allocated":128430080,"data_stored":127308532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":659,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001621"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29548,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491875+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:34:03.491875+0000","last_peered":"2024-09-15T21:34:03.491875+0000","last_clean":"2024-09-15T21:34:03.491875+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:34:03.491875+0000","last_undegraded":"2024-09-15T21:34:03.491875+0000","last_fullsized":"2024-09-15T21:34:03.491875+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20827,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347113+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:34:04.347113+0000","last_peered":"2024-09-15T21:34:04.347113+0000","last_clean":"2024-09-15T21:34:04.347113+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:34:04.347113+0000","last_undegraded":"2024-09-15T21:34:04.347113+0000","last_fullsized":"2024-09-15T21:34:04.347113+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490840+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:34:03.490840+0000","last_peered":"2024-09-15T21:34:03.490840+0000","last_clean":"2024-09-15T21:34:03.490840+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:34:03.490840+0000","last_undegraded":"2024-09-15T21:34:03.490840+0000","last_fullsized":"2024-09-15T21:34:03.490840+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347155+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:34:04.347155+0000","last_peered":"2024-09-15T21:34:04.347155+0000","last_clean":"2024-09-15T21:34:04.347155+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:34:04.347155+0000","last_undegraded":"2024-09-15T21:34:04.347155+0000","last_fullsized":"2024-09-15T21:34:04.347155+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20686,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491487+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:34:03.491487+0000","last_peered":"2024-09-15T21:34:03.491487+0000","last_clean":"2024-09-15T21:34:03.491487+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:34:03.491487+0000","last_undegraded":"2024-09-15T21:34:03.491487+0000","last_fullsized":"2024-09-15T21:34:03.491487+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.565894+0000","last_change":"2024-09-15T21:34:03.565354+0000","last_active":"2024-09-15T21:34:03.565894+0000","last_peered":"2024-09-15T21:34:03.565894+0000","last_clean":"2024-09-15T21:34:03.565894+0000","last_became_active":"2024-09-15T21:34:03.492946+0000","last_became_peered":"2024-09-15T21:34:03.492946+0000","last_unstale":"2024-09-15T21:34:03.565894+0000","last_undegraded":"2024-09-15T21:34:03.565894+0000","last_fullsized":"2024-09-15T21:34:03.565894+0000","mapping_epoch":66,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:26.940164+0000","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20846,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490723+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:34:03.490723+0000","last_peered":"2024-09-15T21:34:03.490723+0000","last_clean":"2024-09-15T21:34:03.490723+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:34:03.490723+0000","last_undegraded":"2024-09-15T21:34:03.490723+0000","last_fullsized":"2024-09-15T21:34:03.490723+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21045,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491479+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:34:03.491479+0000","last_peered":"2024-09-15T21:34:03.491479+0000","last_clean":"2024-09-15T21:34:03.491479+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:34:03.491479+0000","last_undegraded":"2024-09-15T21:34:03.491479+0000","last_fullsized":"2024-09-15T21:34:03.491479+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490643+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:34:03.490643+0000","last_peered":"2024-09-15T21:34:03.490643+0000","last_clean":"2024-09-15T21:34:03.490643+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:34:03.490643+0000","last_undegraded":"2024-09-15T21:34:03.490643+0000","last_fullsized":"2024-09-15T21:34:03.490643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20692,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491207+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:34:03.491207+0000","last_peered":"2024-09-15T21:34:03.491207+0000","last_clean":"2024-09-15T21:34:03.491207+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:34:03.491207+0000","last_undegraded":"2024-09-15T21:34:03.491207+0000","last_fullsized":"2024-09-15T21:34:03.491207+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346961+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:34:04.346961+0000","last_peered":"2024-09-15T21:34:04.346961+0000","last_clean":"2024-09-15T21:34:04.346961+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:34:04.346961+0000","last_undegraded":"2024-09-15T21:34:04.346961+0000","last_fullsized":"2024-09-15T21:34:04.346961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28713,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491776+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:34:03.491776+0000","last_peered":"2024-09-15T21:34:03.491776+0000","last_clean":"2024-09-15T21:34:03.491776+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:34:03.491776+0000","last_undegraded":"2024-09-15T21:34:03.491776+0000","last_fullsized":"2024-09-15T21:34:03.491776+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491687+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:34:03.491687+0000","last_peered":"2024-09-15T21:34:03.491687+0000","last_clean":"2024-09-15T21:34:03.491687+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:34:03.491687+0000","last_undegraded":"2024-09-15T21:34:03.491687+0000","last_fullsized":"2024-09-15T21:34:03.491687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":29000,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347029+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:34:04.347029+0000","last_peered":"2024-09-15T21:34:04.347029+0000","last_clean":"2024-09-15T21:34:04.347029+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:34:04.347029+0000","last_undegraded":"2024-09-15T21:34:04.347029+0000","last_fullsized":"2024-09-15T21:34:04.347029+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491109+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:34:03.491109+0000","last_peered":"2024-09-15T21:34:03.491109+0000","last_clean":"2024-09-15T21:34:03.491109+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:34:03.491109+0000","last_undegraded":"2024-09-15T21:34:03.491109+0000","last_fullsized":"2024-09-15T21:34:03.491109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29067,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346908+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:34:04.346908+0000","last_peered":"2024-09-15T21:34:04.346908+0000","last_clean":"2024-09-15T21:34:04.346908+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:34:04.346908+0000","last_undegraded":"2024-09-15T21:34:04.346908+0000","last_fullsized":"2024-09-15T21:34:04.346908+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20865,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491007+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:34:03.491007+0000","last_peered":"2024-09-15T21:34:03.491007+0000","last_clean":"2024-09-15T21:34:03.491007+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:34:03.491007+0000","last_undegraded":"2024-09-15T21:34:03.491007+0000","last_fullsized":"2024-09-15T21:34:03.491007+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491261+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:34:03.491261+0000","last_peered":"2024-09-15T21:34:03.491261+0000","last_clean":"2024-09-15T21:34:03.491261+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:34:03.491261+0000","last_undegraded":"2024-09-15T21:34:03.491261+0000","last_fullsized":"2024-09-15T21:34:03.491261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21630,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491119+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:34:03.491119+0000","last_peered":"2024-09-15T21:34:03.491119+0000","last_clean":"2024-09-15T21:34:03.491119+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:34:03.491119+0000","last_undegraded":"2024-09-15T21:34:03.491119+0000","last_fullsized":"2024-09-15T21:34:03.491119+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347268+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:34:04.347268+0000","last_peered":"2024-09-15T21:34:04.347268+0000","last_clean":"2024-09-15T21:34:04.347268+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:34:04.347268+0000","last_undegraded":"2024-09-15T21:34:04.347268+0000","last_fullsized":"2024-09-15T21:34:04.347268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28746,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490956+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:34:03.490956+0000","last_peered":"2024-09-15T21:34:03.490956+0000","last_clean":"2024-09-15T21:34:03.490956+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:34:03.490956+0000","last_undegraded":"2024-09-15T21:34:03.490956+0000","last_fullsized":"2024-09-15T21:34:03.490956+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490974+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:34:03.490974+0000","last_peered":"2024-09-15T21:34:03.490974+0000","last_clean":"2024-09-15T21:34:03.490974+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:34:03.490974+0000","last_undegraded":"2024-09-15T21:34:03.490974+0000","last_fullsized":"2024-09-15T21:34:03.490974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21031,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49474,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125779968,"data_stored":125693256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419512,"kb_used_data":58792,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93952328,"statfs":{"total":96636764160,"available":96207183872,"internally_reserved":0,"allocated":60203008,"data_stored":59838867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535004,"kb_used_data":66052,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93836836,"statfs":{"total":96636764160,"available":96088920064,"internally_reserved":0,"allocated":67637248,"data_stored":67253125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545944,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825896,"statfs":{"total":96636764160,"available":96077717504,"internally_reserved":0,"allocated":589824,"data_stored":216540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66584576,"data_stored":66548446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":59166720,"data_stored":59141429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:10.848 INFO:tasks.lost_unfound.ceph_manager:clean! 2024-09-15T21:34:10.848 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-15T21:34:10.862 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-15T21:34:10.862 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:11.144 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:11.145 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:11.157 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-09-15T21:34:10.919454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49475,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493144,"kb_used_data":118112,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281622376,"statfs":{"total":289910292480,"available":288381313024,"internally_reserved":0,"allocated":120946688,"data_stored":119820074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":659,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.432979"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29548,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491875+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:34:03.491875+0000","last_peered":"2024-09-15T21:34:03.491875+0000","last_clean":"2024-09-15T21:34:03.491875+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:34:03.491875+0000","last_undegraded":"2024-09-15T21:34:03.491875+0000","last_fullsized":"2024-09-15T21:34:03.491875+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20827,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347113+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:34:04.347113+0000","last_peered":"2024-09-15T21:34:04.347113+0000","last_clean":"2024-09-15T21:34:04.347113+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:34:04.347113+0000","last_undegraded":"2024-09-15T21:34:04.347113+0000","last_fullsized":"2024-09-15T21:34:04.347113+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490840+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:34:03.490840+0000","last_peered":"2024-09-15T21:34:03.490840+0000","last_clean":"2024-09-15T21:34:03.490840+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:34:03.490840+0000","last_undegraded":"2024-09-15T21:34:03.490840+0000","last_fullsized":"2024-09-15T21:34:03.490840+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347155+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:34:04.347155+0000","last_peered":"2024-09-15T21:34:04.347155+0000","last_clean":"2024-09-15T21:34:04.347155+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:34:04.347155+0000","last_undegraded":"2024-09-15T21:34:04.347155+0000","last_fullsized":"2024-09-15T21:34:04.347155+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20686,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491487+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:34:03.491487+0000","last_peered":"2024-09-15T21:34:03.491487+0000","last_clean":"2024-09-15T21:34:03.491487+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:34:03.491487+0000","last_undegraded":"2024-09-15T21:34:03.491487+0000","last_fullsized":"2024-09-15T21:34:03.491487+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.565894+0000","last_change":"2024-09-15T21:34:03.565354+0000","last_active":"2024-09-15T21:34:03.565894+0000","last_peered":"2024-09-15T21:34:03.565894+0000","last_clean":"2024-09-15T21:34:03.565894+0000","last_became_active":"2024-09-15T21:34:03.492946+0000","last_became_peered":"2024-09-15T21:34:03.492946+0000","last_unstale":"2024-09-15T21:34:03.565894+0000","last_undegraded":"2024-09-15T21:34:03.565894+0000","last_fullsized":"2024-09-15T21:34:03.565894+0000","mapping_epoch":66,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:26.940164+0000","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20846,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490723+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:34:03.490723+0000","last_peered":"2024-09-15T21:34:03.490723+0000","last_clean":"2024-09-15T21:34:03.490723+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:34:03.490723+0000","last_undegraded":"2024-09-15T21:34:03.490723+0000","last_fullsized":"2024-09-15T21:34:03.490723+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21045,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491479+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:34:03.491479+0000","last_peered":"2024-09-15T21:34:03.491479+0000","last_clean":"2024-09-15T21:34:03.491479+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:34:03.491479+0000","last_undegraded":"2024-09-15T21:34:03.491479+0000","last_fullsized":"2024-09-15T21:34:03.491479+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490643+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:34:03.490643+0000","last_peered":"2024-09-15T21:34:03.490643+0000","last_clean":"2024-09-15T21:34:03.490643+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:34:03.490643+0000","last_undegraded":"2024-09-15T21:34:03.490643+0000","last_fullsized":"2024-09-15T21:34:03.490643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20692,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491207+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:34:03.491207+0000","last_peered":"2024-09-15T21:34:03.491207+0000","last_clean":"2024-09-15T21:34:03.491207+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:34:03.491207+0000","last_undegraded":"2024-09-15T21:34:03.491207+0000","last_fullsized":"2024-09-15T21:34:03.491207+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346961+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:34:04.346961+0000","last_peered":"2024-09-15T21:34:04.346961+0000","last_clean":"2024-09-15T21:34:04.346961+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:34:04.346961+0000","last_undegraded":"2024-09-15T21:34:04.346961+0000","last_fullsized":"2024-09-15T21:34:04.346961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28713,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491776+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:34:03.491776+0000","last_peered":"2024-09-15T21:34:03.491776+0000","last_clean":"2024-09-15T21:34:03.491776+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:34:03.491776+0000","last_undegraded":"2024-09-15T21:34:03.491776+0000","last_fullsized":"2024-09-15T21:34:03.491776+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491687+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:34:03.491687+0000","last_peered":"2024-09-15T21:34:03.491687+0000","last_clean":"2024-09-15T21:34:03.491687+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:34:03.491687+0000","last_undegraded":"2024-09-15T21:34:03.491687+0000","last_fullsized":"2024-09-15T21:34:03.491687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":29000,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347029+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:34:04.347029+0000","last_peered":"2024-09-15T21:34:04.347029+0000","last_clean":"2024-09-15T21:34:04.347029+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:34:04.347029+0000","last_undegraded":"2024-09-15T21:34:04.347029+0000","last_fullsized":"2024-09-15T21:34:04.347029+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491109+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:34:03.491109+0000","last_peered":"2024-09-15T21:34:03.491109+0000","last_clean":"2024-09-15T21:34:03.491109+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:34:03.491109+0000","last_undegraded":"2024-09-15T21:34:03.491109+0000","last_fullsized":"2024-09-15T21:34:03.491109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29067,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346908+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:34:04.346908+0000","last_peered":"2024-09-15T21:34:04.346908+0000","last_clean":"2024-09-15T21:34:04.346908+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:34:04.346908+0000","last_undegraded":"2024-09-15T21:34:04.346908+0000","last_fullsized":"2024-09-15T21:34:04.346908+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20865,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491007+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:34:03.491007+0000","last_peered":"2024-09-15T21:34:03.491007+0000","last_clean":"2024-09-15T21:34:03.491007+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:34:03.491007+0000","last_undegraded":"2024-09-15T21:34:03.491007+0000","last_fullsized":"2024-09-15T21:34:03.491007+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491261+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:34:03.491261+0000","last_peered":"2024-09-15T21:34:03.491261+0000","last_clean":"2024-09-15T21:34:03.491261+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:34:03.491261+0000","last_undegraded":"2024-09-15T21:34:03.491261+0000","last_fullsized":"2024-09-15T21:34:03.491261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21630,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491119+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:34:03.491119+0000","last_peered":"2024-09-15T21:34:03.491119+0000","last_clean":"2024-09-15T21:34:03.491119+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:34:03.491119+0000","last_undegraded":"2024-09-15T21:34:03.491119+0000","last_fullsized":"2024-09-15T21:34:03.491119+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347268+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:34:04.347268+0000","last_peered":"2024-09-15T21:34:04.347268+0000","last_clean":"2024-09-15T21:34:04.347268+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:34:04.347268+0000","last_undegraded":"2024-09-15T21:34:04.347268+0000","last_fullsized":"2024-09-15T21:34:04.347268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28746,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490956+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:34:03.490956+0000","last_peered":"2024-09-15T21:34:03.490956+0000","last_clean":"2024-09-15T21:34:03.490956+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:34:03.490956+0000","last_undegraded":"2024-09-15T21:34:03.490956+0000","last_fullsized":"2024-09-15T21:34:03.490956+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490974+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:34:03.490974+0000","last_peered":"2024-09-15T21:34:03.490974+0000","last_clean":"2024-09-15T21:34:03.490974+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:34:03.490974+0000","last_undegraded":"2024-09-15T21:34:03.490974+0000","last_fullsized":"2024-09-15T21:34:03.490974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21031,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49474,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118407168,"data_stored":118320456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414604,"kb_used_data":53884,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93957236,"statfs":{"total":96636764160,"available":96212209664,"internally_reserved":0,"allocated":55177216,"data_stored":54808009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532596,"kb_used_data":63652,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93839244,"statfs":{"total":96636764160,"available":96091385856,"internally_reserved":0,"allocated":65179648,"data_stored":64795525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545944,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825896,"statfs":{"total":96636764160,"available":96077717504,"internally_reserved":0,"allocated":589824,"data_stored":216540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64126976,"data_stored":64090846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54251520,"data_stored":54226229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:11.158 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:11.442 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:11.443 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:11.455 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-09-15T21:34:10.919454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49475,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493144,"kb_used_data":118112,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281622376,"statfs":{"total":289910292480,"available":288381313024,"internally_reserved":0,"allocated":120946688,"data_stored":119820074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":659,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.432979"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29548,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491875+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:34:03.491875+0000","last_peered":"2024-09-15T21:34:03.491875+0000","last_clean":"2024-09-15T21:34:03.491875+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:34:03.491875+0000","last_undegraded":"2024-09-15T21:34:03.491875+0000","last_fullsized":"2024-09-15T21:34:03.491875+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20827,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347113+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:34:04.347113+0000","last_peered":"2024-09-15T21:34:04.347113+0000","last_clean":"2024-09-15T21:34:04.347113+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:34:04.347113+0000","last_undegraded":"2024-09-15T21:34:04.347113+0000","last_fullsized":"2024-09-15T21:34:04.347113+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490840+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:34:03.490840+0000","last_peered":"2024-09-15T21:34:03.490840+0000","last_clean":"2024-09-15T21:34:03.490840+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:34:03.490840+0000","last_undegraded":"2024-09-15T21:34:03.490840+0000","last_fullsized":"2024-09-15T21:34:03.490840+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347155+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:34:04.347155+0000","last_peered":"2024-09-15T21:34:04.347155+0000","last_clean":"2024-09-15T21:34:04.347155+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:34:04.347155+0000","last_undegraded":"2024-09-15T21:34:04.347155+0000","last_fullsized":"2024-09-15T21:34:04.347155+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20686,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491487+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:34:03.491487+0000","last_peered":"2024-09-15T21:34:03.491487+0000","last_clean":"2024-09-15T21:34:03.491487+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:34:03.491487+0000","last_undegraded":"2024-09-15T21:34:03.491487+0000","last_fullsized":"2024-09-15T21:34:03.491487+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.565894+0000","last_change":"2024-09-15T21:34:03.565354+0000","last_active":"2024-09-15T21:34:03.565894+0000","last_peered":"2024-09-15T21:34:03.565894+0000","last_clean":"2024-09-15T21:34:03.565894+0000","last_became_active":"2024-09-15T21:34:03.492946+0000","last_became_peered":"2024-09-15T21:34:03.492946+0000","last_unstale":"2024-09-15T21:34:03.565894+0000","last_undegraded":"2024-09-15T21:34:03.565894+0000","last_fullsized":"2024-09-15T21:34:03.565894+0000","mapping_epoch":66,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:26.940164+0000","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20846,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490723+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:34:03.490723+0000","last_peered":"2024-09-15T21:34:03.490723+0000","last_clean":"2024-09-15T21:34:03.490723+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:34:03.490723+0000","last_undegraded":"2024-09-15T21:34:03.490723+0000","last_fullsized":"2024-09-15T21:34:03.490723+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21045,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491479+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:34:03.491479+0000","last_peered":"2024-09-15T21:34:03.491479+0000","last_clean":"2024-09-15T21:34:03.491479+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:34:03.491479+0000","last_undegraded":"2024-09-15T21:34:03.491479+0000","last_fullsized":"2024-09-15T21:34:03.491479+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490643+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:34:03.490643+0000","last_peered":"2024-09-15T21:34:03.490643+0000","last_clean":"2024-09-15T21:34:03.490643+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:34:03.490643+0000","last_undegraded":"2024-09-15T21:34:03.490643+0000","last_fullsized":"2024-09-15T21:34:03.490643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20692,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491207+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:34:03.491207+0000","last_peered":"2024-09-15T21:34:03.491207+0000","last_clean":"2024-09-15T21:34:03.491207+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:34:03.491207+0000","last_undegraded":"2024-09-15T21:34:03.491207+0000","last_fullsized":"2024-09-15T21:34:03.491207+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346961+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:34:04.346961+0000","last_peered":"2024-09-15T21:34:04.346961+0000","last_clean":"2024-09-15T21:34:04.346961+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:34:04.346961+0000","last_undegraded":"2024-09-15T21:34:04.346961+0000","last_fullsized":"2024-09-15T21:34:04.346961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28713,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491776+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:34:03.491776+0000","last_peered":"2024-09-15T21:34:03.491776+0000","last_clean":"2024-09-15T21:34:03.491776+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:34:03.491776+0000","last_undegraded":"2024-09-15T21:34:03.491776+0000","last_fullsized":"2024-09-15T21:34:03.491776+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491687+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:34:03.491687+0000","last_peered":"2024-09-15T21:34:03.491687+0000","last_clean":"2024-09-15T21:34:03.491687+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:34:03.491687+0000","last_undegraded":"2024-09-15T21:34:03.491687+0000","last_fullsized":"2024-09-15T21:34:03.491687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":29000,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347029+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:34:04.347029+0000","last_peered":"2024-09-15T21:34:04.347029+0000","last_clean":"2024-09-15T21:34:04.347029+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:34:04.347029+0000","last_undegraded":"2024-09-15T21:34:04.347029+0000","last_fullsized":"2024-09-15T21:34:04.347029+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491109+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:34:03.491109+0000","last_peered":"2024-09-15T21:34:03.491109+0000","last_clean":"2024-09-15T21:34:03.491109+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:34:03.491109+0000","last_undegraded":"2024-09-15T21:34:03.491109+0000","last_fullsized":"2024-09-15T21:34:03.491109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29067,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346908+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:34:04.346908+0000","last_peered":"2024-09-15T21:34:04.346908+0000","last_clean":"2024-09-15T21:34:04.346908+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:34:04.346908+0000","last_undegraded":"2024-09-15T21:34:04.346908+0000","last_fullsized":"2024-09-15T21:34:04.346908+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20865,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491007+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:34:03.491007+0000","last_peered":"2024-09-15T21:34:03.491007+0000","last_clean":"2024-09-15T21:34:03.491007+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:34:03.491007+0000","last_undegraded":"2024-09-15T21:34:03.491007+0000","last_fullsized":"2024-09-15T21:34:03.491007+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491261+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:34:03.491261+0000","last_peered":"2024-09-15T21:34:03.491261+0000","last_clean":"2024-09-15T21:34:03.491261+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:34:03.491261+0000","last_undegraded":"2024-09-15T21:34:03.491261+0000","last_fullsized":"2024-09-15T21:34:03.491261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21630,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491119+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:34:03.491119+0000","last_peered":"2024-09-15T21:34:03.491119+0000","last_clean":"2024-09-15T21:34:03.491119+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:34:03.491119+0000","last_undegraded":"2024-09-15T21:34:03.491119+0000","last_fullsized":"2024-09-15T21:34:03.491119+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347268+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:34:04.347268+0000","last_peered":"2024-09-15T21:34:04.347268+0000","last_clean":"2024-09-15T21:34:04.347268+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:34:04.347268+0000","last_undegraded":"2024-09-15T21:34:04.347268+0000","last_fullsized":"2024-09-15T21:34:04.347268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28746,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490956+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:34:03.490956+0000","last_peered":"2024-09-15T21:34:03.490956+0000","last_clean":"2024-09-15T21:34:03.490956+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:34:03.490956+0000","last_undegraded":"2024-09-15T21:34:03.490956+0000","last_fullsized":"2024-09-15T21:34:03.490956+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490974+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:34:03.490974+0000","last_peered":"2024-09-15T21:34:03.490974+0000","last_clean":"2024-09-15T21:34:03.490974+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:34:03.490974+0000","last_undegraded":"2024-09-15T21:34:03.490974+0000","last_fullsized":"2024-09-15T21:34:03.490974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21031,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49474,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118407168,"data_stored":118320456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414604,"kb_used_data":53884,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93957236,"statfs":{"total":96636764160,"available":96212209664,"internally_reserved":0,"allocated":55177216,"data_stored":54808009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532596,"kb_used_data":63652,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93839244,"statfs":{"total":96636764160,"available":96091385856,"internally_reserved":0,"allocated":65179648,"data_stored":64795525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545944,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825896,"statfs":{"total":96636764160,"available":96077717504,"internally_reserved":0,"allocated":589824,"data_stored":216540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64126976,"data_stored":64090846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54251520,"data_stored":54226229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:11.456 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-15T21:34:11.456 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:11.745 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:11.746 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:11.757 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-09-15T21:34:10.919454+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49475,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493144,"kb_used_data":118112,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281622376,"statfs":{"total":289910292480,"available":288381313024,"internally_reserved":0,"allocated":120946688,"data_stored":119820074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":659,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.432979"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29548,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491875+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:34:03.491875+0000","last_peered":"2024-09-15T21:34:03.491875+0000","last_clean":"2024-09-15T21:34:03.491875+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:34:03.491875+0000","last_undegraded":"2024-09-15T21:34:03.491875+0000","last_fullsized":"2024-09-15T21:34:03.491875+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20827,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347113+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:34:04.347113+0000","last_peered":"2024-09-15T21:34:04.347113+0000","last_clean":"2024-09-15T21:34:04.347113+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:34:04.347113+0000","last_undegraded":"2024-09-15T21:34:04.347113+0000","last_fullsized":"2024-09-15T21:34:04.347113+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490840+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:34:03.490840+0000","last_peered":"2024-09-15T21:34:03.490840+0000","last_clean":"2024-09-15T21:34:03.490840+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:34:03.490840+0000","last_undegraded":"2024-09-15T21:34:03.490840+0000","last_fullsized":"2024-09-15T21:34:03.490840+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347155+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:34:04.347155+0000","last_peered":"2024-09-15T21:34:04.347155+0000","last_clean":"2024-09-15T21:34:04.347155+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:34:04.347155+0000","last_undegraded":"2024-09-15T21:34:04.347155+0000","last_fullsized":"2024-09-15T21:34:04.347155+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20686,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491487+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:34:03.491487+0000","last_peered":"2024-09-15T21:34:03.491487+0000","last_clean":"2024-09-15T21:34:03.491487+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:34:03.491487+0000","last_undegraded":"2024-09-15T21:34:03.491487+0000","last_fullsized":"2024-09-15T21:34:03.491487+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.565894+0000","last_change":"2024-09-15T21:34:03.565354+0000","last_active":"2024-09-15T21:34:03.565894+0000","last_peered":"2024-09-15T21:34:03.565894+0000","last_clean":"2024-09-15T21:34:03.565894+0000","last_became_active":"2024-09-15T21:34:03.492946+0000","last_became_peered":"2024-09-15T21:34:03.492946+0000","last_unstale":"2024-09-15T21:34:03.565894+0000","last_undegraded":"2024-09-15T21:34:03.565894+0000","last_fullsized":"2024-09-15T21:34:03.565894+0000","mapping_epoch":66,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:26.940164+0000","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20846,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490723+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:34:03.490723+0000","last_peered":"2024-09-15T21:34:03.490723+0000","last_clean":"2024-09-15T21:34:03.490723+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:34:03.490723+0000","last_undegraded":"2024-09-15T21:34:03.490723+0000","last_fullsized":"2024-09-15T21:34:03.490723+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21045,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491479+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:34:03.491479+0000","last_peered":"2024-09-15T21:34:03.491479+0000","last_clean":"2024-09-15T21:34:03.491479+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:34:03.491479+0000","last_undegraded":"2024-09-15T21:34:03.491479+0000","last_fullsized":"2024-09-15T21:34:03.491479+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490643+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:34:03.490643+0000","last_peered":"2024-09-15T21:34:03.490643+0000","last_clean":"2024-09-15T21:34:03.490643+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:34:03.490643+0000","last_undegraded":"2024-09-15T21:34:03.490643+0000","last_fullsized":"2024-09-15T21:34:03.490643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20692,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491207+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:34:03.491207+0000","last_peered":"2024-09-15T21:34:03.491207+0000","last_clean":"2024-09-15T21:34:03.491207+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:34:03.491207+0000","last_undegraded":"2024-09-15T21:34:03.491207+0000","last_fullsized":"2024-09-15T21:34:03.491207+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346961+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:34:04.346961+0000","last_peered":"2024-09-15T21:34:04.346961+0000","last_clean":"2024-09-15T21:34:04.346961+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:34:04.346961+0000","last_undegraded":"2024-09-15T21:34:04.346961+0000","last_fullsized":"2024-09-15T21:34:04.346961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28713,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491776+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:34:03.491776+0000","last_peered":"2024-09-15T21:34:03.491776+0000","last_clean":"2024-09-15T21:34:03.491776+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:34:03.491776+0000","last_undegraded":"2024-09-15T21:34:03.491776+0000","last_fullsized":"2024-09-15T21:34:03.491776+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491687+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:34:03.491687+0000","last_peered":"2024-09-15T21:34:03.491687+0000","last_clean":"2024-09-15T21:34:03.491687+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:34:03.491687+0000","last_undegraded":"2024-09-15T21:34:03.491687+0000","last_fullsized":"2024-09-15T21:34:03.491687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":29000,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347029+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:34:04.347029+0000","last_peered":"2024-09-15T21:34:04.347029+0000","last_clean":"2024-09-15T21:34:04.347029+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:34:04.347029+0000","last_undegraded":"2024-09-15T21:34:04.347029+0000","last_fullsized":"2024-09-15T21:34:04.347029+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491109+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:34:03.491109+0000","last_peered":"2024-09-15T21:34:03.491109+0000","last_clean":"2024-09-15T21:34:03.491109+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:34:03.491109+0000","last_undegraded":"2024-09-15T21:34:03.491109+0000","last_fullsized":"2024-09-15T21:34:03.491109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29067,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346908+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:34:04.346908+0000","last_peered":"2024-09-15T21:34:04.346908+0000","last_clean":"2024-09-15T21:34:04.346908+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:34:04.346908+0000","last_undegraded":"2024-09-15T21:34:04.346908+0000","last_fullsized":"2024-09-15T21:34:04.346908+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20865,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491007+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:34:03.491007+0000","last_peered":"2024-09-15T21:34:03.491007+0000","last_clean":"2024-09-15T21:34:03.491007+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:34:03.491007+0000","last_undegraded":"2024-09-15T21:34:03.491007+0000","last_fullsized":"2024-09-15T21:34:03.491007+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491261+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:34:03.491261+0000","last_peered":"2024-09-15T21:34:03.491261+0000","last_clean":"2024-09-15T21:34:03.491261+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:34:03.491261+0000","last_undegraded":"2024-09-15T21:34:03.491261+0000","last_fullsized":"2024-09-15T21:34:03.491261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21630,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491119+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:34:03.491119+0000","last_peered":"2024-09-15T21:34:03.491119+0000","last_clean":"2024-09-15T21:34:03.491119+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:34:03.491119+0000","last_undegraded":"2024-09-15T21:34:03.491119+0000","last_fullsized":"2024-09-15T21:34:03.491119+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347268+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:34:04.347268+0000","last_peered":"2024-09-15T21:34:04.347268+0000","last_clean":"2024-09-15T21:34:04.347268+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:34:04.347268+0000","last_undegraded":"2024-09-15T21:34:04.347268+0000","last_fullsized":"2024-09-15T21:34:04.347268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28746,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490956+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:34:03.490956+0000","last_peered":"2024-09-15T21:34:03.490956+0000","last_clean":"2024-09-15T21:34:03.490956+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:34:03.490956+0000","last_undegraded":"2024-09-15T21:34:03.490956+0000","last_fullsized":"2024-09-15T21:34:03.490956+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490974+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:34:03.490974+0000","last_peered":"2024-09-15T21:34:03.490974+0000","last_clean":"2024-09-15T21:34:03.490974+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:34:03.490974+0000","last_undegraded":"2024-09-15T21:34:03.490974+0000","last_fullsized":"2024-09-15T21:34:03.490974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21031,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49474,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118407168,"data_stored":118320456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414604,"kb_used_data":53884,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93957236,"statfs":{"total":96636764160,"available":96212209664,"internally_reserved":0,"allocated":55177216,"data_stored":54808009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532596,"kb_used_data":63652,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93839244,"statfs":{"total":96636764160,"available":96091385856,"internally_reserved":0,"allocated":65179648,"data_stored":64795525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545944,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825896,"statfs":{"total":96636764160,"available":96077717504,"internally_reserved":0,"allocated":589824,"data_stored":216540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64126976,"data_stored":64090846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54251520,"data_stored":54226229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:11.759 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-15T21:34:12.042 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:12.043 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":67,"fsid":"1d60fae4-b8c3-4364-9331-b0445493a043","created":"2024-09-15T21:19:07.254022+0000","modified":"2024-09-15T21:34:03.482920+0000","last_up_change":"2024-09-15T21:21:12.333829+0000","last_in_change":"2024-09-15T21:21:05.287877+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-15T21:19:25.465189+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfound_pool","create_time":"2024-09-15T21:19:34.551635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c55f097a-59ed-4ce1-9803-4c6019429ea8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":65,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6801","nonce":173704734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6803","nonce":173704734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6807","nonce":173704734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":173704734},{"type":"v1","addr":"172.21.15.73:6805","nonce":173704734}]},"public_addr":"172.21.15.73:6801/173704734","cluster_addr":"172.21.15.73:6803/173704734","heartbeat_back_addr":"172.21.15.73:6807/173704734","heartbeat_front_addr":"172.21.15.73:6805/173704734","state":["exists","up"]},{"osd":1,"uuid":"7daff6cd-a09c-4c15-a282-0859e8372dd3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":26,"up_from":41,"up_thru":66,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3018414478},{"type":"v1","addr":"172.21.15.73:6817","nonce":3018414478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":3018414478},{"type":"v1","addr":"172.21.15.73:6819","nonce":3018414478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3018414478},{"type":"v1","addr":"172.21.15.73:6823","nonce":3018414478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":3018414478},{"type":"v1","addr":"172.21.15.73:6821","nonce":3018414478}]},"public_addr":"172.21.15.73:6817/3018414478","cluster_addr":"172.21.15.73:6819/3018414478","heartbeat_back_addr":"172.21.15.73:6823/3018414478","heartbeat_front_addr":"172.21.15.73:6821/3018414478","state":["exists","up"]},{"osd":2,"uuid":"9abad4dd-4a88-4080-9ab4-5c4203d7fd51","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":17,"up_from":31,"up_thru":56,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":508991462},{"type":"v1","addr":"172.21.15.73:6809","nonce":508991462}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":508991462},{"type":"v1","addr":"172.21.15.73:6811","nonce":508991462}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":508991462},{"type":"v1","addr":"172.21.15.73:6815","nonce":508991462}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":508991462},{"type":"v1","addr":"172.21.15.73:6813","nonce":508991462}]},"public_addr":"172.21.15.73:6809/508991462","cluster_addr":"172.21.15.73:6811/508991462","heartbeat_back_addr":"172.21.15.73:6815/508991462","heartbeat_front_addr":"172.21.15.73:6813/508991462","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.867529+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.608114+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-15T21:19:17.891920+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-15T21:34:13.056 INFO:tasks.ceph:Scrubbing osd.0 2024-09-15T21:34:13.056 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:34:13.196 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:34:13.196 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:34:13.196 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:34:13.208 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-15T21:34:13.496 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 0 to deep-scrub 2024-09-15T21:34:13.509 INFO:tasks.ceph:Scrubbing osd.1 2024-09-15T21:34:13.509 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:34:13.631 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:34:13.631 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:34:13.631 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:34:13.641 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-15T21:34:13.925 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 1 to deep-scrub 2024-09-15T21:34:13.937 INFO:tasks.ceph:Scrubbing osd.2 2024-09-15T21:34:13.938 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-15T21:34:14.059 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-15T21:34:14.059 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-15T21:34:14.059 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-15T21:34:14.070 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-15T21:34:14.357 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 2 to deep-scrub 2024-09-15T21:34:14.371 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:14.661 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:14.673 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2024-09-15T21:34:12.919845+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49475,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493144,"kb_used_data":118112,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281622376,"statfs":{"total":289910292480,"available":288381313024,"internally_reserved":0,"allocated":120946688,"data_stored":119820074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":659,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001832"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29548,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491875+0000","last_change":"2024-09-15T21:33:13.430580+0000","last_active":"2024-09-15T21:34:03.491875+0000","last_peered":"2024-09-15T21:34:03.491875+0000","last_clean":"2024-09-15T21:34:03.491875+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:34:03.491875+0000","last_undegraded":"2024-09-15T21:34:03.491875+0000","last_fullsized":"2024-09-15T21:34:03.491875+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:23:02.372812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20674","reported_seq":20827,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347113+0000","last_change":"2024-09-15T21:20:43.498228+0000","last_active":"2024-09-15T21:34:04.347113+0000","last_peered":"2024-09-15T21:34:04.347113+0000","last_clean":"2024-09-15T21:34:04.347113+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:34:04.347113+0000","last_undegraded":"2024-09-15T21:34:04.347113+0000","last_fullsized":"2024-09-15T21:34:04.347113+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1748","last_scrub_stamp":"2024-09-15T21:20:43.498120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:43.498120+0000","objects_scrubbed":1715,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:49:36.252769+0000","scrub_duration":544,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490840+0000","last_change":"2024-09-15T21:30:13.065970+0000","last_active":"2024-09-15T21:34:03.490840+0000","last_peered":"2024-09-15T21:34:03.490840+0000","last_clean":"2024-09-15T21:34:03.490840+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:34:03.490840+0000","last_undegraded":"2024-09-15T21:34:03.490840+0000","last_fullsized":"2024-09-15T21:34:03.490840+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1928","last_scrub_stamp":"2024-09-15T21:20:44.532058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:44.532058+0000","objects_scrubbed":1896,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:31:16.322389+0000","scrub_duration":603,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347155+0000","last_change":"2024-09-15T21:20:25.036489+0000","last_active":"2024-09-15T21:34:04.347155+0000","last_peered":"2024-09-15T21:34:04.347155+0000","last_clean":"2024-09-15T21:34:04.347155+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:34:04.347155+0000","last_undegraded":"2024-09-15T21:34:04.347155+0000","last_fullsized":"2024-09-15T21:34:04.347155+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:19:34.719751+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:43:01.505128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20505","reported_seq":20686,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491487+0000","last_change":"2024-09-15T21:21:03.916238+0000","last_active":"2024-09-15T21:34:03.491487+0000","last_peered":"2024-09-15T21:34:03.491487+0000","last_clean":"2024-09-15T21:34:03.491487+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:34:03.491487+0000","last_undegraded":"2024-09-15T21:34:03.491487+0000","last_fullsized":"2024-09-15T21:34:03.491487+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3578","last_scrub_stamp":"2024-09-15T21:21:03.916069+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:03.916069+0000","objects_scrubbed":3539,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:15:34.782959+0000","scrub_duration":1176,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20635","reported_seq":20795,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.565894+0000","last_change":"2024-09-15T21:34:03.565354+0000","last_active":"2024-09-15T21:34:03.565894+0000","last_peered":"2024-09-15T21:34:03.565894+0000","last_clean":"2024-09-15T21:34:03.565894+0000","last_became_active":"2024-09-15T21:34:03.492946+0000","last_became_peered":"2024-09-15T21:34:03.492946+0000","last_unstale":"2024-09-15T21:34:03.565894+0000","last_undegraded":"2024-09-15T21:34:03.565894+0000","last_fullsized":"2024-09-15T21:34:03.565894+0000","mapping_epoch":66,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3368","last_scrub_stamp":"2024-09-15T21:21:00.739021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:00.739021+0000","objects_scrubbed":3320,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:27:26.940164+0000","scrub_duration":968,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'20670","reported_seq":20846,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490723+0000","last_change":"2024-09-15T21:32:28.441547+0000","last_active":"2024-09-15T21:34:03.490723+0000","last_peered":"2024-09-15T21:34:03.490723+0000","last_clean":"2024-09-15T21:34:03.490723+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:34:03.490723+0000","last_undegraded":"2024-09-15T21:34:03.490723+0000","last_fullsized":"2024-09-15T21:34:03.490723+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3735","last_scrub_stamp":"2024-09-15T21:21:05.072580+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:05.072580+0000","objects_scrubbed":3685,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:47:06.588889+0000","scrub_duration":1290,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21045,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491479+0000","last_change":"2024-09-15T21:20:58.749020+0000","last_active":"2024-09-15T21:34:03.491479+0000","last_peered":"2024-09-15T21:34:03.491479+0000","last_clean":"2024-09-15T21:34:03.491479+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:34:03.491479+0000","last_undegraded":"2024-09-15T21:34:03.491479+0000","last_fullsized":"2024-09-15T21:34:03.491479+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3258","last_scrub_stamp":"2024-09-15T21:20:58.748875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:58.748875+0000","objects_scrubbed":3226,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T22:29:12.628301+0000","scrub_duration":954,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490643+0000","last_change":"2024-09-15T21:21:14.357459+0000","last_active":"2024-09-15T21:34:03.490643+0000","last_peered":"2024-09-15T21:34:03.490643+0000","last_clean":"2024-09-15T21:34:03.490643+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:34:03.490643+0000","last_undegraded":"2024-09-15T21:34:03.490643+0000","last_fullsized":"2024-09-15T21:34:03.490643+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:28:02.459243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'20513","reported_seq":20692,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491207+0000","last_change":"2024-09-15T21:31:17.261082+0000","last_active":"2024-09-15T21:34:03.491207+0000","last_peered":"2024-09-15T21:34:03.491207+0000","last_clean":"2024-09-15T21:34:03.491207+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:34:03.491207+0000","last_undegraded":"2024-09-15T21:34:03.491207+0000","last_fullsized":"2024-09-15T21:34:03.491207+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2088","last_scrub_stamp":"2024-09-15T21:20:46.454755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:46.454755+0000","objects_scrubbed":2062,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:01:10.321479+0000","scrub_duration":567,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346961+0000","last_change":"2024-09-15T21:20:25.037174+0000","last_active":"2024-09-15T21:34:04.346961+0000","last_peered":"2024-09-15T21:34:04.346961+0000","last_clean":"2024-09-15T21:34:04.346961+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:34:04.346961+0000","last_undegraded":"2024-09-15T21:34:04.346961+0000","last_fullsized":"2024-09-15T21:34:04.346961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:44:44.549941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28713,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491776+0000","last_change":"2024-09-15T21:32:50.482953+0000","last_active":"2024-09-15T21:34:03.491776+0000","last_peered":"2024-09-15T21:34:03.491776+0000","last_clean":"2024-09-15T21:34:03.491776+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:34:03.491776+0000","last_undegraded":"2024-09-15T21:34:03.491776+0000","last_fullsized":"2024-09-15T21:34:03.491776+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3360","last_scrub_stamp":"2024-09-15T21:21:01.957162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:01.957162+0000","objects_scrubbed":3319,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:36:48.346380+0000","scrub_duration":1158,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":149,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491687+0000","last_change":"2024-09-15T21:21:14.550976+0000","last_active":"2024-09-15T21:34:03.491687+0000","last_peered":"2024-09-15T21:34:03.491687+0000","last_clean":"2024-09-15T21:34:03.491687+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:34:03.491687+0000","last_undegraded":"2024-09-15T21:34:03.491687+0000","last_fullsized":"2024-09-15T21:34:03.491687+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:50:42.023280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20643","reported_seq":29000,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347029+0000","last_change":"2024-09-15T21:32:05.358330+0000","last_active":"2024-09-15T21:34:04.347029+0000","last_peered":"2024-09-15T21:34:04.347029+0000","last_clean":"2024-09-15T21:34:04.347029+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:34:04.347029+0000","last_undegraded":"2024-09-15T21:34:04.347029+0000","last_fullsized":"2024-09-15T21:34:04.347029+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2164","last_scrub_stamp":"2024-09-15T21:20:47.548692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:47.548692+0000","objects_scrubbed":2137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:23:40.746325+0000","scrub_duration":641,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491109+0000","last_change":"2024-09-15T21:21:14.417247+0000","last_active":"2024-09-15T21:34:03.491109+0000","last_peered":"2024-09-15T21:34:03.491109+0000","last_clean":"2024-09-15T21:34:03.491109+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:34:03.491109+0000","last_undegraded":"2024-09-15T21:34:03.491109+0000","last_fullsized":"2024-09-15T21:34:03.491109+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:45:45.586858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29067,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.346908+0000","last_change":"2024-09-15T21:23:35.904478+0000","last_active":"2024-09-15T21:34:04.346908+0000","last_peered":"2024-09-15T21:34:04.346908+0000","last_clean":"2024-09-15T21:34:04.346908+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:34:04.346908+0000","last_undegraded":"2024-09-15T21:34:04.346908+0000","last_fullsized":"2024-09-15T21:34:04.346908+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2342","last_scrub_stamp":"2024-09-15T21:20:48.647896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:48.647896+0000","objects_scrubbed":2301,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:16.721535+0000","scrub_duration":744,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20865,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491007+0000","last_change":"2024-09-15T21:31:44.349140+0000","last_active":"2024-09-15T21:34:03.491007+0000","last_peered":"2024-09-15T21:34:03.491007+0000","last_clean":"2024-09-15T21:34:03.491007+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:34:03.491007+0000","last_undegraded":"2024-09-15T21:34:03.491007+0000","last_fullsized":"2024-09-15T21:34:03.491007+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3101","last_scrub_stamp":"2024-09-15T21:20:57.978134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:57.978134+0000","objects_scrubbed":3054,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:49:37.486000+0000","scrub_duration":1164,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491261+0000","last_change":"2024-09-15T21:21:14.526639+0000","last_active":"2024-09-15T21:34:03.491261+0000","last_peered":"2024-09-15T21:34:03.491261+0000","last_clean":"2024-09-15T21:34:03.491261+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:34:03.491261+0000","last_undegraded":"2024-09-15T21:34:03.491261+0000","last_fullsized":"2024-09-15T21:34:03.491261+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:45:38.776295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21630,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.491119+0000","last_change":"2024-09-15T21:30:55.214678+0000","last_active":"2024-09-15T21:34:03.491119+0000","last_peered":"2024-09-15T21:34:03.491119+0000","last_clean":"2024-09-15T21:34:03.491119+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:34:03.491119+0000","last_undegraded":"2024-09-15T21:34:03.491119+0000","last_fullsized":"2024-09-15T21:34:03.491119+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3128","last_scrub_stamp":"2024-09-15T21:20:56.700158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:56.700158+0000","objects_scrubbed":3083,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:44:31.453707+0000","scrub_duration":875,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:04.347268+0000","last_change":"2024-09-15T21:20:25.036299+0000","last_active":"2024-09-15T21:34:04.347268+0000","last_peered":"2024-09-15T21:34:04.347268+0000","last_clean":"2024-09-15T21:34:04.347268+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:34:04.347268+0000","last_undegraded":"2024-09-15T21:34:04.347268+0000","last_fullsized":"2024-09-15T21:34:04.347268+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:09:42.204423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28746,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490956+0000","last_change":"2024-09-15T21:27:09.366295+0000","last_active":"2024-09-15T21:34:03.490956+0000","last_peered":"2024-09-15T21:34:03.490956+0000","last_clean":"2024-09-15T21:34:03.490956+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:34:03.490956+0000","last_undegraded":"2024-09-15T21:34:03.490956+0000","last_fullsized":"2024-09-15T21:34:03.490956+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3547","last_scrub_stamp":"2024-09-15T21:21:02.966850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:21:02.966850+0000","objects_scrubbed":3512,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:55:17.977638+0000","scrub_duration":1213,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:21:14.546594+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:01:20.229401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":122,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490974+0000","last_change":"2024-09-15T21:21:14.355314+0000","last_active":"2024-09-15T21:34:03.490974+0000","last_peered":"2024-09-15T21:34:03.490974+0000","last_clean":"2024-09-15T21:34:03.490974+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:34:03.490974+0000","last_undegraded":"2024-09-15T21:34:03.490974+0000","last_fullsized":"2024-09-15T21:34:03.490974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:25.689420+0000","last_clean_scrub_stamp":"2024-09-15T21:19:25.689420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:27:29.727125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'20866","reported_seq":21031,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:03.490925+0000","last_change":"2024-09-15T21:33:38.439569+0000","last_active":"2024-09-15T21:34:03.490925+0000","last_peered":"2024-09-15T21:34:03.490925+0000","last_clean":"2024-09-15T21:34:03.490925+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:34:03.490925+0000","last_undegraded":"2024-09-15T21:34:03.490925+0000","last_fullsized":"2024-09-15T21:34:03.490925+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-15T21:20:59.797974+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:19:34.719751+0000","last_clean_scrub_stamp":"2024-09-15T21:20:59.797974+0000","objects_scrubbed":3286,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:21.308809+0000","scrub_duration":989,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49474,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118407168,"data_stored":118320456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160852,"ondisk_log_size":160852,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414604,"kb_used_data":53884,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93957236,"statfs":{"total":96636764160,"available":96212209664,"internally_reserved":0,"allocated":55177216,"data_stored":54808009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":532596,"kb_used_data":63652,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93839244,"statfs":{"total":96636764160,"available":96091385856,"internally_reserved":0,"allocated":65179648,"data_stored":64795525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545944,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825896,"statfs":{"total":96636764160,"available":96077717504,"internally_reserved":0,"allocated":589824,"data_stored":216540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64126976,"data_stored":64090846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54251520,"data_stored":54226229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:14.677 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-15T21:19:34.719751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=19, tm_sec=34, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.678 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-15T21:20:43.498120+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=20, tm_sec=43, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.678 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-15T21:20:44.532058+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=20, tm_sec=44, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.678 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-15T21:19:34.719751+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=19, tm_sec=34, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.678 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-15T21:21:03.916069+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=3, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.678 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-15T21:21:00.739021+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=0, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.678 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-15T21:21:05.072580+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=5, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.678 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-15T21:20:58.748875+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=20, tm_sec=58, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.679 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-15T21:19:25.689420+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=19, tm_sec=25, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.679 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-15T21:20:46.454755+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=20, tm_sec=46, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.679 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-15T21:19:25.689420+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=19, tm_sec=25, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.679 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-15T21:21:01.957162+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=1, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.679 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-15T21:19:25.689420+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=19, tm_sec=25, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.679 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-15T21:20:47.548692+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=20, tm_sec=47, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.679 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-15T21:19:25.689420+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=19, tm_sec=25, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.679 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-15T21:20:48.647896+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=20, tm_sec=48, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.679 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-15T21:20:57.978134+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=20, tm_sec=57, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.680 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-15T21:19:25.689420+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=19, tm_sec=25, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.680 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-15T21:20:56.700158+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=20, tm_sec=56, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.680 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-15T21:19:25.689420+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=19, tm_sec=25, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.680 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-15T21:21:02.966850+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=21, tm_sec=2, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.680 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-15T21:19:25.689420+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=19, tm_sec=25, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.680 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-15T21:19:25.689420+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=19, tm_sec=25, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.680 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-15T21:20:59.797974+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=20, tm_sec=59, tm_wday=6, tm_yday=259, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=15, tm_hour=21, tm_min=34, tm_sec=12, tm_wday=6, tm_yday=259, tm_isdst=0) 2024-09-15T21:34:14.680 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-15T21:34:34.681 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-15T21:34:34.983 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-15T21:34:34.983 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-09-15T21:34:34.995 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2024-09-15T21:34:34.924836+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4849,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290447,"num_write_kb":500639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49475,"num_bytes_recovered":32631653,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160867,"ondisk_log_size":160867,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462344,"kb_used_data":87320,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281653176,"statfs":{"total":289910292480,"available":288412852224,"internally_reserved":0,"allocated":89415680,"data_stored":88296292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13309,"internal_metadata":1407962115},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002766"},"pg_stats":[{"pgid":"2.f","version":"49'21152","reported_seq":29556,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:18.107633+0000","last_change":"2024-09-15T21:34:18.107633+0000","last_active":"2024-09-15T21:34:18.107633+0000","last_peered":"2024-09-15T21:34:18.107633+0000","last_clean":"2024-09-15T21:34:18.107633+0000","last_became_active":"2024-09-15T21:33:13.358609+0000","last_became_peered":"2024-09-15T21:33:13.358609+0000","last_unstale":"2024-09-15T21:34:18.107633+0000","last_undegraded":"2024-09-15T21:34:18.107633+0000","last_fullsized":"2024-09-15T21:34:18.107633+0000","mapping_epoch":62,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'21152","last_scrub_stamp":"2024-09-15T21:34:18.107542+0000","last_deep_scrub":"49'21152","last_deep_scrub_stamp":"2024-09-15T21:34:18.107542+0000","last_clean_scrub_stamp":"2024-09-15T21:34:18.107542+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:27:09.374252+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21152,"num_write_kb":42304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"67'20676","reported_seq":20838,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:18.307810+0000","last_change":"2024-09-15T21:34:18.307810+0000","last_active":"2024-09-15T21:34:18.307810+0000","last_peered":"2024-09-15T21:34:18.307810+0000","last_clean":"2024-09-15T21:34:18.307810+0000","last_became_active":"2024-09-15T21:20:25.035652+0000","last_became_peered":"2024-09-15T21:20:25.035652+0000","last_unstale":"2024-09-15T21:34:18.307810+0000","last_undegraded":"2024-09-15T21:34:18.307810+0000","last_fullsized":"2024-09-15T21:34:18.307810+0000","mapping_epoch":35,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'20676","last_scrub_stamp":"2024-09-15T21:34:18.307717+0000","last_deep_scrub":"67'20676","last_deep_scrub_stamp":"2024-09-15T21:34:18.307717+0000","last_clean_scrub_stamp":"2024-09-15T21:34:18.307717+0000","objects_scrubbed":2,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:55:08.769203+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":18955,"num_write_kb":34484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'21106","reported_seq":29834,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:25.930342+0000","last_change":"2024-09-15T21:34:25.930342+0000","last_active":"2024-09-15T21:34:25.930342+0000","last_peered":"2024-09-15T21:34:25.930342+0000","last_clean":"2024-09-15T21:34:25.930342+0000","last_became_active":"2024-09-15T21:30:12.967902+0000","last_became_peered":"2024-09-15T21:30:12.967902+0000","last_unstale":"2024-09-15T21:34:25.930342+0000","last_undegraded":"2024-09-15T21:34:25.930342+0000","last_fullsized":"2024-09-15T21:34:25.930342+0000","mapping_epoch":48,"log_start":"45'11100","ondisk_log_start":"45'11100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'21106","last_scrub_stamp":"2024-09-15T21:34:25.930245+0000","last_deep_scrub":"49'21106","last_deep_scrub_stamp":"2024-09-15T21:34:25.930245+0000","last_clean_scrub_stamp":"2024-09-15T21:34:25.930245+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:52:21.742156+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":19206,"num_write_kb":34617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20314","reported_seq":20441,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:17.333091+0000","last_change":"2024-09-15T21:34:17.333091+0000","last_active":"2024-09-15T21:34:17.333091+0000","last_peered":"2024-09-15T21:34:17.333091+0000","last_clean":"2024-09-15T21:34:17.333091+0000","last_became_active":"2024-09-15T21:20:25.036206+0000","last_became_peered":"2024-09-15T21:20:25.036206+0000","last_unstale":"2024-09-15T21:34:17.333091+0000","last_undegraded":"2024-09-15T21:34:17.333091+0000","last_fullsized":"2024-09-15T21:34:17.333091+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20314","last_scrub_stamp":"2024-09-15T21:34:17.332991+0000","last_deep_scrub":"49'20314","last_deep_scrub_stamp":"2024-09-15T21:34:17.332991+0000","last_clean_scrub_stamp":"2024-09-15T21:34:17.332991+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:00:52.235166+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20314,"num_write_kb":40628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"67'20506","reported_seq":20695,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:17.082160+0000","last_change":"2024-09-15T21:34:17.082160+0000","last_active":"2024-09-15T21:34:17.082160+0000","last_peered":"2024-09-15T21:34:17.082160+0000","last_clean":"2024-09-15T21:34:17.082160+0000","last_became_active":"2024-09-15T21:20:24.035872+0000","last_became_peered":"2024-09-15T21:20:24.035872+0000","last_unstale":"2024-09-15T21:34:17.082160+0000","last_undegraded":"2024-09-15T21:34:17.082160+0000","last_fullsized":"2024-09-15T21:34:17.082160+0000","mapping_epoch":34,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'20506","last_scrub_stamp":"2024-09-15T21:34:17.082075+0000","last_deep_scrub":"67'20506","last_deep_scrub_stamp":"2024-09-15T21:34:17.082075+0000","last_clean_scrub_stamp":"2024-09-15T21:34:17.082075+0000","objects_scrubbed":1,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:37:04.840667+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16960,"num_write_kb":26844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"67'20636","reported_seq":20804,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:21.966094+0000","last_change":"2024-09-15T21:34:21.966094+0000","last_active":"2024-09-15T21:34:21.966094+0000","last_peered":"2024-09-15T21:34:21.966094+0000","last_clean":"2024-09-15T21:34:21.966094+0000","last_became_active":"2024-09-15T21:34:03.492946+0000","last_became_peered":"2024-09-15T21:34:03.492946+0000","last_unstale":"2024-09-15T21:34:21.966094+0000","last_undegraded":"2024-09-15T21:34:21.966094+0000","last_fullsized":"2024-09-15T21:34:21.966094+0000","mapping_epoch":66,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'20636","last_scrub_stamp":"2024-09-15T21:34:21.966013+0000","last_deep_scrub":"67'20636","last_deep_scrub_stamp":"2024-09-15T21:34:21.966013+0000","last_clean_scrub_stamp":"2024-09-15T21:34:21.966013+0000","objects_scrubbed":1,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:54.493454+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17311,"num_write_kb":27984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4077,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"67'20672","reported_seq":20856,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:22.950554+0000","last_change":"2024-09-15T21:34:22.950554+0000","last_active":"2024-09-15T21:34:22.950554+0000","last_peered":"2024-09-15T21:34:22.950554+0000","last_clean":"2024-09-15T21:34:22.950554+0000","last_became_active":"2024-09-15T21:32:28.362029+0000","last_became_peered":"2024-09-15T21:32:28.362029+0000","last_unstale":"2024-09-15T21:34:22.950554+0000","last_undegraded":"2024-09-15T21:34:22.950554+0000","last_fullsized":"2024-09-15T21:34:22.950554+0000","mapping_epoch":58,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'20672","last_scrub_stamp":"2024-09-15T21:34:22.950475+0000","last_deep_scrub":"67'20672","last_deep_scrub_stamp":"2024-09-15T21:34:22.950475+0000","last_clean_scrub_stamp":"2024-09-15T21:34:22.950475+0000","objects_scrubbed":2,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:04:06.645275+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16981,"num_write_kb":26596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4161,"num_bytes_recovered":966,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20868","reported_seq":21053,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:15.131613+0000","last_change":"2024-09-15T21:34:15.131613+0000","last_active":"2024-09-15T21:34:15.131613+0000","last_peered":"2024-09-15T21:34:15.131613+0000","last_clean":"2024-09-15T21:34:15.131613+0000","last_became_active":"2024-09-15T21:20:24.032204+0000","last_became_peered":"2024-09-15T21:20:24.032204+0000","last_unstale":"2024-09-15T21:34:15.131613+0000","last_undegraded":"2024-09-15T21:34:15.131613+0000","last_fullsized":"2024-09-15T21:34:15.131613+0000","mapping_epoch":34,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20868","last_scrub_stamp":"2024-09-15T21:34:15.131514+0000","last_deep_scrub":"49'20868","last_deep_scrub_stamp":"2024-09-15T21:34:15.131514+0000","last_clean_scrub_stamp":"2024-09-15T21:34:15.131514+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:22:48.946674+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17640,"num_write_kb":28828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:14.091542+0000","last_change":"2024-09-15T21:34:14.091542+0000","last_active":"2024-09-15T21:34:14.091542+0000","last_peered":"2024-09-15T21:34:14.091542+0000","last_clean":"2024-09-15T21:34:14.091542+0000","last_became_active":"2024-09-15T21:21:14.357141+0000","last_became_peered":"2024-09-15T21:21:14.357141+0000","last_unstale":"2024-09-15T21:34:14.091542+0000","last_undegraded":"2024-09-15T21:34:14.091542+0000","last_fullsized":"2024-09-15T21:34:14.091542+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:34:14.091477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:34:14.091477+0000","last_clean_scrub_stamp":"2024-09-15T21:34:14.091477+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:44:39.779914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050217800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"67'20514","reported_seq":20701,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:18.092374+0000","last_change":"2024-09-15T21:34:18.092374+0000","last_active":"2024-09-15T21:34:18.092374+0000","last_peered":"2024-09-15T21:34:18.092374+0000","last_clean":"2024-09-15T21:34:18.092374+0000","last_became_active":"2024-09-15T21:31:17.164435+0000","last_became_peered":"2024-09-15T21:31:17.164435+0000","last_unstale":"2024-09-15T21:34:18.092374+0000","last_undegraded":"2024-09-15T21:34:18.092374+0000","last_fullsized":"2024-09-15T21:34:18.092374+0000","mapping_epoch":52,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'20514","last_scrub_stamp":"2024-09-15T21:34:18.092297+0000","last_deep_scrub":"67'20514","last_deep_scrub_stamp":"2024-09-15T21:34:18.092297+0000","last_clean_scrub_stamp":"2024-09-15T21:34:18.092297+0000","objects_scrubbed":1,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:07.559011+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":32768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4088,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:16.323458+0000","last_change":"2024-09-15T21:34:16.323458+0000","last_active":"2024-09-15T21:34:16.323458+0000","last_peered":"2024-09-15T21:34:16.323458+0000","last_clean":"2024-09-15T21:34:16.323458+0000","last_became_active":"2024-09-15T21:20:25.036522+0000","last_became_peered":"2024-09-15T21:20:25.036522+0000","last_unstale":"2024-09-15T21:34:16.323458+0000","last_undegraded":"2024-09-15T21:34:16.323458+0000","last_fullsized":"2024-09-15T21:34:16.323458+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:34:16.323370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:34:16.323370+0000","last_clean_scrub_stamp":"2024-09-15T21:34:16.323370+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:03:18.813791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'20482","reported_seq":28721,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:14.152176+0000","last_change":"2024-09-15T21:34:14.152176+0000","last_active":"2024-09-15T21:34:14.152176+0000","last_peered":"2024-09-15T21:34:14.152176+0000","last_clean":"2024-09-15T21:34:14.152176+0000","last_became_active":"2024-09-15T21:32:50.428023+0000","last_became_peered":"2024-09-15T21:32:50.428023+0000","last_unstale":"2024-09-15T21:34:14.152176+0000","last_undegraded":"2024-09-15T21:34:14.152176+0000","last_fullsized":"2024-09-15T21:34:14.152176+0000","mapping_epoch":60,"log_start":"45'10400","ondisk_log_start":"45'10400","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20482","last_scrub_stamp":"2024-09-15T21:34:14.152087+0000","last_deep_scrub":"49'20482","last_deep_scrub_stamp":"2024-09-15T21:34:14.152087+0000","last_clean_scrub_stamp":"2024-09-15T21:34:14.152087+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:37:02.213777+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":27672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4022,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"67'3","reported_seq":158,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:16.079796+0000","last_change":"2024-09-15T21:34:16.079796+0000","last_active":"2024-09-15T21:34:16.079796+0000","last_peered":"2024-09-15T21:34:16.079796+0000","last_clean":"2024-09-15T21:34:16.079796+0000","last_became_active":"2024-09-15T21:21:14.476816+0000","last_became_peered":"2024-09-15T21:21:14.476816+0000","last_unstale":"2024-09-15T21:34:16.079796+0000","last_undegraded":"2024-09-15T21:34:16.079796+0000","last_fullsized":"2024-09-15T21:34:16.079796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'3","last_scrub_stamp":"2024-09-15T21:34:16.079714+0000","last_deep_scrub":"67'3","last_deep_scrub_stamp":"2024-09-15T21:34:16.079714+0000","last_clean_scrub_stamp":"2024-09-15T21:34:16.079714+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:38.168175+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00058958199999999995,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"67'20644","reported_seq":29009,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:19.337951+0000","last_change":"2024-09-15T21:34:19.337951+0000","last_active":"2024-09-15T21:34:19.337951+0000","last_peered":"2024-09-15T21:34:19.337951+0000","last_clean":"2024-09-15T21:34:19.337951+0000","last_became_active":"2024-09-15T21:32:05.308083+0000","last_became_peered":"2024-09-15T21:32:05.308083+0000","last_unstale":"2024-09-15T21:34:19.337951+0000","last_undegraded":"2024-09-15T21:34:19.337951+0000","last_fullsized":"2024-09-15T21:34:19.337951+0000","mapping_epoch":56,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'20644","last_scrub_stamp":"2024-09-15T21:34:19.337847+0000","last_deep_scrub":"67'20644","last_deep_scrub_stamp":"2024-09-15T21:34:19.337847+0000","last_clean_scrub_stamp":"2024-09-15T21:34:19.337847+0000","objects_scrubbed":1,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:44:37.374469+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18502,"num_write_kb":32732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4091,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"67'2","reported_seq":137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:15.139550+0000","last_change":"2024-09-15T21:34:15.139550+0000","last_active":"2024-09-15T21:34:15.139550+0000","last_peered":"2024-09-15T21:34:15.139550+0000","last_clean":"2024-09-15T21:34:15.139550+0000","last_became_active":"2024-09-15T21:21:14.416862+0000","last_became_peered":"2024-09-15T21:21:14.416862+0000","last_unstale":"2024-09-15T21:34:15.139550+0000","last_undegraded":"2024-09-15T21:34:15.139550+0000","last_fullsized":"2024-09-15T21:34:15.139550+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'2","last_scrub_stamp":"2024-09-15T21:34:15.139468+0000","last_deep_scrub":"67'2","last_deep_scrub_stamp":"2024-09-15T21:34:15.139468+0000","last_clean_scrub_stamp":"2024-09-15T21:34:15.139468+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:40.099328+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00067795599999999996,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'20560","reported_seq":29075,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:15.345974+0000","last_change":"2024-09-15T21:34:15.345974+0000","last_active":"2024-09-15T21:34:15.345974+0000","last_peered":"2024-09-15T21:34:15.345974+0000","last_clean":"2024-09-15T21:34:15.345974+0000","last_became_active":"2024-09-15T21:23:35.891141+0000","last_became_peered":"2024-09-15T21:23:35.891141+0000","last_unstale":"2024-09-15T21:34:15.345974+0000","last_undegraded":"2024-09-15T21:34:15.345974+0000","last_fullsized":"2024-09-15T21:34:15.345974+0000","mapping_epoch":44,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20560","last_scrub_stamp":"2024-09-15T21:34:15.345865+0000","last_deep_scrub":"49'20560","last_deep_scrub_stamp":"2024-09-15T21:34:15.345865+0000","last_clean_scrub_stamp":"2024-09-15T21:34:15.345865+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:22:42.861094+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18253,"num_write_kb":31904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4164,"num_bytes_recovered":17047552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20686","reported_seq":20873,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:20.001095+0000","last_change":"2024-09-15T21:34:20.001095+0000","last_active":"2024-09-15T21:34:20.001095+0000","last_peered":"2024-09-15T21:34:20.001095+0000","last_clean":"2024-09-15T21:34:20.001095+0000","last_became_active":"2024-09-15T21:31:44.254288+0000","last_became_peered":"2024-09-15T21:31:44.254288+0000","last_unstale":"2024-09-15T21:34:20.001095+0000","last_undegraded":"2024-09-15T21:34:20.001095+0000","last_fullsized":"2024-09-15T21:34:20.001095+0000","mapping_epoch":54,"log_start":"45'10600","ondisk_log_start":"45'10600","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20686","last_scrub_stamp":"2024-09-15T21:34:20.001008+0000","last_deep_scrub":"49'20686","last_deep_scrub_stamp":"2024-09-15T21:34:20.001008+0000","last_clean_scrub_stamp":"2024-09-15T21:34:20.001008+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:43:54.314816+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17630,"num_write_kb":29152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4096,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:16.099684+0000","last_change":"2024-09-15T21:34:16.099684+0000","last_active":"2024-09-15T21:34:16.099684+0000","last_peered":"2024-09-15T21:34:16.099684+0000","last_clean":"2024-09-15T21:34:16.099684+0000","last_became_active":"2024-09-15T21:21:14.522777+0000","last_became_peered":"2024-09-15T21:21:14.522777+0000","last_unstale":"2024-09-15T21:34:16.099684+0000","last_undegraded":"2024-09-15T21:34:16.099684+0000","last_fullsized":"2024-09-15T21:34:16.099684+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:34:16.099562+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:34:16.099562+0000","last_clean_scrub_stamp":"2024-09-15T21:34:16.099562+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T23:41:23.862477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085541100000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'20590","reported_seq":21638,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:23.947942+0000","last_change":"2024-09-15T21:34:23.947942+0000","last_active":"2024-09-15T21:34:23.947942+0000","last_peered":"2024-09-15T21:34:23.947942+0000","last_clean":"2024-09-15T21:34:23.947942+0000","last_became_active":"2024-09-15T21:30:55.121448+0000","last_became_peered":"2024-09-15T21:30:55.121448+0000","last_unstale":"2024-09-15T21:34:23.947942+0000","last_undegraded":"2024-09-15T21:34:23.947942+0000","last_fullsized":"2024-09-15T21:34:23.947942+0000","mapping_epoch":50,"log_start":"45'10500","ondisk_log_start":"45'10500","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20590","last_scrub_stamp":"2024-09-15T21:34:23.946848+0000","last_deep_scrub":"49'20590","last_deep_scrub_stamp":"2024-09-15T21:34:23.946848+0000","last_clean_scrub_stamp":"2024-09-15T21:34:23.946848+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:16:39.384359+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":28832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4142,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:14.376093+0000","last_change":"2024-09-15T21:34:14.376093+0000","last_active":"2024-09-15T21:34:14.376093+0000","last_peered":"2024-09-15T21:34:14.376093+0000","last_clean":"2024-09-15T21:34:14.376093+0000","last_became_active":"2024-09-15T21:20:25.036032+0000","last_became_peered":"2024-09-15T21:20:25.036032+0000","last_unstale":"2024-09-15T21:34:14.376093+0000","last_undegraded":"2024-09-15T21:34:14.376093+0000","last_fullsized":"2024-09-15T21:34:14.376093+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:34:14.376030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:34:14.376030+0000","last_clean_scrub_stamp":"2024-09-15T21:34:14.376030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:31:20.904477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20388","reported_seq":28754,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:24.903856+0000","last_change":"2024-09-15T21:34:24.903856+0000","last_active":"2024-09-15T21:34:24.903856+0000","last_peered":"2024-09-15T21:34:24.903856+0000","last_clean":"2024-09-15T21:34:24.903856+0000","last_became_active":"2024-09-15T21:27:09.322269+0000","last_became_peered":"2024-09-15T21:27:09.322269+0000","last_unstale":"2024-09-15T21:34:24.903856+0000","last_undegraded":"2024-09-15T21:34:24.903856+0000","last_fullsized":"2024-09-15T21:34:24.903856+0000","mapping_epoch":46,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20388","last_scrub_stamp":"2024-09-15T21:34:24.903747+0000","last_deep_scrub":"49'20388","last_deep_scrub_stamp":"2024-09-15T21:34:24.903747+0000","last_clean_scrub_stamp":"2024-09-15T21:34:24.903747+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:32.637106+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16872,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4084,"num_bytes_recovered":15577088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:20.998426+0000","last_change":"2024-09-15T21:34:20.998426+0000","last_active":"2024-09-15T21:34:20.998426+0000","last_peered":"2024-09-15T21:34:20.998426+0000","last_clean":"2024-09-15T21:34:20.998426+0000","last_became_active":"2024-09-15T21:21:14.546254+0000","last_became_peered":"2024-09-15T21:21:14.546254+0000","last_unstale":"2024-09-15T21:34:20.998426+0000","last_undegraded":"2024-09-15T21:34:20.998426+0000","last_fullsized":"2024-09-15T21:34:20.998426+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:34:20.998344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:34:20.998344+0000","last_clean_scrub_stamp":"2024-09-15T21:34:20.998344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-16T21:46:16.822967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076649499999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":131,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:17.065335+0000","last_change":"2024-09-15T21:34:17.065335+0000","last_active":"2024-09-15T21:34:17.065335+0000","last_peered":"2024-09-15T21:34:17.065335+0000","last_clean":"2024-09-15T21:34:17.065335+0000","last_became_active":"2024-09-15T21:21:14.354953+0000","last_became_peered":"2024-09-15T21:21:14.354953+0000","last_unstale":"2024-09-15T21:34:17.065335+0000","last_undegraded":"2024-09-15T21:34:17.065335+0000","last_fullsized":"2024-09-15T21:34:17.065335+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-15T21:34:17.065249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-15T21:34:17.065249+0000","last_clean_scrub_stamp":"2024-09-15T21:34:17.065249+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:54:26.805977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062736699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"67'20868","reported_seq":21041,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-15T21:34:19.051190+0000","last_change":"2024-09-15T21:34:19.051190+0000","last_active":"2024-09-15T21:34:19.051190+0000","last_peered":"2024-09-15T21:34:19.051190+0000","last_clean":"2024-09-15T21:34:19.051190+0000","last_became_active":"2024-09-15T21:33:38.357917+0000","last_became_peered":"2024-09-15T21:33:38.357917+0000","last_unstale":"2024-09-15T21:34:19.051190+0000","last_undegraded":"2024-09-15T21:34:19.051190+0000","last_fullsized":"2024-09-15T21:34:19.051190+0000","mapping_epoch":64,"log_start":"45'10800","ondisk_log_start":"45'10800","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'20868","last_scrub_stamp":"2024-09-15T21:34:19.051104+0000","last_deep_scrub":"67'20868","last_deep_scrub_stamp":"2024-09-15T21:34:19.051104+0000","last_clean_scrub_stamp":"2024-09-15T21:34:19.051104+0000","objects_scrubbed":2,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:22:49.915596+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17570,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4149,"num_bytes_recovered":483,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":4830,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":290445,"num_write_kb":500637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":49474,"num_bytes_recovered":32631634,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":86753280,"data_stored":86673794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160862,"ondisk_log_size":160862,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":395812,"kb_used_data":35092,"kb_used_omap":4,"kb_used_meta":360699,"kb_avail":93976028,"statfs":{"total":96636764160,"available":96231452672,"internally_reserved":0,"allocated":35934208,"data_stored":35568614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4439,"internal_metadata":369356457},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":25,"seq":107374182579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520596,"kb_used_data":51652,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93851244,"statfs":{"total":96636764160,"available":96103673856,"internally_reserved":0,"allocated":52891648,"data_stored":52511138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":480177835},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545936,"kb_used_data":576,"kb_used_omap":4,"kb_used_meta":545339,"kb_avail":93825904,"statfs":{"total":96636764160,"available":96077725696,"internally_reserved":0,"allocated":589824,"data_stored":216540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":558427823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":51838976,"data_stored":51806459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34885632,"data_stored":34863954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-15T21:34:35.000 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-15T21:34:35.298 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-15T21:34:35.298 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-15T21:34:35.298 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-15T21:34:35.298 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:34:40.407 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-15T21:34:41.400 INFO:tasks.ceph.osd.0:Stopped 2024-09-15T21:34:41.401 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-15T21:34:41.401 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:34:41.478 INFO:tasks.ceph.osd.1:Stopped 2024-09-15T21:34:41.478 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-15T21:34:41.478 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:34:45.811 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-15T21:34:45.811 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-15T21:34:47.579 INFO:tasks.ceph.osd.2:Stopped 2024-09-15T21:34:47.579 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-15T21:34:47.580 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-15T21:34:47.580 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:34:47.611 INFO:tasks.ceph.mgr.x:Stopped 2024-09-15T21:34:47.611 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-15T21:34:47.611 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-15T21:34:47.611 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:34:47.665 INFO:tasks.ceph.mon.a:Stopped 2024-09-15T21:34:47.665 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-15T21:34:47.665 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:34:47.718 INFO:tasks.ceph.mon.b:Stopped 2024-09-15T21:34:47.718 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-15T21:34:47.718 INFO:teuthology.orchestra.run:waiting for 300 2024-09-15T21:34:47.770 INFO:tasks.ceph.mon.c:Stopped 2024-09-15T21:34:47.771 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-15T21:34:47.771 DEBUG:teuthology.orchestra.run.smithi073:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-15T21:34:47.874 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-15T21:34:47.875 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-15T21:34:48.085 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-15T21:34:48.085 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-15T21:34:48.234 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi073.front.sepia.ceph.com 2024-09-15T21:34:48.234 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-15T21:34:48.304 INFO:tasks.ceph:Archiving mon data... 2024-09-15T21:34:48.305 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905683/data/mon.a.tgz 2024-09-15T21:34:48.305 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-09-15T21:34:48.330 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.uX4VJw5Ur6 2024-09-15T21:34:48.330 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.uX4VJw5Ur6 2024-09-15T21:34:48.462 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.uX4VJw5Ur6 2024-09-15T21:34:48.566 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.uX4VJw5Ur6 is 288KB 2024-09-15T21:34:48.663 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.uX4VJw5Ur6 2024-09-15T21:34:48.678 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905683/data/mon.b.tgz 2024-09-15T21:34:48.679 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-09-15T21:34:48.733 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.FJ4Ex1yRip 2024-09-15T21:34:48.733 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.FJ4Ex1yRip 2024-09-15T21:34:48.847 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.FJ4Ex1yRip 2024-09-15T21:34:48.930 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.FJ4Ex1yRip is 289KB 2024-09-15T21:34:48.997 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.FJ4Ex1yRip 2024-09-15T21:34:49.011 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905683/data/mon.c.tgz 2024-09-15T21:34:49.012 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-09-15T21:34:49.066 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.pTArwre4TP 2024-09-15T21:34:49.066 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.pTArwre4TP 2024-09-15T21:34:49.179 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.pTArwre4TP 2024-09-15T21:34:49.264 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.pTArwre4TP is 289KB 2024-09-15T21:34:49.359 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.pTArwre4TP 2024-09-15T21:34:49.373 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-15T21:34:49.374 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-15T21:34:49.507 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-15T21:34:49.508 INFO:tasks.ceph:Archiving crash dumps... 2024-09-15T21:34:49.509 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905683/remote/smithi073/crash 2024-09-15T21:34:49.510 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-15T21:34:49.572 INFO:tasks.ceph:Compressing logs... 2024-09-15T21:34:49.573 DEBUG:teuthology.orchestra.run.smithi073:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-15T21:34:49.934 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-15T21:34:49.934 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37685.log 2024-09-15T21:34:49.935 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57284.log 2024-09-15T21:34:49.935 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54846.log 2024-09-15T21:34:49.935 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37685.log.gz 2024-09-15T21:34:49.935 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63310.log 2024-09-15T21:34:49.935 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57284.log.gz 2024-09-15T21:34:49.935 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.54846.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50016.log 2024-09-15T21:34:49.935 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54846.log.gz 2024-09-15T21:34:49.935 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56568.log 2024-09-15T21:34:49.935 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63310.log.gz 2024-09-15T21:34:49.935 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49134.log 2024-09-15T21:34:49.935 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50016.log.gz 2024-09-15T21:34:49.935 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70704.log 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56568.log.gz 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64528.log 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49134.log.gz 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66880.log 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70704.log.gz 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67468.log 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64528.log.gz 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42809.log 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66880.log.gz 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56736.log 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67468.log.gz 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68309.log 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42809.log.gz 2024-09-15T21:34:49.936 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56610.log 2024-09-15T21:34:49.937 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56736.log.gz 2024-09-15T21:34:49.937 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46992.log 2024-09-15T21:34:49.937 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68309.log.gz 2024-09-15T21:34:49.937 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52788.log 2024-09-15T21:34:49.937 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56610.log.gz 2024-09-15T21:34:49.937 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57114.log 2024-09-15T21:34:49.937 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46992.log.gz 2024-09-15T21:34:49.937 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70746.log 2024-09-15T21:34:49.937 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52788.log.gz 2024-09-15T21:34:49.937 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60745.log 2024-09-15T21:34:49.941 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57114.log.gz 2024-09-15T21:34:49.941 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36213.log 2024-09-15T21:34:49.941 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70746.log.gz 2024-09-15T21:34:49.941 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58221.log 2024-09-15T21:34:49.941 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60745.log.gz 2024-09-15T21:34:49.941 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56232.log 2024-09-15T21:34:49.941 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36213.log.gz 2024-09-15T21:34:49.941 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38324.log 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58221.log.gz 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61334.log 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56232.log.gz 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51318.log 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38324.log.gz 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35154.log 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61334.log.gz 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55812.log 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51318.log.gz 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58896.log 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35154.log.gz 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69360.log 2024-09-15T21:34:49.942 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55812.log.gz 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50226.log 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58896.log.gz 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55056.log 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69360.log.gz 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57072.log 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50226.log.gz 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40046.log 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55056.log.gz 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31956.log 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57072.log.gz 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36000.log 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40046.log.gz 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66250.log 2024-09-15T21:34:49.943 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31956.log.gz 2024-09-15T21:34:49.944 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36000.log.gz 2024-09-15T21:34:49.944 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50646.log 2024-09-15T21:34:49.944 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51990.log 2024-09-15T21:34:49.944 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66250.log.gz 2024-09-15T21:34:49.944 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58308.log 2024-09-15T21:34:49.944 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50646.log.gz 2024-09-15T21:34:49.944 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51990.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.51990.log.gz /var/log/ceph/ceph-client.admin.42076.log 2024-09-15T21:34:49.944 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-09-15T21:34:49.944 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35916.log 2024-09-15T21:34:49.944 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58308.log.gz 2024-09-15T21:34:49.944 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52242.log 2024-09-15T21:34:49.944 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42076.log.gz 2024-09-15T21:34:49.944 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69948.log 2024-09-15T21:34:49.945 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35916.log.gz 2024-09-15T21:34:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37963.log 2024-09-15T21:34:49.945 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52242.log.gz 2024-09-15T21:34:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50940.log 2024-09-15T21:34:49.945 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69948.log.gz 2024-09-15T21:34:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59400.log 2024-09-15T21:34:49.945 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37963.log.gz 2024-09-15T21:34:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50940.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.37463.log 2024-09-15T21:34:49.945 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50940.log.gz 2024-09-15T21:34:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44610.log 2024-09-15T21:34:49.945 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59400.log.gz 2024-09-15T21:34:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54762.log 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37463.log.gz 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68771.log 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44610.log.gz 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48294.log 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54762.log.gz 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68771.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.53040.log -- replaced with /var/log/ceph/ceph-client.admin.68771.log.gz 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59946.log 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48294.log.gz 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53040.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53040.log.gz 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68225.log 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69570.log 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59946.log.gz 2024-09-15T21:34:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68225.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.68225.log.gz 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43481.log 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67216.log 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69570.log.gz 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47118.log 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43481.log.gz 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42683.log 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67216.log.gz 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68477.log 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47118.log.gz 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66754.log 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42683.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42683.log.gz 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37426.log 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68477.log.gz 2024-09-15T21:34:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56484.log 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66754.log.gz 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.37426.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.71880.log 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37426.log.gz 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58686.log 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56484.log.gz 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69150.log 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71880.log.gz 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50898.log 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58686.log.gz 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.69150.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44795.log 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69150.log.gz 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53124.log 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50898.log.gz 2024-09-15T21:34:49.948 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69066.log 2024-09-15T21:34:49.949 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44795.log.gz 2024-09-15T21:34:49.949 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41657.log 2024-09-15T21:34:49.949 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53124.log.gz 2024-09-15T21:34:49.949 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56064.log 2024-09-15T21:34:49.949 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69066.log.gz 2024-09-15T21:34:49.949 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59358.log 2024-09-15T21:34:49.949 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41657.log.gz 2024-09-15T21:34:49.949 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59064.log 2024-09-15T21:34:49.949 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56064.log.gz 2024-09-15T21:34:49.949 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57199.log/var/log/ceph/ceph-client.admin.59358.log: 2024-09-15T21:34:49.949 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59358.log.gz 2024-09-15T21:34:49.949 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37759.log 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59064.log.gz 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52074.log 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57199.log.gz 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37759.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37759.log.gz -5 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-client.admin.51528.log 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44027.log 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52074.log.gz 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67637.log 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51528.log.gz 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57969.log 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44027.log.gz 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67637.log.gz 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59610.log 2024-09-15T21:34:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51612.log 2024-09-15T21:34:49.951 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57969.log.gz 2024-09-15T21:34:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62554.log 2024-09-15T21:34:49.951 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59610.log.gz 2024-09-15T21:34:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51612.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.51612.log.gz --verbose 2024-09-15T21:34:49.951 INFO:teuthology.orchestra.run.smithi073.stderr: -- /var/log/ceph/ceph-client.admin.64864.log 2024-09-15T21:34:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70284.log 2024-09-15T21:34:49.951 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62554.log.gz 2024-09-15T21:34:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55350.log 2024-09-15T21:34:49.951 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64864.log.gz 2024-09-15T21:34:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40298.log 2024-09-15T21:34:49.951 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70284.log.gz 2024-09-15T21:34:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36255.log 2024-09-15T21:34:49.951 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55350.log.gz 2024-09-15T21:34:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54720.log 2024-09-15T21:34:49.952 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40298.log.gz 2024-09-15T21:34:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47328.log 2024-09-15T21:34:49.952 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36255.log.gz 2024-09-15T21:34:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54720.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39943.log 2024-09-15T21:34:49.952 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54720.log.gz 2024-09-15T21:34:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52494.log 2024-09-15T21:34:49.952 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47328.log.gz 2024-09-15T21:34:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68729.log 2024-09-15T21:34:49.952 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39943.log.gz 2024-09-15T21:34:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35494.log 2024-09-15T21:34:49.952 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52494.log.gz 2024-09-15T21:34:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54006.log 2024-09-15T21:34:49.953 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68729.log.gz 2024-09-15T21:34:49.953 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49512.log 2024-09-15T21:34:49.953 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35494.log.gz 2024-09-15T21:34:49.953 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63436.log 2024-09-15T21:34:49.953 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54006.log.gz 2024-09-15T21:34:49.953 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52452.log 2024-09-15T21:34:49.953 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49512.log.gz 2024-09-15T21:34:49.953 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66796.log 2024-09-15T21:34:49.953 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63436.log.gz 2024-09-15T21:34:49.953 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55434.log 2024-09-15T21:34:49.953 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52452.log.gz 2024-09-15T21:34:49.953 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35958.log 2024-09-15T21:34:49.953 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66796.log.gz 2024-09-15T21:34:49.954 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55434.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.55434.log.gz 2024-09-15T21:34:49.954 INFO:teuthology.orchestra.run.smithi073.stderr: /var/log/ceph/ceph-client.admin.56904.log 2024-09-15T21:34:49.954 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42641.log 2024-09-15T21:34:49.954 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35958.log.gz 2024-09-15T21:34:49.954 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63940.log 2024-09-15T21:34:49.954 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56904.log.gz 2024-09-15T21:34:49.954 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42641.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.63100.log 2024-09-15T21:34:49.954 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42641.log.gz 2024-09-15T21:34:49.954 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65326.log 2024-09-15T21:34:49.954 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63940.log.gz 2024-09-15T21:34:49.954 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46110.log 2024-09-15T21:34:49.955 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63100.log.gz 2024-09-15T21:34:49.955 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68141.log 2024-09-15T21:34:49.955 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65326.log.gz 2024-09-15T21:34:49.955 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69864.log 2024-09-15T21:34:49.955 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46110.log.gz 2024-09-15T21:34:49.955 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65872.log 2024-09-15T21:34:49.955 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68141.log.gz 2024-09-15T21:34:49.955 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57705.log 2024-09-15T21:34:49.955 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69864.log.gz 2024-09-15T21:34:49.955 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69276.log 2024-09-15T21:34:49.955 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65872.log.gz 2024-09-15T21:34:49.955 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54384.log 2024-09-15T21:34:49.956 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57705.log.gz 2024-09-15T21:34:49.956 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48000.log 2024-09-15T21:34:49.956 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69276.log.gz 2024-09-15T21:34:49.956 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62302.log 2024-09-15T21:34:49.956 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54384.log.gz 2024-09-15T21:34:49.956 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48000.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-15T21:34:49.956 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48000.log.gz 2024-09-15T21:34:49.956 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50352.log 2024-09-15T21:34:49.956 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62302.log.gz 2024-09-15T21:34:49.957 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47706.log 2024-09-15T21:34:49.957 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64444.log 2024-09-15T21:34:49.957 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50352.log.gz 2024-09-15T21:34:49.957 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51444.log 2024-09-15T21:34:49.957 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47706.log.gz 2024-09-15T21:34:49.957 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41847.log 2024-09-15T21:34:49.957 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64444.log.gz 2024-09-15T21:34:49.957 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51444.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.51444.log.gz 2024-09-15T21:34:49.957 INFO:teuthology.orchestra.run.smithi073.stderr: /var/log/ceph/ceph-client.admin.38287.log 2024-09-15T21:34:49.957 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50772.log 2024-09-15T21:34:49.957 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41847.log.gz 2024-09-15T21:34:49.957 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38287.log.gz 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47664.log 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50562.log 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50772.log.gz 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47664.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47664.log.gz 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47496.log 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38953.log 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50562.log.gz 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47496.log.gz 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41573.log 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47874.log 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38953.log.gz 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41573.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41573.log.gz 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61376.log 2024-09-15T21:34:49.958 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50856.log 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47874.log.gz 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61376.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61376.log.gz 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63814.log 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48756.log 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50856.log.gz 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68099.log 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63814.log.gz 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56862.log 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48756.log.gz 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49680.log 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68099.log.gz 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71124.log 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56862.log.gz 2024-09-15T21:34:49.959 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60788.log 2024-09-15T21:34:49.960 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49680.log.gz 2024-09-15T21:34:49.960 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63268.log 2024-09-15T21:34:49.960 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71124.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71124.log.gz 2024-09-15T21:34:49.960 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41656.log 2024-09-15T21:34:49.960 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60788.log.gz 2024-09-15T21:34:49.960 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42473.log 2024-09-15T21:34:49.960 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63268.log.gz 2024-09-15T21:34:49.960 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71544.log 2024-09-15T21:34:49.960 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41656.log.gz 2024-09-15T21:34:49.960 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49008.log 2024-09-15T21:34:49.960 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42473.log.gz 2024-09-15T21:34:49.960 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56106.log 2024-09-15T21:34:49.961 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71544.log.gz 2024-09-15T21:34:49.961 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40299.log 2024-09-15T21:34:49.961 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49008.log.gz 2024-09-15T21:34:49.961 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37241.log 2024-09-15T21:34:49.961 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56106.log.gz 2024-09-15T21:34:49.961 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40299.log.gz 2024-09-15T21:34:49.961 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69780.log 2024-09-15T21:34:49.961 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49218.log 2024-09-15T21:34:49.961 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37241.log.gz 2024-09-15T21:34:49.961 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46572.log 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69780.log.gz 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44069.log 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49218.log.gz 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57663.log 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46572.log.gz 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36396.log 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44069.log.gz 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.57663.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.64696.log 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57663.log.gz 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.36396.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57494.log 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36396.log.gz 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68057.log 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64696.log.gz 2024-09-15T21:34:49.962 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.57494.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.69696.log 2024-09-15T21:34:49.963 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57494.log.gz 2024-09-15T21:34:49.963 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44462.log 2024-09-15T21:34:49.963 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68057.log.gz 2024-09-15T21:34:49.963 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35042.log 2024-09-15T21:34:49.963 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69696.log.gz 2024-09-15T21:34:49.963 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35370.log 2024-09-15T21:34:49.963 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44462.log.gz 2024-09-15T21:34:49.963 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60998.log 2024-09-15T21:34:49.963 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35042.log.gz 2024-09-15T21:34:49.963 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55140.log 2024-09-15T21:34:49.963 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35370.log.gz 2024-09-15T21:34:49.963 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60998.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60998.log.gz 2024-09-15T21:34:49.963 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55140.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55140.log.gz 2024-09-15T21:34:49.964 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65410.log 2024-09-15T21:34:49.964 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37315.log 2024-09-15T21:34:49.964 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61670.log 2024-09-15T21:34:49.964 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65410.log.gz 2024-09-15T21:34:49.964 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68183.log 2024-09-15T21:34:49.964 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37315.log.gz 2024-09-15T21:34:49.964 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63982.log 2024-09-15T21:34:49.964 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61670.log.gz 2024-09-15T21:34:49.964 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37648.log 2024-09-15T21:34:49.964 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68183.log.gz 2024-09-15T21:34:49.964 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60030.log 2024-09-15T21:34:49.964 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63982.log.gz 2024-09-15T21:34:49.964 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43271.log 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37648.log.gz 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58854.log 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60030.log.gz 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57621.log 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43271.log.gz 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47076.log 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58854.log.gz 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63352.log 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57621.log.gz 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68981.log 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47076.log.gz 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69108.log 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63352.log.gz 2024-09-15T21:34:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68981.log.gz 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70620.log 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32291.log 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69108.log.gz 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67426.log 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70620.log.gz 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48336.log 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32291.log.gz 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43145.log 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67426.log.gz 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64024.log 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48336.log.gz 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66502.log 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43145.log.gz 2024-09-15T21:34:49.966 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59988.log 2024-09-15T21:34:49.967 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64024.log.gz 2024-09-15T21:34:49.967 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52914.log 2024-09-15T21:34:49.967 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66502.log.gz 2024-09-15T21:34:49.967 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67847.log 2024-09-15T21:34:49.967 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59988.log.gz 2024-09-15T21:34:49.967 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69486.log 2024-09-15T21:34:49.967 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52914.log.gz 2024-09-15T21:34:49.967 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71250.log 2024-09-15T21:34:49.967 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67847.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67847.log.gz 2024-09-15T21:34:49.967 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69486.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69486.log.gz 2024-09-15T21:34:49.967 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71964.log 2024-09-15T21:34:49.967 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69528.log 2024-09-15T21:34:49.967 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71250.log.gz 2024-09-15T21:34:49.968 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53586.log 2024-09-15T21:34:49.968 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71964.log.gz 2024-09-15T21:34:49.968 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54510.log 2024-09-15T21:34:49.968 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69528.log.gz 2024-09-15T21:34:49.968 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45054.log 2024-09-15T21:34:49.968 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53586.log.gz 2024-09-15T21:34:49.968 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52704.log 2024-09-15T21:34:49.968 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54510.log.gz 2024-09-15T21:34:49.968 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67552.log 2024-09-15T21:34:49.968 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45054.log.gz 2024-09-15T21:34:49.968 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67342.log 2024-09-15T21:34:49.968 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52704.log.gz 2024-09-15T21:34:49.968 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58350.log 2024-09-15T21:34:49.969 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67552.log.gz 2024-09-15T21:34:49.969 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67931.log 2024-09-15T21:34:49.969 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67342.log.gz 2024-09-15T21:34:49.969 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58350.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.58350.log.gz 2024-09-15T21:34:49.969 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44758.log 2024-09-15T21:34:49.969 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64990.log 2024-09-15T21:34:49.969 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67931.log.gz 2024-09-15T21:34:49.969 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35746.log 2024-09-15T21:34:49.969 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44758.log.gz 2024-09-15T21:34:49.969 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54636.log 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64990.log.gz 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43627.log 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35746.log.gz 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47286.log 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54636.log.gz 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48462.log 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43627.log.gz 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49764.log 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47286.log.gz 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41511.log 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48462.log.gz 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62512.log 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49764.log.gz 2024-09-15T21:34:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41511.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.41511.log.gz --verbose 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr: -- /var/log/ceph/ceph-client.admin.50436.log 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43313.log 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62512.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62512.log.gz 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36796.log 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50436.log.gz 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51822.log 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43313.log.gz 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58560.log 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36796.log.gz 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50730.log 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51822.log.gz 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51738.log 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58560.log.gz 2024-09-15T21:34:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57368.log 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50730.log.gz 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71292.log 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51738.log.gz 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57368.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.57368.log.gz -- 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr: /var/log/ceph/ceph-client.admin.42160.log 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43794.log 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71292.log.gz 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54468.log 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42160.log.gz 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49428.log 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43794.log.gz 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48588.log 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54468.log.gz 2024-09-15T21:34:49.972 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48966.log 2024-09-15T21:34:49.973 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49428.log.gz 2024-09-15T21:34:49.973 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66586.log 2024-09-15T21:34:49.973 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48588.log.gz 2024-09-15T21:34:49.973 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60576.log 2024-09-15T21:34:49.973 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48966.log.gz 2024-09-15T21:34:49.973 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67258.log 2024-09-15T21:34:49.973 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66586.log.gz 2024-09-15T21:34:49.973 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63772.log 2024-09-15T21:34:49.973 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60576.log.gz 2024-09-15T21:34:49.973 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46068.log 2024-09-15T21:34:49.973 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67258.log.gz 2024-09-15T21:34:49.973 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43565.log 2024-09-15T21:34:49.973 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63772.log.gz 2024-09-15T21:34:49.974 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70998.log 2024-09-15T21:34:49.974 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46068.log.gz 2024-09-15T21:34:49.974 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52368.log 2024-09-15T21:34:49.974 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43565.log.gz 2024-09-15T21:34:49.974 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70998.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70998.log.gz 2024-09-15T21:34:49.974 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71796.log 2024-09-15T21:34:49.974 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66376.log 2024-09-15T21:34:49.974 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52368.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52368.log.gz 2024-09-15T21:34:49.974 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55476.log 2024-09-15T21:34:49.975 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71796.log.gz 2024-09-15T21:34:49.975 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47790.log 2024-09-15T21:34:49.975 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66376.log.gz 2024-09-15T21:34:49.975 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60830.log 2024-09-15T21:34:49.975 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55476.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55476.log.gz 2024-09-15T21:34:49.975 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51696.log 2024-09-15T21:34:49.975 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47790.log.gz 2024-09-15T21:34:49.975 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67805.log 2024-09-15T21:34:49.975 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60830.log.gz 2024-09-15T21:34:49.975 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36922.log 2024-09-15T21:34:49.975 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51696.log.gz 2024-09-15T21:34:49.975 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56694.log 2024-09-15T21:34:49.976 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67805.log.gz 2024-09-15T21:34:49.976 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.36922.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.71922.log 2024-09-15T21:34:49.976 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36922.log.gz 2024-09-15T21:34:49.976 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63058.log 2024-09-15T21:34:49.976 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56694.log.gz 2024-09-15T21:34:49.977 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71922.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71922.log.gz 2024-09-15T21:34:49.977 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50478.log 2024-09-15T21:34:49.977 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56274.log 2024-09-15T21:34:49.977 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63058.log.gz 2024-09-15T21:34:49.977 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68813.log 2024-09-15T21:34:49.977 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50478.log.gz 2024-09-15T21:34:49.977 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57835.log 2024-09-15T21:34:49.977 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56274.log.gz 2024-09-15T21:34:49.977 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70578.log 2024-09-15T21:34:49.977 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68813.log.gz 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36126.log 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57835.log.gz 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44943.log 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70578.log.gz 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36126.log.gz 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47832.log 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35832.log 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44943.log.gz 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48924.log 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47832.log.gz 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65116.log 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35832.log.gz 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36982.log 2024-09-15T21:34:49.978 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48924.log.gz 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55938.log 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65116.log.gz 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38398.log 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36982.log.gz 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55644.log 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55938.log.gz 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68855.log 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38398.log.gz 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38583.log 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55644.log.gz 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53334.log 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68855.log.gz 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38583.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38583.log.gz -5 2024-09-15T21:34:49.979 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59274.log 2024-09-15T21:34:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38657.log 2024-09-15T21:34:49.980 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53334.log.gz 2024-09-15T21:34:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49386.log 2024-09-15T21:34:49.980 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59274.log.gz 2024-09-15T21:34:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61040.log 2024-09-15T21:34:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38657.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38657.log.gz 2024-09-15T21:34:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35706.log 2024-09-15T21:34:49.980 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49386.log.gz 2024-09-15T21:34:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71040.log 2024-09-15T21:34:49.980 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61040.log.gz 2024-09-15T21:34:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62132.log 2024-09-15T21:34:49.980 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35706.log.gz 2024-09-15T21:34:49.980 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60661.log 2024-09-15T21:34:49.981 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71040.log.gz 2024-09-15T21:34:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67048.log 2024-09-15T21:34:49.981 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62132.log.gz 2024-09-15T21:34:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51066.log 2024-09-15T21:34:49.981 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60661.log.gz 2024-09-15T21:34:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65662.log 2024-09-15T21:34:49.981 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67048.log.gz 2024-09-15T21:34:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42557.log 2024-09-15T21:34:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51066.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51066.log.gz 2024-09-15T21:34:49.981 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32494.log 2024-09-15T21:34:49.981 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65662.log.gz 2024-09-15T21:34:49.982 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42557.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42557.log.gz 2024-09-15T21:34:49.982 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62680.log 2024-09-15T21:34:49.982 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58602.log 2024-09-15T21:34:49.982 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32494.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32494.log.gz 2024-09-15T21:34:49.982 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70368.log 2024-09-15T21:34:49.982 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62680.log.gz 2024-09-15T21:34:49.982 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54342.log 2024-09-15T21:34:49.982 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58602.log.gz 2024-09-15T21:34:49.982 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49722.log 2024-09-15T21:34:49.982 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70368.log.gz 2024-09-15T21:34:49.982 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65032.log 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54342.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54342.log.gz 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38620.log 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49722.log.gz 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57242.log 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65032.log.gz 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59820.log 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38620.log.gz 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62217.log 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57242.log.gz 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57030.log 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59820.log.gz 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49554.log 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62217.log.gz 2024-09-15T21:34:49.983 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71082.log 2024-09-15T21:34:49.984 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57030.log.gz 2024-09-15T21:34:49.984 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55392.log 2024-09-15T21:34:49.984 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49554.log.gz 2024-09-15T21:34:49.984 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42077.log 2024-09-15T21:34:49.984 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71082.log.gz 2024-09-15T21:34:49.984 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40130.log 2024-09-15T21:34:49.984 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55392.log.gz 2024-09-15T21:34:49.984 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64402.log 2024-09-15T21:34:49.984 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42077.log.gz 2024-09-15T21:34:49.984 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44277.log 2024-09-15T21:34:49.984 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40130.log.gz 2024-09-15T21:34:49.984 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52578.log 2024-09-15T21:34:49.984 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64402.log.gz 2024-09-15T21:34:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44277.log.gz 2024-09-15T21:34:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70872.log 2024-09-15T21:34:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40531.log 2024-09-15T21:34:49.985 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52578.log.gz 2024-09-15T21:34:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37352.log 2024-09-15T21:34:49.985 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70872.log.gz 2024-09-15T21:34:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51234.log 2024-09-15T21:34:49.985 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40531.log.gz 2024-09-15T21:34:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65200.log 2024-09-15T21:34:49.985 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37352.log.gz 2024-09-15T21:34:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55014.log 2024-09-15T21:34:49.985 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51234.log.gz 2024-09-15T21:34:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.65200.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46698.log 2024-09-15T21:34:49.986 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65200.log.gz 2024-09-15T21:34:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61124.log 2024-09-15T21:34:49.986 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55014.log.gz 2024-09-15T21:34:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63520.log 2024-09-15T21:34:49.986 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46698.log.gz 2024-09-15T21:34:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47958.log 2024-09-15T21:34:49.986 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61124.log.gz 2024-09-15T21:34:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56820.log 2024-09-15T21:34:49.986 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63520.log.gz 2024-09-15T21:34:49.986 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61922.log 2024-09-15T21:34:49.986 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47958.log.gz 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41993.log 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56820.log.gz 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71670.log 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61922.log.gz 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41993.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41993.log.gz 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62722.log 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71670.log.gz 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66460.log 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.62722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48714.log 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62722.log.gz 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49890.log 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66460.log.gz 2024-09-15T21:34:49.987 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41931.log 2024-09-15T21:34:49.988 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48714.log.gz 2024-09-15T21:34:49.988 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32331.log 2024-09-15T21:34:49.988 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49890.log.gz 2024-09-15T21:34:49.988 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51402.log 2024-09-15T21:34:49.988 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41931.log.gz 2024-09-15T21:34:49.988 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70494.log 2024-09-15T21:34:49.988 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32331.log.gz 2024-09-15T21:34:49.988 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43626.log 2024-09-15T21:34:49.988 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51402.log.gz 2024-09-15T21:34:49.988 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43859.log 2024-09-15T21:34:49.988 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70494.log.gz 2024-09-15T21:34:49.988 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67510.log 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43626.log.gz 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53418.log 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43859.log.gz 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44425.log 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67510.log.gz 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46908.log 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53418.log.gz 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57156.log 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44425.log.gz 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68687.log 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46908.log.gz 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57156.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.57156.log.gz 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.48126.log 2024-09-15T21:34:49.989 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44906.log 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68687.log.gz 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58938.log 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48126.log.gz 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71502.log 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44906.log.gz 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46404.log 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58938.log.gz 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60240.log 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71502.log.gz 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59022.log 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46404.log.gz 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49092.log 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60240.log.gz 2024-09-15T21:34:49.990 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38435.log 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59022.log.gz 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37611.log 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49092.log.gz 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63646.log 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38435.log.gz 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49344.log 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37611.log.gz 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56358.log 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63646.log.gz 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67973.log 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49344.log.gz 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64066.log 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56358.log.gz 2024-09-15T21:34:49.991 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44980.log 2024-09-15T21:34:49.992 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67973.log.gz 2024-09-15T21:34:49.992 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54552.log 2024-09-15T21:34:49.992 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64066.log.gz 2024-09-15T21:34:49.992 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44980.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.48084.log -- replaced with /var/log/ceph/ceph-client.admin.44980.log.gz 2024-09-15T21:34:49.992 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-09-15T21:34:49.992 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60450.log 2024-09-15T21:34:49.992 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54552.log.gz 2024-09-15T21:34:49.992 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66964.log 2024-09-15T21:34:49.992 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48084.log.gz 2024-09-15T21:34:49.992 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65704.log 2024-09-15T21:34:49.992 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60450.log.gz 2024-09-15T21:34:49.992 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62386.log 2024-09-15T21:34:49.992 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66964.log.gz 2024-09-15T21:34:49.993 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50142.log 2024-09-15T21:34:49.993 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65704.log.gz 2024-09-15T21:34:49.993 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37907.log 2024-09-15T21:34:49.993 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62386.log.gz 2024-09-15T21:34:49.993 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50142.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.50142.log.gz /var/log/ceph/ceph-client.admin.36480.log 2024-09-15T21:34:49.993 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-09-15T21:34:49.993 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37056.log 2024-09-15T21:34:49.993 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37907.log.gz 2024-09-15T21:34:49.993 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60872.log 2024-09-15T21:34:49.993 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36480.log.gz 2024-09-15T21:34:49.993 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60703.log 2024-09-15T21:34:49.994 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37056.log.gz 2024-09-15T21:34:49.994 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60872.log.gz 2024-09-15T21:34:49.994 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62090.log 2024-09-15T21:34:49.994 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58392.log 2024-09-15T21:34:49.994 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60703.log.gz 2024-09-15T21:34:49.994 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.62090.log: /var/log/ceph/ceph-client.admin.53838.log 2024-09-15T21:34:49.994 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62090.log.gz 2024-09-15T21:34:49.994 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56778.log 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58392.log.gz 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53838.log.gz 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54132.log 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58011.log 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56778.log.gz 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65578.log 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54132.log.gz 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47412.log 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58011.log.gz 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65578.log.gzgzip 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38546.log 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53796.log 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47412.log.gz 2024-09-15T21:34:49.995 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44499.log 2024-09-15T21:34:49.996 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38546.log.gz 2024-09-15T21:34:49.996 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58980.log 2024-09-15T21:34:49.996 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53796.log.gz 2024-09-15T21:34:49.996 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61628.log 2024-09-15T21:34:49.996 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44499.log.gz 2024-09-15T21:34:49.996 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38176.log 2024-09-15T21:34:49.996 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58980.log.gz 2024-09-15T21:34:49.996 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61628.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61628.log.gz 2024-09-15T21:34:49.996 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45110.log 2024-09-15T21:34:49.996 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48546.log 2024-09-15T21:34:49.996 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38176.log.gz 2024-09-15T21:34:49.996 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70242.log 2024-09-15T21:34:49.996 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45110.log.gz 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72004.log 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48546.log.gz 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54174.log 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70242.log.gz 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.72004.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72004.log.gz 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36042.log 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51570.log 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54174.log.gz 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53754.log 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36042.log.gz 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50268.log 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51570.log.gz 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49302.log 2024-09-15T21:34:49.997 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53754.log.gz 2024-09-15T21:34:49.998 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52200.log 2024-09-15T21:34:49.998 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50268.log.gz 2024-09-15T21:34:49.998 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49302.log.gz 2024-09-15T21:34:49.998 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50688.log 2024-09-15T21:34:49.998 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69612.log 2024-09-15T21:34:49.998 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52200.log.gz 2024-09-15T21:34:49.998 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52326.log 2024-09-15T21:34:49.998 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50688.log.gz 2024-09-15T21:34:49.998 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55182.log 2024-09-15T21:34:49.998 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69612.log.gz 2024-09-15T21:34:49.998 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56148.log 2024-09-15T21:34:49.998 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52326.log.gz 2024-09-15T21:34:49.998 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56316.log 2024-09-15T21:34:49.999 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55182.log.gz 2024-09-15T21:34:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35745.log 2024-09-15T21:34:49.999 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56148.log.gz 2024-09-15T21:34:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35412.log 2024-09-15T21:34:49.999 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56316.log.gz 2024-09-15T21:34:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40131.log 2024-09-15T21:34:49.999 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35745.log.gz 2024-09-15T21:34:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55560.log 2024-09-15T21:34:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35412.log.gz 2024-09-15T21:34:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63184.log 2024-09-15T21:34:49.999 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40131.log.gz 2024-09-15T21:34:49.999 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35210.log 2024-09-15T21:34:49.999 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55560.log.gz 2024-09-15T21:34:50.000 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60914.log 2024-09-15T21:34:50.000 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63184.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63184.log.gz 2024-09-15T21:34:50.000 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35493.log 2024-09-15T21:34:50.000 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35210.log.gz 2024-09-15T21:34:50.000 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47916.log 2024-09-15T21:34:50.000 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60914.log.gz 2024-09-15T21:34:50.000 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71166.log 2024-09-15T21:34:50.000 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35493.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35493.log.gz 2024-09-15T21:34:50.000 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60324.log 2024-09-15T21:34:50.000 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47916.log.gz 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49932.log 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71166.log.gz 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71334.log 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60324.log.gz 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66040.log 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49932.log.gz 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42515.log 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71334.log.gz 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47580.log 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66040.log.gz 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42515.log: /var/log/ceph/ceph-client.admin.55098.log 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42515.log.gz 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50520.log 2024-09-15T21:34:50.001 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47580.log.gz 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55098.log.gz 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38102.log 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58137.log 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50520.log.gz 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62344.log 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38102.log.gz 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56946.log 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58137.log.gz 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62344.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62344.log.gz 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36297.log 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70200.log 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56946.log.gz 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36297.log.gz 2024-09-15T21:34:50.002 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52998.log 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68897.log 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70200.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70200.log.gz 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38916.log 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52998.log.gz 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69318.log 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68897.log.gz 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38916.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43523.log 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38916.log.gz 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70830.log 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69318.log.gz 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59568.log 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43523.log.gz 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr: 96.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-15T21:34:50.003 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36479.log 2024-09-15T21:34:50.004 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70830.log.gz 2024-09-15T21:34:50.004 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65914.log 2024-09-15T21:34:50.004 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59568.log.gz 2024-09-15T21:34:50.004 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62428.log 2024-09-15T21:34:50.004 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36479.log.gz 2024-09-15T21:34:50.004 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53166.log 2024-09-15T21:34:50.004 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65914.log.gz 2024-09-15T21:34:50.004 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37722.log 2024-09-15T21:34:50.004 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62428.log.gz 2024-09-15T21:34:50.004 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66670.log 2024-09-15T21:34:50.004 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53166.log.gz 2024-09-15T21:34:50.004 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48672.log 2024-09-15T21:34:50.004 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37722.log.gz 2024-09-15T21:34:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58476.log 2024-09-15T21:34:50.005 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66670.log.gz 2024-09-15T21:34:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48672.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-15T21:34:50.005 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48672.log.gz 2024-09-15T21:34:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61880.log 2024-09-15T21:34:50.005 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58476.log.gz 2024-09-15T21:34:50.005 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35619.log 2024-09-15T21:34:50.006 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58518.log 2024-09-15T21:34:50.006 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61880.log.gz 2024-09-15T21:34:50.006 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42389.log 2024-09-15T21:34:50.006 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35619.log.gz 2024-09-15T21:34:50.006 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51192.log 2024-09-15T21:34:50.006 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58518.log.gz 2024-09-15T21:34:50.006 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67595.log 2024-09-15T21:34:50.007 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42389.log.gz 2024-09-15T21:34:50.007 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36084.log 2024-09-15T21:34:50.007 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51192.log.gz 2024-09-15T21:34:50.007 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46530.log 2024-09-15T21:34:50.007 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67595.log.gz 2024-09-15T21:34:50.007 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67679.log 2024-09-15T21:34:50.007 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36084.log.gz 2024-09-15T21:34:50.007 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68351.log 2024-09-15T21:34:50.007 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46530.log.gz 2024-09-15T21:34:50.008 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43229.log 2024-09-15T21:34:50.008 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67679.log.gz 2024-09-15T21:34:50.008 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37574.log 2024-09-15T21:34:50.008 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68351.log.gz 2024-09-15T21:34:50.008 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61250.log 2024-09-15T21:34:50.008 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43229.log.gz 2024-09-15T21:34:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56988.log 2024-09-15T21:34:50.009 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37574.log.gz 2024-09-15T21:34:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60619.log 2024-09-15T21:34:50.009 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61250.log.gz 2024-09-15T21:34:50.009 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56988.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.56988.log.gz /var/log/ceph/ceph-client.admin.65368.log 2024-09-15T21:34:50.010 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-09-15T21:34:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42935.log 2024-09-15T21:34:50.010 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60619.log.gz 2024-09-15T21:34:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62932.log 2024-09-15T21:34:50.010 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65368.log.gz 2024-09-15T21:34:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55728.log 2024-09-15T21:34:50.010 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42935.log.gz 2024-09-15T21:34:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64612.log 2024-09-15T21:34:50.010 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62932.log.gz 2024-09-15T21:34:50.010 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47202.log 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55728.log.gz 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53502.log 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64612.log.gz 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38768.log 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47202.log.gz 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66292.log 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53502.log.gz 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44240.log 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38768.log.gz 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50100.log 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66292.log.gz 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44240.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44240.log.gz 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61082.log 2024-09-15T21:34:50.011 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61796.log 2024-09-15T21:34:50.012 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50100.log.gz 2024-09-15T21:34:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68603.log 2024-09-15T21:34:50.012 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61082.log.gz 2024-09-15T21:34:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68645.log 2024-09-15T21:34:50.012 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61796.log.gz 2024-09-15T21:34:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38879.log 2024-09-15T21:34:50.012 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68603.log.gz 2024-09-15T21:34:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61292.log 2024-09-15T21:34:50.012 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68645.log.gz 2024-09-15T21:34:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37833.log 2024-09-15T21:34:50.012 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38879.log.gz 2024-09-15T21:34:50.012 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71712.log 2024-09-15T21:34:50.012 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61292.log.gz 2024-09-15T21:34:50.013 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56526.log 2024-09-15T21:34:50.013 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37833.log.gz 2024-09-15T21:34:50.013 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46950.log 2024-09-15T21:34:50.013 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71712.log.gz 2024-09-15T21:34:50.013 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44314.log 2024-09-15T21:34:50.013 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56526.log.gz 2024-09-15T21:34:50.013 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48378.log 2024-09-15T21:34:50.013 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46950.log.gz 2024-09-15T21:34:50.013 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57578.log 2024-09-15T21:34:50.013 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44314.log.gz 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46236.log 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48378.log.gz 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60956.log 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57578.log.gz 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46236.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46236.log.gz 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69738.log 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65158.log 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60956.log.gz 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71586.log 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69738.log.gz 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64192.log 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65158.log.gz 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71586.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71586.log.gz 2024-09-15T21:34:50.014 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52536.log 2024-09-15T21:34:50.015 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61166.log 2024-09-15T21:34:50.015 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64192.log.gz 2024-09-15T21:34:50.015 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52830.log 2024-09-15T21:34:50.015 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52536.log.gz 2024-09-15T21:34:50.015 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35620.log 2024-09-15T21:34:50.015 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61166.log.gz 2024-09-15T21:34:50.015 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58095.log 2024-09-15T21:34:50.015 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52830.log.gz 2024-09-15T21:34:50.015 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61544.log 2024-09-15T21:34:50.015 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35620.log.gz 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51948.log 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58095.log.gz 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49470.log 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61544.log.gz 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51948.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.47454.log -- replaced with /var/log/ceph/ceph-client.admin.51948.log.gz 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48630.log 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49470.log.gz 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42225.log 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47454.log.gz 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42893.log 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48630.log.gz 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42225.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.58728.log 2024-09-15T21:34:50.016 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42225.log.gz 2024-09-15T21:34:50.017 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46446.log 2024-09-15T21:34:50.017 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42893.log.gz 2024-09-15T21:34:50.017 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38065.log 2024-09-15T21:34:50.017 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58728.log.gz 2024-09-15T21:34:50.017 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55896.log 2024-09-15T21:34:50.017 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46446.log.gz 2024-09-15T21:34:50.017 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53670.log 2024-09-15T21:34:50.017 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38065.log.gz 2024-09-15T21:34:50.017 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70536.log 2024-09-15T21:34:50.018 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55896.log.gz 2024-09-15T21:34:50.018 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67132.log 2024-09-15T21:34:50.018 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53670.log.gz 2024-09-15T21:34:50.018 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62890.log 2024-09-15T21:34:50.018 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70536.log.gz 2024-09-15T21:34:50.018 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44721.log 2024-09-15T21:34:50.018 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67132.log.gz 2024-09-15T21:34:50.018 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43943.log 2024-09-15T21:34:50.018 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62890.log.gz 2024-09-15T21:34:50.018 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43985.log 2024-09-15T21:34:50.018 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44721.log.gz 2024-09-15T21:34:50.018 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44092.log 2024-09-15T21:34:50.018 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43943.log.gz 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67384.log 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43985.log.gz 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69192.log 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44092.log.gz 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47160.log 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67384.log.gz 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49974.log 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69192.log.gz 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47160.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64654.log 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47160.log.gz 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46824.log 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49974.log.gz 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44869.log 2024-09-15T21:34:50.019 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64654.log.gz 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55224.log 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46824.log.gz 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47538.log 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44869.log.gz 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71754.log 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55224.log.gz 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53460.log 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47538.log.gz 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64906.log 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71754.log.gz 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53460.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53460.log.gz 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69444.log 2024-09-15T21:34:50.020 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64906.log.gz 2024-09-15T21:34:50.021 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35874.log 2024-09-15T21:34:50.021 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.69444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35286.log 2024-09-15T21:34:50.021 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69444.log.gz 2024-09-15T21:34:50.021 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59442.log 2024-09-15T21:34:50.021 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35874.log.gz 2024-09-15T21:34:50.021 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42767.log 2024-09-15T21:34:50.021 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35286.log.gz 2024-09-15T21:34:50.021 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65452.log 2024-09-15T21:34:50.021 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59442.log.gz 2024-09-15T21:34:50.021 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38005.log 2024-09-15T21:34:50.022 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42767.log.gz 2024-09-15T21:34:50.022 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48504.log 2024-09-15T21:34:50.022 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65452.log.gz 2024-09-15T21:34:50.022 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41469.log 2024-09-15T21:34:50.022 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38005.log.gz 2024-09-15T21:34:50.022 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48504.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48504.log.gz 2024-09-15T21:34:50.022 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41740.log 2024-09-15T21:34:50.022 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51150.log 2024-09-15T21:34:50.022 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41469.log.gz 2024-09-15T21:34:50.022 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51906.log 2024-09-15T21:34:50.022 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41740.log.gz 2024-09-15T21:34:50.023 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55686.log 2024-09-15T21:34:50.023 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51150.log.gz 2024-09-15T21:34:50.023 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70410.log 2024-09-15T21:34:50.023 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51906.log.gz 2024-09-15T21:34:50.023 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37500.log 2024-09-15T21:34:50.023 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55686.log.gz 2024-09-15T21:34:50.023 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38139.log 2024-09-15T21:34:50.023 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70410.log.gz 2024-09-15T21:34:50.023 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44536.log 2024-09-15T21:34:50.023 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37500.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37500.log.gz 2024-09-15T21:34:50.023 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52410.log 2024-09-15T21:34:50.023 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38139.log.gz 2024-09-15T21:34:50.023 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51654.log 2024-09-15T21:34:50.024 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44536.log.gz 2024-09-15T21:34:50.024 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69402.log 2024-09-15T21:34:50.024 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52410.log.gz 2024-09-15T21:34:50.024 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32430.log 2024-09-15T21:34:50.024 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51654.log.gz 2024-09-15T21:34:50.024 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51780.log 2024-09-15T21:34:50.024 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69402.log.gz 2024-09-15T21:34:50.024 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61502.log 2024-09-15T21:34:50.024 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32430.log.gz 2024-09-15T21:34:50.024 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44573.log 2024-09-15T21:34:50.024 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51780.log.gz 2024-09-15T21:34:50.024 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64822.log 2024-09-15T21:34:50.024 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61502.log.gz 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58770.log 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44573.log.gz 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59652.log 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64822.log.gz 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63016.log 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58770.log.gz 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59652.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59652.log.gz 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38805.log 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38990.log 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63016.log.gz 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66334.log 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38805.log.gz 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66124.log 2024-09-15T21:34:50.025 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38990.log.gz 2024-09-15T21:34:50.026 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50604.log 2024-09-15T21:34:50.026 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66334.log.gz 2024-09-15T21:34:50.026 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66124.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.66124.log.gz /var/log/ceph/ceph-client.admin.40489.log 2024-09-15T21:34:50.026 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-09-15T21:34:50.026 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49848.log 2024-09-15T21:34:50.026 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50604.log.gz 2024-09-15T21:34:50.026 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35580.log 2024-09-15T21:34:50.026 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40489.log.gz 2024-09-15T21:34:50.026 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69024.log 2024-09-15T21:34:50.026 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49848.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49848.log.gz 2024-09-15T21:34:50.026 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42431.log 2024-09-15T21:34:50.026 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35580.log.gz 2024-09-15T21:34:50.026 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43187.log 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69024.log.gz 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48882.log 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42431.log.gz 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56652.log 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43187.log.gz 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44684.log 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48882.log.gz 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53376.log 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56652.log.gz 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36754.log 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44684.log.gz 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36712.log 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53376.log.gz 2024-09-15T21:34:50.027 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48798.log 2024-09-15T21:34:50.028 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36754.log.gz 2024-09-15T21:34:50.028 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53880.log 2024-09-15T21:34:50.028 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36712.log.gz 2024-09-15T21:34:50.028 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48798.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.48798.log.gz -- 2024-09-15T21:34:50.028 INFO:teuthology.orchestra.run.smithi073.stderr: /var/log/ceph/ceph-client.admin.58263.log 2024-09-15T21:34:50.028 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70914.log 2024-09-15T21:34:50.028 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53880.log.gz 2024-09-15T21:34:50.028 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46782.log 2024-09-15T21:34:50.028 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58263.log.gz 2024-09-15T21:34:50.028 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62260.log 2024-09-15T21:34:50.028 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70914.log.gz 2024-09-15T21:34:50.028 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46782.log: -- /var/log/ceph/ceph-client.admin.64486.log 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46782.log.gz 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46488.log 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62260.log.gz 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43710.log 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64486.log.gz 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69822.log 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46488.log.gz 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57410.log 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43710.log.gz 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70662.log 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69822.log.gz 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57410.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57410.log.gz 2024-09-15T21:34:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46614.log 2024-09-15T21:34:50.030 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70662.log.gz 2024-09-15T21:34:50.030 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35492.log 2024-09-15T21:34:50.030 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66922.log 2024-09-15T21:34:50.030 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46614.log.gz 2024-09-15T21:34:50.030 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.35492.log: /var/log/ceph/ceph-client.admin.71418.log 2024-09-15T21:34:50.030 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35492.log.gz 2024-09-15T21:34:50.030 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41572.log 2024-09-15T21:34:50.030 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66922.log.gz 2024-09-15T21:34:50.030 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71418.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71418.log.gz 2024-09-15T21:34:50.030 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59526.log 2024-09-15T21:34:50.030 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60198.log 2024-09-15T21:34:50.031 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41572.log.gz 2024-09-15T21:34:50.031 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47034.log 2024-09-15T21:34:50.031 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59526.log.gz 2024-09-15T21:34:50.031 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36320.log 2024-09-15T21:34:50.031 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60198.log.gz 2024-09-15T21:34:50.031 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47034.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.42248.log 2024-09-15T21:34:50.031 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47034.log.gz 2024-09-15T21:34:50.031 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65536.log 2024-09-15T21:34:50.031 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36320.log.gz 2024-09-15T21:34:50.031 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65746.log 2024-09-15T21:34:50.031 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42248.log.gz 2024-09-15T21:34:50.031 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62596.log 2024-09-15T21:34:50.032 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65536.log.gz 2024-09-15T21:34:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62974.log 2024-09-15T21:34:50.032 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65746.log.gz 2024-09-15T21:34:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53544.log 2024-09-15T21:34:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62596.log.gz 2024-09-15T21:34:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47748.log 2024-09-15T21:34:50.032 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62974.log.gz 2024-09-15T21:34:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53544.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53544.log.gz 2024-09-15T21:34:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52662.log 2024-09-15T21:34:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45017.log 2024-09-15T21:34:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47748.log.gz 2024-09-15T21:34:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53208.log 2024-09-15T21:34:50.032 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52662.log.gz 2024-09-15T21:34:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.45017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40214.log 2024-09-15T21:34:50.033 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45017.log.gz 2024-09-15T21:34:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53712.log 2024-09-15T21:34:50.033 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53208.log.gz 2024-09-15T21:34:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51276.log 2024-09-15T21:34:50.033 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40214.log.gz 2024-09-15T21:34:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64234.log 2024-09-15T21:34:50.033 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53712.log.gz 2024-09-15T21:34:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51276.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51276.log.gz 2024-09-15T21:34:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59190.log 2024-09-15T21:34:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64234.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27522.log 2024-09-15T21:34:50.033 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64234.log.gz 2024-09-15T21:34:50.034 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59694.log 2024-09-15T21:34:50.034 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59190.log.gz 2024-09-15T21:34:50.034 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.tmp-client.admin.27522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46152.log 2024-09-15T21:34:50.034 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27522.log.gz 2024-09-15T21:34:50.034 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36945.log 2024-09-15T21:34:50.034 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59694.log.gz 2024-09-15T21:34:50.034 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36647.log 2024-09-15T21:34:50.034 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46152.log.gz 2024-09-15T21:34:50.034 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32131.log 2024-09-15T21:34:50.034 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36945.log.gz 2024-09-15T21:34:50.034 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70116.log 2024-09-15T21:34:50.034 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36647.log.gz 2024-09-15T21:34:50.034 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49806.log 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32131.log.gz 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59106.log 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70116.log.gz 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64360.log 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49806.log.gz 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70032.log 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59106.log.gz 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43019.log 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64360.log.gz 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37537.log 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70032.log.gz 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53628.log 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43019.log.gz 2024-09-15T21:34:50.035 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63394.log 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37537.log.gz 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43711.log 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53628.log.gz 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67763.log 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63394.log.gz 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40047.log 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43711.log.gz 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42725.log 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67763.log.gz 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60072.log 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40047.log.gz 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66544.log 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42725.log.gz 2024-09-15T21:34:50.036 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46740.log 2024-09-15T21:34:50.037 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60072.log.gz 2024-09-15T21:34:50.037 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43397.log 2024-09-15T21:34:50.037 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66544.log.gz 2024-09-15T21:34:50.037 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46740.log.gz 2024-09-15T21:34:50.037 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58812.log 2024-09-15T21:34:50.037 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57326.log 2024-09-15T21:34:50.037 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43397.log.gz 2024-09-15T21:34:50.037 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43061.log 2024-09-15T21:34:50.037 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58812.log.gz 2024-09-15T21:34:50.037 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43439.log 2024-09-15T21:34:50.037 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57326.log.gz 2024-09-15T21:34:50.037 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51864.log 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43061.log.gz 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43439.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43439.log.gz 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50394.log 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51864.log.gz 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69990.log 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50394.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.50394.log.gz 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64738.log 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48210.log 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69990.log.gz 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40405.log 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64738.log.gz 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65620.log 2024-09-15T21:34:50.038 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48210.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48210.log.gz 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40405.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.40405.log.gz --verbose 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr: -- /var/log/ceph/ceph-client.admin.62764.log 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44203.log 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65620.log.gz 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50058.log 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62764.log.gz 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66712.log 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44203.log.gz 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69906.log 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50058.log.gz 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55308.log 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66712.log.gz 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70326.log 2024-09-15T21:34:50.039 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69906.log.gz 2024-09-15T21:34:50.040 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71208.log 2024-09-15T21:34:50.040 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55308.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55308.log.gz 2024-09-15T21:34:50.040 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49260.log 2024-09-15T21:34:50.040 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70326.log.gz 2024-09-15T21:34:50.040 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44166.log 2024-09-15T21:34:50.040 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71208.log.gz 2024-09-15T21:34:50.040 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62470.log 2024-09-15T21:34:50.040 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49260.log.gz 2024-09-15T21:34:50.040 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65788.log 2024-09-15T21:34:50.040 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44166.log.gz 2024-09-15T21:34:50.040 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69234.log 2024-09-15T21:34:50.040 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62470.log.gz 2024-09-15T21:34:50.041 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38694.log 2024-09-15T21:34:50.041 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65788.log.gz 2024-09-15T21:34:50.041 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69234.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.69234.log.gz /var/log/ceph/ceph-client.admin.71628.log 2024-09-15T21:34:50.041 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-09-15T21:34:50.041 INFO:teuthology.orchestra.run.smithi073.stderr: 95.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-15T21:34:50.041 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41992.log 2024-09-15T21:34:50.041 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38694.log.gz 2024-09-15T21:34:50.041 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42161.log 2024-09-15T21:34:50.041 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71628.log.gz 2024-09-15T21:34:50.041 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41992.log: /var/log/ceph/ceph-client.admin.36838.log 2024-09-15T21:34:50.041 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41992.log.gz 2024-09-15T21:34:50.042 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57747.log 2024-09-15T21:34:50.042 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42161.log.gz 2024-09-15T21:34:50.042 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36838.log.gz 2024-09-15T21:34:50.042 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65074.log 2024-09-15T21:34:50.042 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.57747.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46278.log 2024-09-15T21:34:50.042 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57747.log.gz 2024-09-15T21:34:50.042 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65074.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-15T21:34:50.042 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65074.log.gz 2024-09-15T21:34:50.042 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54258.log 2024-09-15T21:34:50.042 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46278.log.gz 2024-09-15T21:34:50.042 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54804.log 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64108.log 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54258.log.gz 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38361.log 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54804.log.gz 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58644.log 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64108.log.gz 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48420.log 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38361.log.gz 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42305.log 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58644.log.gz 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56190.log 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48420.log.gz 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60366.log 2024-09-15T21:34:50.043 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42305.log.gz 2024-09-15T21:34:50.044 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56190.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56190.log.gz 2024-09-15T21:34:50.044 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71838.log 2024-09-15T21:34:50.044 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60366.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60366.log.gz 2024-09-15T21:34:50.044 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61964.log 2024-09-15T21:34:50.044 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71376.log 2024-09-15T21:34:50.044 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71838.log.gz 2024-09-15T21:34:50.044 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61208.log 2024-09-15T21:34:50.044 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61964.log.gz 2024-09-15T21:34:50.044 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64570.log 2024-09-15T21:34:50.044 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71376.log.gz 2024-09-15T21:34:50.044 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61712.log 2024-09-15T21:34:50.044 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61208.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61208.log.gz 2024-09-15T21:34:50.044 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44647.log 2024-09-15T21:34:50.045 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64570.log.gz 2024-09-15T21:34:50.045 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49050.log 2024-09-15T21:34:50.045 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61712.log.gz 2024-09-15T21:34:50.045 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47370.log 2024-09-15T21:34:50.045 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44647.log.gz 2024-09-15T21:34:50.045 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40215.log 2024-09-15T21:34:50.045 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49050.log.gz 2024-09-15T21:34:50.045 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60408.log 2024-09-15T21:34:50.045 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47370.log.gz 2024-09-15T21:34:50.045 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51024.log 2024-09-15T21:34:50.045 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40215.log.gz 2024-09-15T21:34:50.045 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52956.log 2024-09-15T21:34:50.045 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60408.log.gz 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52032.log 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51024.log.gz 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66208.log 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52956.log.gz 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52032.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62048.log 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52032.log.gz 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66208.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.66208.log.gz 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37167.log 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57790.log 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62048.log.gz 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65242.log 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37167.log.gz 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43355.log 2024-09-15T21:34:50.046 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57790.log.gz 2024-09-15T21:34:50.047 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53922.log 2024-09-15T21:34:50.047 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65242.log.gz 2024-09-15T21:34:50.047 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36563.log 2024-09-15T21:34:50.047 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43355.log.gz 2024-09-15T21:34:50.047 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56442.log 2024-09-15T21:34:50.047 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53922.log.gz 2024-09-15T21:34:50.047 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42599.log 2024-09-15T21:34:50.047 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36563.log.gz 2024-09-15T21:34:50.047 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66838.log 2024-09-15T21:34:50.047 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56442.log.gz 2024-09-15T21:34:50.047 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44351.log 2024-09-15T21:34:50.047 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42599.log.gz 2024-09-15T21:34:50.047 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52284.log 2024-09-15T21:34:50.048 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66838.log.gz 2024-09-15T21:34:50.048 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36168.log 2024-09-15T21:34:50.048 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44351.log.gz 2024-09-15T21:34:50.048 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55602.log 2024-09-15T21:34:50.048 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52284.log.gz 2024-09-15T21:34:50.048 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67006.log 2024-09-15T21:34:50.048 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36168.log.gz 2024-09-15T21:34:50.048 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59316.log 2024-09-15T21:34:50.048 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55602.log.gz 2024-09-15T21:34:50.048 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39985.log 2024-09-15T21:34:50.048 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67006.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67006.log.gz 2024-09-15T21:34:50.048 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55266.log 2024-09-15T21:34:50.048 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59316.log.gz 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51108.log 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39985.log.gz 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37796.log 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55266.log.gz 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64780.log 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51108.log.gz 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57927.log 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37796.log.gz 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58179.log 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64780.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64780.log.gz 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5/var/log/ceph/ceph-client.admin.57927.log: --verbose -- /var/log/ceph/ceph-client.admin.54300.log 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57927.log.gz 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60492.log 2024-09-15T21:34:50.049 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58179.log.gz 2024-09-15T21:34:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42977.log 2024-09-15T21:34:50.050 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54300.log.gz 2024-09-15T21:34:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68015.log 2024-09-15T21:34:50.050 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60492.log.gz 2024-09-15T21:34:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46320.log 2024-09-15T21:34:50.050 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42977.log.gz 2024-09-15T21:34:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44388.log 2024-09-15T21:34:50.050 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68015.log.gz 2024-09-15T21:34:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50184.log 2024-09-15T21:34:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46320.log.gz 2024-09-15T21:34:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50982.log 2024-09-15T21:34:50.051 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44388.log.gz 2024-09-15T21:34:50.051 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44129.log 2024-09-15T21:34:50.051 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50184.log.gz 2024-09-15T21:34:50.051 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50310.log 2024-09-15T21:34:50.051 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50982.log.gz 2024-09-15T21:34:50.051 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37204.log 2024-09-15T21:34:50.051 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44129.log.gz 2024-09-15T21:34:50.051 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50310.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-15T21:34:50.051 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50310.log.gz 2024-09-15T21:34:50.051 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42851.log 2024-09-15T21:34:50.051 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37204.log.gz 2024-09-15T21:34:50.051 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71460.log 2024-09-15T21:34:50.051 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56400.log 2024-09-15T21:34:50.052 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42851.log.gz 2024-09-15T21:34:50.052 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.71460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70452.log 2024-09-15T21:34:50.052 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71460.log.gz 2024-09-15T21:34:50.052 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67174.log 2024-09-15T21:34:50.052 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56400.log.gz 2024-09-15T21:34:50.052 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55854.log 2024-09-15T21:34:50.052 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70452.log.gz 2024-09-15T21:34:50.052 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68519.log 2024-09-15T21:34:50.052 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67174.log.gz 2024-09-15T21:34:50.052 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38028.log 2024-09-15T21:34:50.052 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55854.log.gz 2024-09-15T21:34:50.052 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57536.log 2024-09-15T21:34:50.052 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68519.log.gz 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69654.log 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38028.log.gz 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57536.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37019.log 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57536.log.gz 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.69654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54090.log 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69654.log.gz 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61460.log 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37019.log.gz 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66082.log 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54090.log.gz 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54888.log 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61460.log.gz 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62638.log 2024-09-15T21:34:50.053 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66082.log.gz 2024-09-15T21:34:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67090.log 2024-09-15T21:34:50.054 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54888.log.gz 2024-09-15T21:34:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62638.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.62638.log.gz 2024-09-15T21:34:50.054 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49638.log 2024-09-15T21:34:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48840.log 2024-09-15T21:34:50.054 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67090.log.gz 2024-09-15T21:34:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36564.log 2024-09-15T21:34:50.054 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49638.log.gz 2024-09-15T21:34:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68267.log 2024-09-15T21:34:50.054 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48840.log.gz 2024-09-15T21:34:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43103.log 2024-09-15T21:34:50.054 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36564.log.gz 2024-09-15T21:34:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37278.log 2024-09-15T21:34:50.055 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68267.log.gz 2024-09-15T21:34:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59778.log 2024-09-15T21:34:50.055 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43103.log.gz 2024-09-15T21:34:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52620.log 2024-09-15T21:34:50.055 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37278.log.gz 2024-09-15T21:34:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37093.log 2024-09-15T21:34:50.055 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59778.log.gz 2024-09-15T21:34:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41741.log 2024-09-15T21:34:50.055 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52620.log.gz 2024-09-15T21:34:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37093.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37093.log.gz -5 2024-09-15T21:34:50.055 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-client.admin.63478.log 2024-09-15T21:34:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63856.log 2024-09-15T21:34:50.056 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41741.log.gz 2024-09-15T21:34:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51486.log 2024-09-15T21:34:50.056 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63478.log.gz 2024-09-15T21:34:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63730.log 2024-09-15T21:34:50.056 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63856.log.gz 2024-09-15T21:34:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49596.log 2024-09-15T21:34:50.056 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51486.log.gz 2024-09-15T21:34:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35328.log 2024-09-15T21:34:50.056 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63730.log.gz 2024-09-15T21:34:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49596.log.gzgzip 2024-09-15T21:34:50.057 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68939.log 2024-09-15T21:34:50.057 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54972.log 2024-09-15T21:34:50.057 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35328.log.gz 2024-09-15T21:34:50.057 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68939.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.68939.log.gz 2024-09-15T21:34:50.057 INFO:teuthology.orchestra.run.smithi073.stderr: /var/log/ceph/ceph-client.admin.48168.log 2024-09-15T21:34:50.057 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38731.log 2024-09-15T21:34:50.057 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54972.log.gz 2024-09-15T21:34:50.057 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52746.log 2024-09-15T21:34:50.057 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48168.log.gz 2024-09-15T21:34:50.057 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58434.log 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38731.log.gz 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52746.log.gz 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47244.log 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36648.log 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58434.log.gz 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54594.log 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47244.log.gz 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61754.log 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36648.log.gz 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63142.log 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54594.log.gz 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40447.log 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61754.log.gz 2024-09-15T21:34:50.058 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63142.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.63142.log.gz --verbose -- 2024-09-15T21:34:50.059 INFO:teuthology.orchestra.run.smithi073.stderr: /var/log/ceph/ceph-client.admin.62174.log 2024-09-15T21:34:50.059 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46866.log 2024-09-15T21:34:50.059 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40447.log.gz 2024-09-15T21:34:50.059 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35235.log 2024-09-15T21:34:50.059 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62174.log.gz 2024-09-15T21:34:50.059 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63226.log 2024-09-15T21:34:50.059 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46866.log.gz 2024-09-15T21:34:50.059 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68393.log 2024-09-15T21:34:50.059 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35235.log.gz 2024-09-15T21:34:50.059 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63226.log.gz 2024-09-15T21:34:50.059 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53082.log 2024-09-15T21:34:50.059 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41805.log 2024-09-15T21:34:50.059 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68393.log.gz 2024-09-15T21:34:50.060 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54930.log 2024-09-15T21:34:50.060 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53082.log.gz 2024-09-15T21:34:50.060 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52116.log 2024-09-15T21:34:50.060 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41805.log.gz 2024-09-15T21:34:50.060 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66628.log 2024-09-15T21:34:50.060 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54930.log.gz 2024-09-15T21:34:50.060 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59862.log 2024-09-15T21:34:50.060 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52116.log.gz 2024-09-15T21:34:50.060 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51360.log 2024-09-15T21:34:50.060 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66628.log.gz 2024-09-15T21:34:50.060 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63688.log 2024-09-15T21:34:50.060 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59862.log.gz 2024-09-15T21:34:50.060 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.51360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54426.log 2024-09-15T21:34:50.061 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51360.log.gz 2024-09-15T21:34:50.061 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62806.log 2024-09-15T21:34:50.061 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63688.log.gz 2024-09-15T21:34:50.061 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37130.log 2024-09-15T21:34:50.061 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54426.log.gz 2024-09-15T21:34:50.061 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61418.log 2024-09-15T21:34:50.061 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62806.log.gz 2024-09-15T21:34:50.061 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52158.log 2024-09-15T21:34:50.061 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37130.log.gz 2024-09-15T21:34:50.061 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32251.log 2024-09-15T21:34:50.061 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61418.log.gz 2024-09-15T21:34:50.061 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53964.log 2024-09-15T21:34:50.062 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52158.log.gz 2024-09-15T21:34:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61838.log 2024-09-15T21:34:50.062 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32251.log.gz 2024-09-15T21:34:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48252.log 2024-09-15T21:34:50.062 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53964.log.gz 2024-09-15T21:34:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60114.log 2024-09-15T21:34:50.062 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61838.log.gz 2024-09-15T21:34:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48252.log.gz 2024-09-15T21:34:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52872.log 2024-09-15T21:34:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38250.log 2024-09-15T21:34:50.062 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60114.log.gz 2024-09-15T21:34:50.062 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.52872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64948.log 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52872.log.gz 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57884.log 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38250.log.gz 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63562.log 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64948.log.gz 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46362.log 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57884.log.gz 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36880.log 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63562.log.gz 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43901.log 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46362.log.gz 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53292.log 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36880.log.gz 2024-09-15T21:34:50.063 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50814.log 2024-09-15T21:34:50.064 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43901.log.gz 2024-09-15T21:34:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57452.log 2024-09-15T21:34:50.064 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53292.log.gz 2024-09-15T21:34:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.50814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46194.log 2024-09-15T21:34:50.064 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50814.log.gz 2024-09-15T21:34:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.57452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70074.log 2024-09-15T21:34:50.064 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57452.log.gz 2024-09-15T21:34:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55770.log 2024-09-15T21:34:50.064 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46194.log.gz 2024-09-15T21:34:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55518.log 2024-09-15T21:34:50.064 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70074.log.gz 2024-09-15T21:34:50.064 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60282.log 2024-09-15T21:34:50.064 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55770.log.gz 2024-09-15T21:34:50.065 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44832.log 2024-09-15T21:34:50.065 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55518.log.gz 2024-09-15T21:34:50.065 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59904.log 2024-09-15T21:34:50.065 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60282.log.gz 2024-09-15T21:34:50.065 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.44832.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44832.log.gz 2024-09-15T21:34:50.065 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38509.log 2024-09-15T21:34:50.065 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70788.log 2024-09-15T21:34:50.065 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59904.log.gz 2024-09-15T21:34:50.065 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43795.log 2024-09-15T21:34:50.065 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38509.log.gz 2024-09-15T21:34:50.065 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49176.log 2024-09-15T21:34:50.065 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70788.log.gz 2024-09-15T21:34:50.065 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.43795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37389.log 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43795.log.gz 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.49176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66166.log 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49176.log.gz 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59484.log 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37389.log.gz 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67300.log 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66166.log.gz 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65956.log 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59484.log.gz 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59232.log 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67300.log.gz 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65494.log 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65956.log.gz 2024-09-15T21:34:50.066 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59232.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.59232.log.gz 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56022.log 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41889.log 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65494.log.gz 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.56022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64276.log 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56022.log.gz 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.41889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68435.log 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41889.log.gz 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67889.log 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64276.log.gz 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59736.log 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68435.log.gz 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62848.log 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67889.log.gz 2024-09-15T21:34:50.067 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47622.log 2024-09-15T21:34:50.068 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59736.log.gz 2024-09-15T21:34:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65284.log 2024-09-15T21:34:50.068 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62848.log.gz 2024-09-15T21:34:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.47622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65998.log 2024-09-15T21:34:50.068 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47622.log.gz 2024-09-15T21:34:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60534.log 2024-09-15T21:34:50.068 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65284.log.gz 2024-09-15T21:34:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65830.log 2024-09-15T21:34:50.068 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65998.log.gz 2024-09-15T21:34:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64150.log 2024-09-15T21:34:50.068 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60534.log.gz 2024-09-15T21:34:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.65830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68561.log 2024-09-15T21:34:50.069 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65830.log.gz 2024-09-15T21:34:50.069 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55980.log 2024-09-15T21:34:50.069 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64150.log.gz 2024-09-15T21:34:50.069 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.68561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42347.log 2024-09-15T21:34:50.069 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68561.log.gz 2024-09-15T21:34:50.069 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.55980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61586.log 2024-09-15T21:34:50.069 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55980.log.gz 2024-09-15T21:34:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.42347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67721.log 2024-09-15T21:34:50.070 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42347.log.gz 2024-09-15T21:34:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.61586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54216.log 2024-09-15T21:34:50.070 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61586.log.gz 2024-09-15T21:34:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.67721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66418.log 2024-09-15T21:34:50.070 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67721.log.gz 2024-09-15T21:34:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46656.log 2024-09-15T21:34:50.070 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54216.log.gz 2024-09-15T21:34:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.66418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70158.log 2024-09-15T21:34:50.070 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66418.log.gz 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.46656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38213.log 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46656.log.gz 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63604.log 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70158.log.gz 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63898.log 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38213.log.gz 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64318.log 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63604.log.gz 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.63898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38472.log 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63898.log.gz 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40363.log 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.64318.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64318.log.gz 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37870.log 2024-09-15T21:34:50.071 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38472.log.gz 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.40363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40363.log.gz 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38842.log 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62006.log 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37870.log.gz 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54678.log 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38842.log.gz 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54048.log 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.62006.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62006.log.gz 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58053.log 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54678.log.gz 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.54048.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54048.log.gz 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59148.log 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.58053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70956.log 2024-09-15T21:34:50.072 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58053.log.gz 2024-09-15T21:34:50.073 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.59148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60156.log 2024-09-15T21:34:50.073 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59148.log.gz 2024-09-15T21:34:50.073 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.70956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48042.log 2024-09-15T21:34:50.073 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70956.log.gz 2024-09-15T21:34:50.073 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53250.log 2024-09-15T21:34:50.073 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.60156.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60156.log.gz 2024-09-15T21:34:50.073 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.48042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36395.log 2024-09-15T21:34:50.073 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48042.log.gz 2024-09-15T21:34:50.073 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.53250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53250.log.gz 2024-09-15T21:34:50.073 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36395.log.gz 2024-09-15T21:34:50.664 INFO:teuthology.orchestra.run.smithi073.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-15T21:34:50.812 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-15T21:34:51.348 INFO:teuthology.orchestra.run.smithi073.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-15T21:34:51.416 INFO:teuthology.orchestra.run.smithi073.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-15T21:34:53.744 INFO:teuthology.orchestra.run.smithi073.stderr: 89.9% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-15T21:42:23.769 INFO:teuthology.orchestra.run.smithi073.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-15T21:46:58.783 INFO:teuthology.orchestra.run.smithi073.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-15T21:48:15.613 INFO:teuthology.orchestra.run.smithi073.stderr: 91.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-15T21:48:15.614 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-09-15T21:48:15.615 INFO:teuthology.orchestra.run.smithi073.stderr:real 13m25.986s 2024-09-15T21:48:15.615 INFO:teuthology.orchestra.run.smithi073.stderr:user 5m52.299s 2024-09-15T21:48:15.615 INFO:teuthology.orchestra.run.smithi073.stderr:sys 0m15.877s 2024-09-15T21:48:15.615 INFO:tasks.ceph:Archiving logs... 2024-09-15T21:48:15.615 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905683/remote/smithi073/log 2024-09-15T21:48:15.616 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-15T21:49:20.517 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-15T21:49:20.543 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-15T21:49:20.544 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-15T21:49:20.631 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-15T21:49:20.640 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-15T21:49:20.640 DEBUG:teuthology.orchestra.run.smithi073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-15T21:49:20.658 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-09-15T21:49:20.693 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-15T21:49:20.693 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-09-15T21:49:20.694 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv01.front.sepia.ceph.com 3 6 377 33 +1262us[+1266us] +/- 50ms 2024-09-15T21:49:20.694 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 33 -509us[ -509us] +/- 60ms 2024-09-15T21:49:20.694 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 -1259us[-1259us] +/- 70ms 2024-09-15T21:49:20.694 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-15T21:49:20.695 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-15T21:49:20.765 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-15T21:49:20.765 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-15T21:49:21.011 DEBUG:teuthology.orchestra.run.smithi073:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-15T21:49:21.166 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-15T21:49:21.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:49:21.197 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-15T21:49:21.206 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-15T21:49:21.339 INFO:teuthology.task.internal:Duration was 2203.469418 seconds 2024-09-15T21:49:21.339 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-15T21:49:21.410 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-15T21:49:21.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-15T21:49:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-15T21:49:21.878 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-15T21:49:21.878 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi073.front.sepia.ceph.com 2024-09-15T21:49:21.878 DEBUG:teuthology.orchestra.run.smithi073:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-15T21:49:21.901 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-15T21:49:21.901 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-15T21:49:21.970 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-15T21:49:21.971 DEBUG:teuthology.orchestra.run.smithi073:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-15T21:49:22.148 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-15T21:49:22.158 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-15T21:49:22.158 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-15T21:49:22.191 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-15T21:49:22.251 DEBUG:teuthology.orchestra.run.smithi073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:49:22.289 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = core 2024-09-15T21:49:22.311 DEBUG:teuthology.orchestra.run.smithi073:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-15T21:49:22.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-15T21:49:22.366 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-15T21:49:22.376 INFO:teuthology.task.internal:Transferring archived files... 2024-09-15T21:49:22.377 DEBUG:teuthology.misc:Transferring archived files from smithi073:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_20:00:16-rados-main-distro-default-smithi/7905683/remote/smithi073 2024-09-15T21:49:22.378 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-15T21:49:22.442 INFO:teuthology.task.internal:Removing archive directory... 2024-09-15T21:49:22.442 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-15T21:49:22.491 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-15T21:49:22.502 INFO:teuthology.task.internal:Not uploading archives. 2024-09-15T21:49:22.502 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-15T21:49:22.535 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-15T21:49:22.535 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-15T21:49:22.551 INFO:teuthology.orchestra.run.smithi073.stdout: 265336 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 15 21:49 /home/ubuntu/cephtest 2024-09-15T21:49:22.552 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-15T21:49:22.685 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-15T21:49:22.882 INFO:teuthology.run:Summary data: description: rados/singleton/{all/lost-unfound mon_election/classic msgr-failures/none msgr/async objectstore/{bluestore-options/write$/{write_random} bluestore/bluestore-comp-zstd} rados supported-random-distro$/{centos_latest}} duration: 2203.469418287277 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-15T21:49:22.882 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-15T21:49:22.969 INFO:teuthology.run:pass